Backlog/lib/sdl3/parser/DEPENDENCY_IMPLEMENTATION_PLAN.md

Dependency Resolution Implementation Plan (v2)

Core Insight

Single-file generation with on-demand type resolution: Parse the primary header completely, identify missing types, then parse included headers ONLY for those specific types. Append them to the same output file. Zig's structural typing handles everything else.

Why This Works

1. No modules needed - One output file with all types
2. Zig deduplicates - Multiple parsings of same type are safe
3. Forward references work - Zig allows using types before they're defined
4. Simple implementation - Just append to existing output

Three-Phase Algorithm

Phase 1: Primary Header Parsing (Existing)

Input: SDL_gpu.h
Output: declarations[], generated_code

Phase 2: Missing Type Detection (NEW)

1. Scan generated_code for all type references
2. Build set of defined_types from declarations[]
3. missing_types = referenced_types - defined_types

Phase 3: Dependency Resolution (NEW)

For each missing_type:
  1. Parse each included header
  2. If type found, extract declaration
  3. Append to output

Implementation Details

Implementation Details

Component 1: Type Reference Scanner

Purpose: Find all type names used in generated code

Location: New file src/dependency_resolver.zig

pub const TypeReference = struct {
    name: []const u8,
    source_location: []const u8, // For debugging
};

pub fn scanTypeReferences(decls: []const Declaration) ![]TypeReference {
    var refs = ArrayList(TypeReference).init(allocator);
    
    for (decls) |decl| {
        switch (decl) {
            .function_decl => |func| {
                // Scan return type
                try scanType(func.return_type, &refs);
                // Scan parameters
                for (func.params) |param| {
                    try scanType(param.type_name, &refs);
                }
            },
            .struct_decl => |struct_decl| {
                // Scan field types
                for (struct_decl.fields) |field| {
                    try scanType(field.type_name, &refs);
                }
            },
            // opaque/enum don't reference other types
            else => {},
        }
    }
    
    return refs.toOwnedSlice();
}

fn scanType(type_str: []const u8, refs: *ArrayList(TypeReference)) !void {
    // Extract base type from "?*const SDL_Type" → "SDL_Type"
    const base_type = extractBaseType(type_str);
    if (isSDLType(base_type)) {
        try refs.append(.{ .name = base_type, .source_location = type_str });
    }
}

Key functions:

• extractBaseType(): Strip pointers, const, optional from type string
• isSDLType(): Check if starts with "SDL_" or is known SDL type
• Handle edge cases: arrays, function pointers (skip for now)

Component 2: Defined Type Collector

Purpose: Track what types are already defined

pub fn collectDefinedTypes(decls: []const Declaration) StringHashMap(void) {
    var defined = StringHashMap(void).init(allocator);
    
    for (decls) |decl| {
        const type_name = switch (decl) {
            .opaque_type => |o| o.name,
            .enum_decl => |e| e.name,
            .struct_decl => |s| s.name,
            .flags_decl => |f| f.name,
            .function_decl => continue, // Functions don't define types
        };
        try defined.put(type_name, {});
    }
    
    return defined;
}

Component 3: Include Header Parser

Purpose: Extract #include directives

pub fn parseIncludes(source: []const u8) ![]const []const u8 {
    var includes = ArrayList([]const u8).init(allocator);
    
    var lines = std.mem.split(u8, source, "\n");
    while (lines.next()) |line| {
        // Match: #include <SDL3/SDL_something.h>
        if (std.mem.indexOf(u8, line, "#include <SDL3/")) |start| {
            const after_open = start + "#include <SDL3/".len;
            if (std.mem.indexOf(u8, line[after_open..], ">")) |end| {
                const header_name = line[after_open..][0..end];
                try includes.append(try allocator.dupe(u8, header_name));
            }
        }
    }
    
    return includes.toOwnedSlice();
}

Component 4: Selective Type Extractor

Purpose: Find specific type in a header

pub fn extractTypeFromHeader(
    allocator: Allocator,
    header_source: []const u8,
    type_name: []const u8, // e.g., "SDL_Rect"
) !?Declaration {
    // Parse the header
    var scanner = Scanner.init(allocator, header_source);
    const all_decls = try scanner.scan();
    defer scanner.deinit();
    
    // Find matching declaration
    for (all_decls) |decl| {
        const decl_name = switch (decl) {
            .opaque_type => |o| o.name,
            .enum_decl => |e| e.name,
            .struct_decl => |s| s.name,
            .flags_decl => |f| f.name,
            else => continue,
        };
        
        if (std.mem.eql(u8, decl_name, type_name)) {
            return try decl.clone(allocator); // Deep copy
        }
    }
    
    return null; // Not found
}

Component 5: Main Integration

Location: Modify src/parser.zig main function

pub fn main() !void {
    // ... existing setup ...
    
    // 1. Parse primary header (existing)
    var scanner = Scanner.init(allocator, source);
    const primary_decls = try scanner.scan();
    
    // 2. Identify missing types (NEW)
    const references = try scanTypeReferences(primary_decls);
    const defined = collectDefinedTypes(primary_decls);
    
    var missing = ArrayList([]const u8).init(allocator);
    for (references) |ref| {
        if (!defined.contains(ref.name)) {
            try missing.append(ref.name);
        }
    }
    
    // 3. Extract missing types from dependencies (NEW)
    var dependency_decls = ArrayList(Declaration).init(allocator);
    
    if (missing.items.len > 0) {
        const includes = try parseIncludes(source);
        const header_dir = std.fs.path.dirname(header_path) orelse ".";
        
        for (missing.items) |missing_type| {
            var found = false;
            for (includes) |include| {
                const dep_path = try std.fs.path.join(
                    allocator,
                    &[_][]const u8{ header_dir, include }
                );
                defer allocator.free(dep_path);
                
                const dep_source = std.fs.cwd().readFileAlloc(
                    allocator,
                    dep_path,
                    10 * 1024 * 1024
                ) catch continue;
                defer allocator.free(dep_source);
                
                if (try extractTypeFromHeader(allocator, dep_source, missing_type)) |decl| {
                    try dependency_decls.append(decl);
                    found = true;
                    break;
                }
            }
            
            if (!found) {
                std.debug.print(
                    "Warning: Could not find definition for type: {s}\n",
                    .{missing_type}
                );
            }
        }
    }
    
    // 4. Combine declarations
    var all_decls = ArrayList(Declaration).init(allocator);
    try all_decls.appendSlice(dependency_decls.items); // Dependencies first!
    try all_decls.appendSlice(primary_decls);
    
    // 5. Generate code (existing, but with all declarations)
    const output = try codegen.generate(allocator, all_decls.items);
    
    // ... rest of existing code ...
}

Key Implementation Decisions

1. Dependencies go FIRST in output

   • Ensures types are defined before use
   • More logical reading order

2. Deep copy declarations

   • Avoid lifetime issues with parsed headers
   • Each declaration owns its strings

3. Warning for missing types

   • Don't fail build, just warn
   • Allows gradual improvement

4. Skip function pointers/unions

   • Add TODO comments
   • Focus on common cases first

5. Cache parsed headers

   • Parse each dependency header once
   • Extract multiple types from same parse

Testing Approach

Unit Tests

test "scanTypeReferences finds SDL types" {
    const decls = [_]Declaration{
        .{ .function_decl = .{
            .name = "test",
            .return_type = "?*SDL_Window",
            .params = &[_]Param{
                .{ .name = "rect", .type_name = "*const SDL_Rect" },
            },
        }},
    };
    
    const refs = try scanTypeReferences(&decls);
    try testing.expect(refs.len == 2);
    try testing.expectEqualStrings("SDL_Window", refs[0].name);
    try testing.expectEqualStrings("SDL_Rect", refs[1].name);
}

test "collectDefinedTypes tracks declarations" {
    const decls = [_]Declaration{
        .{ .opaque_type = .{ .name = "SDL_Device" }},
        .{ .struct_decl = .{ .name = "SDL_Info", .fields = &[_]Field{} }},
    };
    
    const defined = collectDefinedTypes(&decls);
    try testing.expect(defined.contains("SDL_Device"));
    try testing.expect(defined.contains("SDL_Info"));
}

Integration Test

test "full dependency resolution with SDL_gpu.h" {
    const source = try std.fs.cwd().readFileAlloc(
        testing.allocator,
        "SDL/include/SDL3/SDL_gpu.h",
        10 * 1024 * 1024
    );
    defer testing.allocator.free(source);
    
    // Parse and resolve
    const output = try parseWithDependencies(testing.allocator, source, "SDL/include/SDL3");
    defer testing.allocator.free(output);
    
    // Verify missing types are present
    try testing.expect(std.mem.indexOf(u8, output, "pub const Window = opaque") != null);
    try testing.expect(std.mem.indexOf(u8, output, "pub const Rect = extern struct") != null);
    try testing.expect(std.mem.indexOf(u8, output, "pub const FColor = extern struct") != null);
    
    // Verify it compiles
    var ast = try std.zig.Ast.parse(testing.allocator, output, .zig);
    defer ast.deinit(testing.allocator);
    try testing.expect(ast.errors.len == 0);
}

Validation Steps

1. Remove manual definitions from mock_test.zig:

   - pub const Window = opaque {};
   - pub const Rect = extern struct { ... };
   - pub const FColor = extern struct { ... };
   

2. Import generated file directly:

   const gpu = @import("../../zig-out/gpu_test.zig");
   

3. Run tests:

   zig build test-mocks  # Should still pass!
   

Success Metrics

✅ scanTypeReferences finds 30+ type references in SDL_gpu.h ✅ collectDefinedTypes tracks 169 defined types ✅ Missing types: Window, Rect, FColor, FlipMode, PropertiesID detected ✅ All 5 missing types extracted from dependency headers ✅ Generated code compiles without manual definitions ✅ All 11 tests pass ✅ Build time increase < 1 second

Rollout Plan

1. Day 1: Implement Components 1-2 (type scanning/collecting)
2. Day 2: Implement Components 3-4 (include parsing/type extraction)
3. Day 3: Integrate into main, test with SDL_gpu.h
4. Day 4: Refine, handle edge cases, update tests
5. Day 5: Documentation, final validation

Risks & Mitigation

Risk	Mitigation
Can't find header files	Require header directory as input
Type not in any header	Emit warning, generate placeholder
Parsing dependency fails	Catch error, continue with other headers
Performance (parsing multiple headers)	Cache parsed headers, parse once
Circular dependencies	Not an issue - all types in one file

---

This plan is ready to implement. Each component is well-defined with clear inputs/outputs, error handling, and test cases.

File: src/type_collector.zig

const TypeCollector = struct {
    defined_types: StringHashMap(void),  // Types defined in primary header
    referenced_types: StringHashMap(void), // Types used in signatures
    
    pub fn collectFromDeclarations(decls: []Declaration) TypeCollector;
    pub fn getMissingTypes() []const []const u8;
};

Tasks:

• Scan all declarations for type definitions (opaque, enum, struct, flags)
• Scan all function signatures for type references
• Return set difference: referenced - defined

Phase 2: Include Directive Parsing (1 hour)

File: src/patterns.zig (extend existing)

pub fn parseIncludes(source: []const u8) ![]const []const u8 {
    // Find all #include <SDL3/header.h> directives
    // Return list of header filenames
}

Tasks:

• Add regex/pattern for #include <SDL3/...>
• Extract header filename from directive
• Return list of included headers

Phase 3: Selective Type Extraction (2-3 hours)

File: src/type_extractor.zig

pub fn extractType(
    allocator: Allocator,
    header_source: []const u8,
    type_name: []const u8,  // e.g., "SDL_Rect"
) !?Declaration {
    // Parse header looking for specific type
    // Return the declaration if found
}

pub fn extractTypes(
    allocator: Allocator,
    header_paths: []const []const u8,
    missing_types: []const []const u8,
) ![]Declaration {
    // For each missing type:
    //   For each header:
    //     Try to extract the type
    //     If found, add to results
    // Return all found declarations
}

Tasks:

• Reuse existing Scanner but filter by type name
• Handle opaque types: typedef struct SDL_Type SDL_Type;
• Handle structs: typedef struct { ... } SDL_Type;
• Handle enums: typedef enum { ... } SDL_Type;
• Handle simple typedefs: typedef uint32_t SDL_Type;

Phase 4: Integration (1-2 hours)

File: src/parser.zig (modify main function)

pub fn main() !void {
    // 1. Parse primary header
    var scanner = Scanner.init(allocator, source);
    const decls = try scanner.scan();
    
    // 2. Collect missing types
    const collector = TypeCollector.collectFromDeclarations(decls);
    const missing_types = try collector.getMissingTypes(allocator);
    
    // 3. Parse included headers for missing types
    const includes = try parseIncludes(source);
    const header_dir = getHeaderDirectory(header_path);
    const dependency_decls = try extractTypes(
        allocator,
        header_dir,
        includes,
        missing_types
    );
    
    // 4. Generate code with dependencies appended
    var all_decls = std.ArrayList(Declaration).init(allocator);
    try all_decls.appendSlice(decls);
    try all_decls.appendSlice(dependency_decls);
    
    const output = try codegen.generate(allocator, all_decls.items);
    
    // 5. Write output
    try writeOutput(output_file, output);
}

Tasks:

• Wire together all components
• Handle header path resolution
• Add comment separators for dependencies
• Update error handling

Phase 5: Code Generation Enhancement (1 hour)

File: src/codegen.zig (modify)

Add dependency section:

fn generate() ![]const u8 {
    try output.appendSlice("pub const c = @import(\"c.zig\").c;\n\n");
    
    // Add comment if we have dependencies
    if (has_dependency_decls) {
        try output.appendSlice(
            \\// Dependencies from included headers
            \\// These types are referenced by the primary header
            \\
        );
    }
    
    // Generate all declarations (primary + dependencies)
    for (decls) |decl| {
        try generateDeclaration(decl);
    }
}

Tasks:

• Add dependency comment section
• Mark which declarations are dependencies (optional)
• Ensure proper ordering (dependencies before usage)

Example Output

pub const c = @import("c.zig").c;

// Dependencies from included headers
// These types are referenced by SDL_gpu.h

// From SDL_rect.h
pub const Rect = extern struct {
    x: i32,
    y: i32,
    w: i32,
    h: i32,
};

// From SDL_pixels.h
pub const FColor = extern struct {
    r: f32,
    g: f32,
    b: f32,
    a: f32,
};

// From SDL_video.h
pub const Window = opaque {};

// From SDL_properties.h
pub const PropertiesID = u32;

// SDL_gpu.h declarations
pub const GPUDevice = opaque {
    pub inline fn windowSupportsGPUSwapchainComposition(
        gpudevice: *GPUDevice,
        window: ?*Window,  // ✅ Now defined!
        swapchain_composition: GPUSwapchainComposition
    ) bool { ... }
};

pub const GPURenderPass = opaque {
    pub inline fn setGPUScissor(
        gpurenderpass: *GPURenderPass,
        scissor: *const Rect  // ✅ Now defined!
    ) void { ... }
};

Edge Cases

1. Type not found in any header

   • Emit warning
   • Generate placeholder: pub const TypeName = opaque {};

2. Circular dependencies

   • Not an issue - all types in one file
   • Zig allows forward references

3. Multiple definitions

   • Keep first definition found
   • Zig will error if layouts differ (good!)

4. Typedef chains

   • typedef SDL_Type1 Type2;
   • Resolve transitively or use Zig's type alias

5. Complex types

   • Function pointers: Skip for now, add TODO comment
   • Unions: Skip for now, add TODO comment
   • Nested structs: Should work fine

Testing Strategy

1. Unit tests for each component:

   • TypeCollector: Test with known declarations
   • parseIncludes: Test with sample headers
   • extractType: Test finding types in headers

2. Integration test:

   • Parse SDL_gpu.h
   • Verify missing types are detected
   • Verify dependencies are extracted
   • Verify output compiles

3. Validation:

   • Remove manual type definitions from mock_test.zig
   • Import actual generated file
   • All tests should still pass

Success Criteria

✅ Parser detects 5 missing types from SDL_gpu.h ✅ Parser extracts types from dependency headers ✅ Generated code compiles standalone ✅ All tests pass without manual type definitions ✅ Single output file contains all needed types

Time Estimate

• Phase 1 (Type Collection): 1-2 hours
• Phase 2 (Include Parsing): 1 hour
• Phase 3 (Type Extraction): 2-3 hours
• Phase 4 (Integration): 1-2 hours
• Phase 5 (Code Gen Enhancement): 1 hour
• Testing & Refinement: 2 hours

Total: 8-11 hours

Advantages of This Approach

1. Simple: Single output file, no module management
2. Fast: Only parse dependency headers when needed
3. Minimal: Only extract required types
4. Robust: Zig handles duplicate definitions
5. Maintainable: Clear separation in output

Open Questions

1. Should we cache parsed dependency headers?

   • Answer: Yes, parse once, extract many types

2. How to handle nested dependencies (Type A needs Type B)?

   • Answer: Recursive extraction, track visited types

3. Should dependencies go at top or bottom of file?

   • Answer: Top, before primary declarations use them

4. What about #define constants?

   • Answer: Skip for now, out of scope

Next Steps

1. Implement TypeCollector
2. Implement include parsing
3. Implement type extraction
4. Wire together in main
5. Test with SDL_gpu.h
6. Update documentation

---

Ready to implement? This plan provides:

• Clear phases with time estimates
• Concrete code examples
• Handles edge cases
• Simple single-file output
• Full testing strategy