Add comprehensive dependency resolution implementation plan

Plan for on-demand type resolution: - Single-file output with dependencies appended - Parse included headers only for missing types - Three-phase algorithm: parse, detect, resolve - Five components with clear interfaces and tests Key features: - Type reference scanner (finds all type usage) - Defined type collector (tracks what exists) - Include header parser (extracts #include directives) - Selective type extractor (finds specific types) - Main integration (wires everything together) Advantages: - Simple: one output file, no modules - Fast: parse dependencies once, extract many types - Minimal: only extract required types - Robust: Zig handles duplicate definitions - Testable: each component independently tested Ready to implement with: - Detailed code examples for each component - Complete test strategy - Success metrics - 5-day rollout plan - Risk mitigation
2026-01-22 01:46:58 -08:00 · 2026-01-22 01:46:58 -08:00 · c7c7440c14
parent 4c354f093c
commit c7c7440c14
1 changed files with 667 additions and 0 deletions
--- a/lib/sdl3/parser/DEPENDENCY_IMPLEMENTATION_PLAN.md
+++ b/lib/sdl3/parser/DEPENDENCY_IMPLEMENTATION_PLAN.md
@ -0,0 +1,667 @@
 # Dependency Resolution Implementation Plan (v2)
 ## Core Insight
 **Single-file generation with on-demand type resolution**: Parse the primary header completely, identify missing types, then parse included headers ONLY for those specific types. Append them to the same output file. Zig's structural typing handles everything else.
 ## Why This Works
 1. **No modules needed** - One output file with all types
 2. **Zig deduplicates** - Multiple parsings of same type are safe
 3. **Forward references work** - Zig allows using types before they're defined
 4. **Simple implementation** - Just append to existing output
 ## Three-Phase Algorithm
 ### Phase 1: Primary Header Parsing (Existing)
 ```
 Input: SDL_gpu.h
 Output: declarations[], generated_code
 ```
 ### Phase 2: Missing Type Detection (NEW)
 ```
 1. Scan generated_code for all type references
 2. Build set of defined_types from declarations[]
 3. missing_types = referenced_types - defined_types
 ```
 ### Phase 3: Dependency Resolution (NEW)
 ```
 For each missing_type:
  1. Parse each included header
  2. If type found, extract declaration
  3. Append to output
 ```
 ## Implementation Details
 ## Implementation Details
 ### Component 1: Type Reference Scanner
 **Purpose**: Find all type names used in generated code
 **Location**: New file `src/dependency_resolver.zig`
 ```zig
 pub const TypeReference = struct {
    name: []const u8,
    source_location: []const u8, // For debugging
 };
 pub fn scanTypeReferences(decls: []const Declaration) ![]TypeReference {
    var refs = ArrayList(TypeReference).init(allocator);
    for (decls) |decl| {
        switch (decl) {
            .function_decl => |func| {
                // Scan return type
                try scanType(func.return_type, &refs);
                // Scan parameters
                for (func.params) |param| {
                    try scanType(param.type_name, &refs);
                }
            },
            .struct_decl => |struct_decl| {
                // Scan field types
                for (struct_decl.fields) |field| {
                    try scanType(field.type_name, &refs);
                }
            },
            // opaque/enum don't reference other types
            else => {},
        }
    }
    return refs.toOwnedSlice();
 }
 fn scanType(type_str: []const u8, refs: *ArrayList(TypeReference)) !void {
    // Extract base type from "?*const SDL_Type" → "SDL_Type"
    const base_type = extractBaseType(type_str);
    if (isSDLType(base_type)) {
        try refs.append(.{ .name = base_type, .source_location = type_str });
    }
 }
 ```
 **Key functions**:
 - `extractBaseType()`: Strip pointers, const, optional from type string
 - `isSDLType()`: Check if starts with "SDL_" or is known SDL type
 - Handle edge cases: arrays, function pointers (skip for now)
 ### Component 2: Defined Type Collector
 **Purpose**: Track what types are already defined
 ```zig
 pub fn collectDefinedTypes(decls: []const Declaration) StringHashMap(void) {
    var defined = StringHashMap(void).init(allocator);
    for (decls) |decl| {
        const type_name = switch (decl) {
            .opaque_type => |o| o.name,
            .enum_decl => |e| e.name,
            .struct_decl => |s| s.name,
            .flags_decl => |f| f.name,
            .function_decl => continue, // Functions don't define types
        };
        try defined.put(type_name, {});
    }
    return defined;
 }
 ```
 ### Component 3: Include Header Parser
 **Purpose**: Extract #include directives
 ```zig
 pub fn parseIncludes(source: []const u8) ![]const []const u8 {
    var includes = ArrayList([]const u8).init(allocator);
    var lines = std.mem.split(u8, source, "\n");
    while (lines.next()) |line| {
        // Match: #include <SDL3/SDL_something.h>
        if (std.mem.indexOf(u8, line, "#include <SDL3/")) |start| {
            const after_open = start + "#include <SDL3/".len;
            if (std.mem.indexOf(u8, line[after_open..], ">")) |end| {
                const header_name = line[after_open..][0..end];
                try includes.append(try allocator.dupe(u8, header_name));
            }
        }
    }
    return includes.toOwnedSlice();
 }
 ```
 ### Component 4: Selective Type Extractor
 **Purpose**: Find specific type in a header
 ```zig
 pub fn extractTypeFromHeader(
    allocator: Allocator,
    header_source: []const u8,
    type_name: []const u8, // e.g., "SDL_Rect"
 ) !?Declaration {
    // Parse the header
    var scanner = Scanner.init(allocator, header_source);
    const all_decls = try scanner.scan();
    defer scanner.deinit();
    // Find matching declaration
    for (all_decls) |decl| {
        const decl_name = switch (decl) {
            .opaque_type => |o| o.name,
            .enum_decl => |e| e.name,
            .struct_decl => |s| s.name,
            .flags_decl => |f| f.name,
            else => continue,
        };
        if (std.mem.eql(u8, decl_name, type_name)) {
            return try decl.clone(allocator); // Deep copy
        }
    }
    return null; // Not found
 }
 ```
 ### Component 5: Main Integration
 **Location**: Modify `src/parser.zig` main function
 ```zig
 pub fn main() !void {
    // ... existing setup ...
    // 1. Parse primary header (existing)
    var scanner = Scanner.init(allocator, source);
    const primary_decls = try scanner.scan();
    // 2. Identify missing types (NEW)
    const references = try scanTypeReferences(primary_decls);
    const defined = collectDefinedTypes(primary_decls);
    var missing = ArrayList([]const u8).init(allocator);
    for (references) |ref| {
        if (!defined.contains(ref.name)) {
            try missing.append(ref.name);
        }
    }
    // 3. Extract missing types from dependencies (NEW)
    var dependency_decls = ArrayList(Declaration).init(allocator);
    if (missing.items.len > 0) {
        const includes = try parseIncludes(source);
        const header_dir = std.fs.path.dirname(header_path) orelse ".";
        for (missing.items) |missing_type| {
            var found = false;
            for (includes) |include| {
                const dep_path = try std.fs.path.join(
                    allocator,
                    &[_][]const u8{ header_dir, include }
                );
                defer allocator.free(dep_path);
                const dep_source = std.fs.cwd().readFileAlloc(
                    allocator,
                    dep_path,
                    10 * 1024 * 1024
                ) catch continue;
                defer allocator.free(dep_source);
                if (try extractTypeFromHeader(allocator, dep_source, missing_type)) |decl| {
                    try dependency_decls.append(decl);
                    found = true;
                    break;
                }
            }
            if (!found) {
                std.debug.print(
                    "Warning: Could not find definition for type: {s}\n",
                    .{missing_type}
                );
            }
        }
    }
    // 4. Combine declarations
    var all_decls = ArrayList(Declaration).init(allocator);
    try all_decls.appendSlice(dependency_decls.items); // Dependencies first!
    try all_decls.appendSlice(primary_decls);
    // 5. Generate code (existing, but with all declarations)
    const output = try codegen.generate(allocator, all_decls.items);
    // ... rest of existing code ...
 }
 ```
 ## Key Implementation Decisions
 1. **Dependencies go FIRST** in output
   - Ensures types are defined before use
   - More logical reading order
 2. **Deep copy declarations**
   - Avoid lifetime issues with parsed headers
   - Each declaration owns its strings
 3. **Warning for missing types**
   - Don't fail build, just warn
   - Allows gradual improvement
 4. **Skip function pointers/unions**
   - Add TODO comments
   - Focus on common cases first
 5. **Cache parsed headers**
   - Parse each dependency header once
   - Extract multiple types from same parse
 ## Testing Approach
 ### Unit Tests
 ```zig
 test "scanTypeReferences finds SDL types" {
    const decls = [_]Declaration{
        .{ .function_decl = .{
            .name = "test",
            .return_type = "?*SDL_Window",
            .params = &[_]Param{
                .{ .name = "rect", .type_name = "*const SDL_Rect" },
            },
        }},
    };
    const refs = try scanTypeReferences(&decls);
    try testing.expect(refs.len == 2);
    try testing.expectEqualStrings("SDL_Window", refs[0].name);
    try testing.expectEqualStrings("SDL_Rect", refs[1].name);
 }
 test "collectDefinedTypes tracks declarations" {
    const decls = [_]Declaration{
        .{ .opaque_type = .{ .name = "SDL_Device" }},
        .{ .struct_decl = .{ .name = "SDL_Info", .fields = &[_]Field{} }},
    };
    const defined = collectDefinedTypes(&decls);
    try testing.expect(defined.contains("SDL_Device"));
    try testing.expect(defined.contains("SDL_Info"));
 }
 ```
 ### Integration Test
 ```zig
 test "full dependency resolution with SDL_gpu.h" {
    const source = try std.fs.cwd().readFileAlloc(
        testing.allocator,
        "SDL/include/SDL3/SDL_gpu.h",
        10 * 1024 * 1024
    );
    defer testing.allocator.free(source);
    // Parse and resolve
    const output = try parseWithDependencies(testing.allocator, source, "SDL/include/SDL3");
    defer testing.allocator.free(output);
    // Verify missing types are present
    try testing.expect(std.mem.indexOf(u8, output, "pub const Window = opaque") != null);
    try testing.expect(std.mem.indexOf(u8, output, "pub const Rect = extern struct") != null);
    try testing.expect(std.mem.indexOf(u8, output, "pub const FColor = extern struct") != null);
    // Verify it compiles
    var ast = try std.zig.Ast.parse(testing.allocator, output, .zig);
    defer ast.deinit(testing.allocator);
    try testing.expect(ast.errors.len == 0);
 }
 ```
 ## Validation Steps
 1. **Remove manual definitions** from mock_test.zig:
   ```diff
   - pub const Window = opaque {};
   - pub const Rect = extern struct { ... };
   - pub const FColor = extern struct { ... };
   ```
 2. **Import generated file** directly:
   ```zig
   const gpu = @import("../../zig-out/gpu_test.zig");
   ```
 3. **Run tests**:
   ```bash
   zig build test-mocks  # Should still pass!
   ```
 ## Success Metrics
 ✅ `scanTypeReferences` finds 30+ type references in SDL_gpu.h
 ✅ `collectDefinedTypes` tracks 169 defined types
 ✅ Missing types: Window, Rect, FColor, FlipMode, PropertiesID detected
 ✅ All 5 missing types extracted from dependency headers
 ✅ Generated code compiles without manual definitions
 ✅ All 11 tests pass
 ✅ Build time increase < 1 second
 ## Rollout Plan
 1. **Day 1**: Implement Components 1-2 (type scanning/collecting)
 2. **Day 2**: Implement Components 3-4 (include parsing/type extraction)
 3. **Day 3**: Integrate into main, test with SDL_gpu.h
 4. **Day 4**: Refine, handle edge cases, update tests
 5. **Day 5**: Documentation, final validation
 ## Risks & Mitigation
 | Risk | Mitigation |
 |------|-----------|
 | Can't find header files | Require header directory as input |
 | Type not in any header | Emit warning, generate placeholder |
 | Parsing dependency fails | Catch error, continue with other headers |
 | Performance (parsing multiple headers) | Cache parsed headers, parse once |
 | Circular dependencies | Not an issue - all types in one file |
 ---
 This plan is **ready to implement**. Each component is well-defined with clear inputs/outputs, error handling, and test cases.
 **File**: `src/type_collector.zig`
 ```zig
 const TypeCollector = struct {
    defined_types: StringHashMap(void),  // Types defined in primary header
    referenced_types: StringHashMap(void), // Types used in signatures
    pub fn collectFromDeclarations(decls: []Declaration) TypeCollector;
    pub fn getMissingTypes() []const []const u8;
 };
 ```
 **Tasks**:
 - Scan all declarations for type definitions (opaque, enum, struct, flags)
 - Scan all function signatures for type references
 - Return set difference: referenced - defined
 ### Phase 2: Include Directive Parsing (1 hour)
 **File**: `src/patterns.zig` (extend existing)
 ```zig
 pub fn parseIncludes(source: []const u8) ![]const []const u8 {
    // Find all #include <SDL3/header.h> directives
    // Return list of header filenames
 }
 ```
 **Tasks**:
 - Add regex/pattern for `#include <SDL3/...>`
 - Extract header filename from directive
 - Return list of included headers
 ### Phase 3: Selective Type Extraction (2-3 hours)
 **File**: `src/type_extractor.zig`
 ```zig
 pub fn extractType(
    allocator: Allocator,
    header_source: []const u8,
    type_name: []const u8,  // e.g., "SDL_Rect"
 ) !?Declaration {
    // Parse header looking for specific type
    // Return the declaration if found
 }
 pub fn extractTypes(
    allocator: Allocator,
    header_paths: []const []const u8,
    missing_types: []const []const u8,
 ) ![]Declaration {
    // For each missing type:
    //   For each header:
    //     Try to extract the type
    //     If found, add to results
    // Return all found declarations
 }
 ```
 **Tasks**:
 - Reuse existing Scanner but filter by type name
 - Handle opaque types: `typedef struct SDL_Type SDL_Type;`
 - Handle structs: `typedef struct { ... } SDL_Type;`
 - Handle enums: `typedef enum { ... } SDL_Type;`
 - Handle simple typedefs: `typedef uint32_t SDL_Type;`
 ### Phase 4: Integration (1-2 hours)
 **File**: `src/parser.zig` (modify main function)
 ```zig
 pub fn main() !void {
    // 1. Parse primary header
    var scanner = Scanner.init(allocator, source);
    const decls = try scanner.scan();
    // 2. Collect missing types
    const collector = TypeCollector.collectFromDeclarations(decls);
    const missing_types = try collector.getMissingTypes(allocator);
    // 3. Parse included headers for missing types
    const includes = try parseIncludes(source);
    const header_dir = getHeaderDirectory(header_path);
    const dependency_decls = try extractTypes(
        allocator,
        header_dir,
        includes,
        missing_types
    );
    // 4. Generate code with dependencies appended
    var all_decls = std.ArrayList(Declaration).init(allocator);
    try all_decls.appendSlice(decls);
    try all_decls.appendSlice(dependency_decls);
    const output = try codegen.generate(allocator, all_decls.items);
    // 5. Write output
    try writeOutput(output_file, output);
 }
 ```
 **Tasks**:
 - Wire together all components
 - Handle header path resolution
 - Add comment separators for dependencies
 - Update error handling
 ### Phase 5: Code Generation Enhancement (1 hour)
 **File**: `src/codegen.zig` (modify)
 Add dependency section:
 ```zig
 fn generate() ![]const u8 {
    try output.appendSlice("pub const c = @import(\"c.zig\").c;\n\n");
    // Add comment if we have dependencies
    if (has_dependency_decls) {
        try output.appendSlice(
            \\// Dependencies from included headers
            \\// These types are referenced by the primary header
            \\
        );
    }
    // Generate all declarations (primary + dependencies)
    for (decls) |decl| {
        try generateDeclaration(decl);
    }
 }
 ```
 **Tasks**:
 - Add dependency comment section
 - Mark which declarations are dependencies (optional)
 - Ensure proper ordering (dependencies before usage)
 ## Example Output
 ```zig
 pub const c = @import("c.zig").c;
 // Dependencies from included headers
 // These types are referenced by SDL_gpu.h
 // From SDL_rect.h
 pub const Rect = extern struct {
    x: i32,
    y: i32,
    w: i32,
    h: i32,
 };
 // From SDL_pixels.h
 pub const FColor = extern struct {
    r: f32,
    g: f32,
    b: f32,
    a: f32,
 };
 // From SDL_video.h
 pub const Window = opaque {};
 // From SDL_properties.h
 pub const PropertiesID = u32;
 // SDL_gpu.h declarations
 pub const GPUDevice = opaque {
    pub inline fn windowSupportsGPUSwapchainComposition(
        gpudevice: *GPUDevice,
        window: ?*Window,  // ✅ Now defined!
        swapchain_composition: GPUSwapchainComposition
    ) bool { ... }
 };
 pub const GPURenderPass = opaque {
    pub inline fn setGPUScissor(
        gpurenderpass: *GPURenderPass,
        scissor: *const Rect  // ✅ Now defined!
    ) void { ... }
 };
 ```
 ## Edge Cases
 1. **Type not found in any header**
   - Emit warning
   - Generate placeholder: `pub const TypeName = opaque {};`
 2. **Circular dependencies**
   - Not an issue - all types in one file
   - Zig allows forward references
 3. **Multiple definitions**
   - Keep first definition found
   - Zig will error if layouts differ (good!)
 4. **Typedef chains**
   - `typedef SDL_Type1 Type2;`
   - Resolve transitively or use Zig's type alias
 5. **Complex types**
   - Function pointers: Skip for now, add TODO comment
   - Unions: Skip for now, add TODO comment
   - Nested structs: Should work fine
 ## Testing Strategy
 1. **Unit tests** for each component:
   - TypeCollector: Test with known declarations
   - parseIncludes: Test with sample headers
   - extractType: Test finding types in headers
 2. **Integration test**:
   - Parse SDL_gpu.h
   - Verify missing types are detected
   - Verify dependencies are extracted
   - Verify output compiles
 3. **Validation**:
   - Remove manual type definitions from mock_test.zig
   - Import actual generated file
   - All tests should still pass
 ## Success Criteria
 ✅ Parser detects 5 missing types from SDL_gpu.h
 ✅ Parser extracts types from dependency headers
 ✅ Generated code compiles standalone
 ✅ All tests pass without manual type definitions
 ✅ Single output file contains all needed types
 ## Time Estimate
 - Phase 1 (Type Collection): 1-2 hours
 - Phase 2 (Include Parsing): 1 hour
 - Phase 3 (Type Extraction): 2-3 hours
 - Phase 4 (Integration): 1-2 hours
 - Phase 5 (Code Gen Enhancement): 1 hour
 - Testing & Refinement: 2 hours
 **Total: 8-11 hours**
 ## Advantages of This Approach
 1. **Simple**: Single output file, no module management
 2. **Fast**: Only parse dependency headers when needed
 3. **Minimal**: Only extract required types
 4. **Robust**: Zig handles duplicate definitions
 5. **Maintainable**: Clear separation in output
 ## Open Questions
 1. Should we cache parsed dependency headers?
   - **Answer**: Yes, parse once, extract many types
 2. How to handle nested dependencies (Type A needs Type B)?
   - **Answer**: Recursive extraction, track visited types
 3. Should dependencies go at top or bottom of file?
   - **Answer**: Top, before primary declarations use them
 4. What about #define constants?
   - **Answer**: Skip for now, out of scope
 ## Next Steps
 1. Implement TypeCollector
 2. Implement include parsing
 3. Implement type extraction
 4. Wire together in main
 5. Test with SDL_gpu.h
 6. Update documentation
 ---
 Ready to implement? This plan provides:
 - Clear phases with time estimates
 - Concrete code examples
 - Handles edge cases
 - Simple single-file output
 - Full testing strategy