# Dependency Resolution Implementation Plan (v2) ## Core Insight **Single-file generation with on-demand type resolution**: Parse the primary header completely, identify missing types, then parse included headers ONLY for those specific types. Append them to the same output file. Zig's structural typing handles everything else. ## Why This Works 1. **No modules needed** - One output file with all types 2. **Zig deduplicates** - Multiple parsings of same type are safe 3. **Forward references work** - Zig allows using types before they're defined 4. **Simple implementation** - Just append to existing output ## Three-Phase Algorithm ### Phase 1: Primary Header Parsing (Existing) ``` Input: SDL_gpu.h Output: declarations[], generated_code ``` ### Phase 2: Missing Type Detection (NEW) ``` 1. Scan generated_code for all type references 2. Build set of defined_types from declarations[] 3. missing_types = referenced_types - defined_types ``` ### Phase 3: Dependency Resolution (NEW) ``` For each missing_type: 1. Parse each included header 2. If type found, extract declaration 3. Append to output ``` ## Implementation Details ## Implementation Details ### Component 1: Type Reference Scanner **Purpose**: Find all type names used in generated code **Location**: New file `src/dependency_resolver.zig` ```zig pub const TypeReference = struct { name: []const u8, source_location: []const u8, // For debugging }; pub fn scanTypeReferences(decls: []const Declaration) ![]TypeReference { var refs = ArrayList(TypeReference).init(allocator); for (decls) |decl| { switch (decl) { .function_decl => |func| { // Scan return type try scanType(func.return_type, &refs); // Scan parameters for (func.params) |param| { try scanType(param.type_name, &refs); } }, .struct_decl => |struct_decl| { // Scan field types for (struct_decl.fields) |field| { try scanType(field.type_name, &refs); } }, // opaque/enum don't reference other types else => {}, } } return refs.toOwnedSlice(); } fn scanType(type_str: []const u8, refs: *ArrayList(TypeReference)) !void { // Extract base type from "?*const SDL_Type" → "SDL_Type" const base_type = extractBaseType(type_str); if (isSDLType(base_type)) { try refs.append(.{ .name = base_type, .source_location = type_str }); } } ``` **Key functions**: - `extractBaseType()`: Strip pointers, const, optional from type string - `isSDLType()`: Check if starts with "SDL_" or is known SDL type - Handle edge cases: arrays, function pointers (skip for now) ### Component 2: Defined Type Collector **Purpose**: Track what types are already defined ```zig pub fn collectDefinedTypes(decls: []const Declaration) StringHashMap(void) { var defined = StringHashMap(void).init(allocator); for (decls) |decl| { const type_name = switch (decl) { .opaque_type => |o| o.name, .enum_decl => |e| e.name, .struct_decl => |s| s.name, .flags_decl => |f| f.name, .function_decl => continue, // Functions don't define types }; try defined.put(type_name, {}); } return defined; } ``` ### Component 3: Include Header Parser **Purpose**: Extract #include directives ```zig pub fn parseIncludes(source: []const u8) ![]const []const u8 { var includes = ArrayList([]const u8).init(allocator); var lines = std.mem.split(u8, source, "\n"); while (lines.next()) |line| { // Match: #include if (std.mem.indexOf(u8, line, "#include ")) |end| { const header_name = line[after_open..][0..end]; try includes.append(try allocator.dupe(u8, header_name)); } } } return includes.toOwnedSlice(); } ``` ### Component 4: Selective Type Extractor **Purpose**: Find specific type in a header ```zig pub fn extractTypeFromHeader( allocator: Allocator, header_source: []const u8, type_name: []const u8, // e.g., "SDL_Rect" ) !?Declaration { // Parse the header var scanner = Scanner.init(allocator, header_source); const all_decls = try scanner.scan(); defer scanner.deinit(); // Find matching declaration for (all_decls) |decl| { const decl_name = switch (decl) { .opaque_type => |o| o.name, .enum_decl => |e| e.name, .struct_decl => |s| s.name, .flags_decl => |f| f.name, else => continue, }; if (std.mem.eql(u8, decl_name, type_name)) { return try decl.clone(allocator); // Deep copy } } return null; // Not found } ``` ### Component 5: Main Integration **Location**: Modify `src/parser.zig` main function ```zig pub fn main() !void { // ... existing setup ... // 1. Parse primary header (existing) var scanner = Scanner.init(allocator, source); const primary_decls = try scanner.scan(); // 2. Identify missing types (NEW) const references = try scanTypeReferences(primary_decls); const defined = collectDefinedTypes(primary_decls); var missing = ArrayList([]const u8).init(allocator); for (references) |ref| { if (!defined.contains(ref.name)) { try missing.append(ref.name); } } // 3. Extract missing types from dependencies (NEW) var dependency_decls = ArrayList(Declaration).init(allocator); if (missing.items.len > 0) { const includes = try parseIncludes(source); const header_dir = std.fs.path.dirname(header_path) orelse "."; for (missing.items) |missing_type| { var found = false; for (includes) |include| { const dep_path = try std.fs.path.join( allocator, &[_][]const u8{ header_dir, include } ); defer allocator.free(dep_path); const dep_source = std.fs.cwd().readFileAlloc( allocator, dep_path, 10 * 1024 * 1024 ) catch continue; defer allocator.free(dep_source); if (try extractTypeFromHeader(allocator, dep_source, missing_type)) |decl| { try dependency_decls.append(decl); found = true; break; } } if (!found) { std.debug.print( "Warning: Could not find definition for type: {s}\n", .{missing_type} ); } } } // 4. Combine declarations var all_decls = ArrayList(Declaration).init(allocator); try all_decls.appendSlice(dependency_decls.items); // Dependencies first! try all_decls.appendSlice(primary_decls); // 5. Generate code (existing, but with all declarations) const output = try codegen.generate(allocator, all_decls.items); // ... rest of existing code ... } ``` ## Key Implementation Decisions 1. **Dependencies go FIRST** in output - Ensures types are defined before use - More logical reading order 2. **Deep copy declarations** - Avoid lifetime issues with parsed headers - Each declaration owns its strings 3. **Warning for missing types** - Don't fail build, just warn - Allows gradual improvement 4. **Skip function pointers/unions** - Add TODO comments - Focus on common cases first 5. **Cache parsed headers** - Parse each dependency header once - Extract multiple types from same parse ## Testing Approach ### Unit Tests ```zig test "scanTypeReferences finds SDL types" { const decls = [_]Declaration{ .{ .function_decl = .{ .name = "test", .return_type = "?*SDL_Window", .params = &[_]Param{ .{ .name = "rect", .type_name = "*const SDL_Rect" }, }, }}, }; const refs = try scanTypeReferences(&decls); try testing.expect(refs.len == 2); try testing.expectEqualStrings("SDL_Window", refs[0].name); try testing.expectEqualStrings("SDL_Rect", refs[1].name); } test "collectDefinedTypes tracks declarations" { const decls = [_]Declaration{ .{ .opaque_type = .{ .name = "SDL_Device" }}, .{ .struct_decl = .{ .name = "SDL_Info", .fields = &[_]Field{} }}, }; const defined = collectDefinedTypes(&decls); try testing.expect(defined.contains("SDL_Device")); try testing.expect(defined.contains("SDL_Info")); } ``` ### Integration Test ```zig test "full dependency resolution with SDL_gpu.h" { const source = try std.fs.cwd().readFileAlloc( testing.allocator, "SDL/include/SDL3/SDL_gpu.h", 10 * 1024 * 1024 ); defer testing.allocator.free(source); // Parse and resolve const output = try parseWithDependencies(testing.allocator, source, "SDL/include/SDL3"); defer testing.allocator.free(output); // Verify missing types are present try testing.expect(std.mem.indexOf(u8, output, "pub const Window = opaque") != null); try testing.expect(std.mem.indexOf(u8, output, "pub const Rect = extern struct") != null); try testing.expect(std.mem.indexOf(u8, output, "pub const FColor = extern struct") != null); // Verify it compiles var ast = try std.zig.Ast.parse(testing.allocator, output, .zig); defer ast.deinit(testing.allocator); try testing.expect(ast.errors.len == 0); } ``` ## Validation Steps 1. **Remove manual definitions** from mock_test.zig: ```diff - pub const Window = opaque {}; - pub const Rect = extern struct { ... }; - pub const FColor = extern struct { ... }; ``` 2. **Import generated file** directly: ```zig const gpu = @import("../../zig-out/gpu_test.zig"); ``` 3. **Run tests**: ```bash zig build test-mocks # Should still pass! ``` ## Success Metrics ✅ `scanTypeReferences` finds 30+ type references in SDL_gpu.h ✅ `collectDefinedTypes` tracks 169 defined types ✅ Missing types: Window, Rect, FColor, FlipMode, PropertiesID detected ✅ All 5 missing types extracted from dependency headers ✅ Generated code compiles without manual definitions ✅ All 11 tests pass ✅ Build time increase < 1 second ## Rollout Plan 1. **Day 1**: Implement Components 1-2 (type scanning/collecting) 2. **Day 2**: Implement Components 3-4 (include parsing/type extraction) 3. **Day 3**: Integrate into main, test with SDL_gpu.h 4. **Day 4**: Refine, handle edge cases, update tests 5. **Day 5**: Documentation, final validation ## Risks & Mitigation | Risk | Mitigation | |------|-----------| | Can't find header files | Require header directory as input | | Type not in any header | Emit warning, generate placeholder | | Parsing dependency fails | Catch error, continue with other headers | | Performance (parsing multiple headers) | Cache parsed headers, parse once | | Circular dependencies | Not an issue - all types in one file | --- This plan is **ready to implement**. Each component is well-defined with clear inputs/outputs, error handling, and test cases. **File**: `src/type_collector.zig` ```zig const TypeCollector = struct { defined_types: StringHashMap(void), // Types defined in primary header referenced_types: StringHashMap(void), // Types used in signatures pub fn collectFromDeclarations(decls: []Declaration) TypeCollector; pub fn getMissingTypes() []const []const u8; }; ``` **Tasks**: - Scan all declarations for type definitions (opaque, enum, struct, flags) - Scan all function signatures for type references - Return set difference: referenced - defined ### Phase 2: Include Directive Parsing (1 hour) **File**: `src/patterns.zig` (extend existing) ```zig pub fn parseIncludes(source: []const u8) ![]const []const u8 { // Find all #include directives // Return list of header filenames } ``` **Tasks**: - Add regex/pattern for `#include ` - Extract header filename from directive - Return list of included headers ### Phase 3: Selective Type Extraction (2-3 hours) **File**: `src/type_extractor.zig` ```zig pub fn extractType( allocator: Allocator, header_source: []const u8, type_name: []const u8, // e.g., "SDL_Rect" ) !?Declaration { // Parse header looking for specific type // Return the declaration if found } pub fn extractTypes( allocator: Allocator, header_paths: []const []const u8, missing_types: []const []const u8, ) ![]Declaration { // For each missing type: // For each header: // Try to extract the type // If found, add to results // Return all found declarations } ``` **Tasks**: - Reuse existing Scanner but filter by type name - Handle opaque types: `typedef struct SDL_Type SDL_Type;` - Handle structs: `typedef struct { ... } SDL_Type;` - Handle enums: `typedef enum { ... } SDL_Type;` - Handle simple typedefs: `typedef uint32_t SDL_Type;` ### Phase 4: Integration (1-2 hours) **File**: `src/parser.zig` (modify main function) ```zig pub fn main() !void { // 1. Parse primary header var scanner = Scanner.init(allocator, source); const decls = try scanner.scan(); // 2. Collect missing types const collector = TypeCollector.collectFromDeclarations(decls); const missing_types = try collector.getMissingTypes(allocator); // 3. Parse included headers for missing types const includes = try parseIncludes(source); const header_dir = getHeaderDirectory(header_path); const dependency_decls = try extractTypes( allocator, header_dir, includes, missing_types ); // 4. Generate code with dependencies appended var all_decls = std.ArrayList(Declaration).init(allocator); try all_decls.appendSlice(decls); try all_decls.appendSlice(dependency_decls); const output = try codegen.generate(allocator, all_decls.items); // 5. Write output try writeOutput(output_file, output); } ``` **Tasks**: - Wire together all components - Handle header path resolution - Add comment separators for dependencies - Update error handling ### Phase 5: Code Generation Enhancement (1 hour) **File**: `src/codegen.zig` (modify) Add dependency section: ```zig fn generate() ![]const u8 { try output.appendSlice("pub const c = @import(\"c.zig\").c;\n\n"); // Add comment if we have dependencies if (has_dependency_decls) { try output.appendSlice( \\// Dependencies from included headers \\// These types are referenced by the primary header \\ ); } // Generate all declarations (primary + dependencies) for (decls) |decl| { try generateDeclaration(decl); } } ``` **Tasks**: - Add dependency comment section - Mark which declarations are dependencies (optional) - Ensure proper ordering (dependencies before usage) ## Example Output ```zig pub const c = @import("c.zig").c; // Dependencies from included headers // These types are referenced by SDL_gpu.h // From SDL_rect.h pub const Rect = extern struct { x: i32, y: i32, w: i32, h: i32, }; // From SDL_pixels.h pub const FColor = extern struct { r: f32, g: f32, b: f32, a: f32, }; // From SDL_video.h pub const Window = opaque {}; // From SDL_properties.h pub const PropertiesID = u32; // SDL_gpu.h declarations pub const GPUDevice = opaque { pub inline fn windowSupportsGPUSwapchainComposition( gpudevice: *GPUDevice, window: ?*Window, // ✅ Now defined! swapchain_composition: GPUSwapchainComposition ) bool { ... } }; pub const GPURenderPass = opaque { pub inline fn setGPUScissor( gpurenderpass: *GPURenderPass, scissor: *const Rect // ✅ Now defined! ) void { ... } }; ``` ## Edge Cases 1. **Type not found in any header** - Emit warning - Generate placeholder: `pub const TypeName = opaque {};` 2. **Circular dependencies** - Not an issue - all types in one file - Zig allows forward references 3. **Multiple definitions** - Keep first definition found - Zig will error if layouts differ (good!) 4. **Typedef chains** - `typedef SDL_Type1 Type2;` - Resolve transitively or use Zig's type alias 5. **Complex types** - Function pointers: Skip for now, add TODO comment - Unions: Skip for now, add TODO comment - Nested structs: Should work fine ## Testing Strategy 1. **Unit tests** for each component: - TypeCollector: Test with known declarations - parseIncludes: Test with sample headers - extractType: Test finding types in headers 2. **Integration test**: - Parse SDL_gpu.h - Verify missing types are detected - Verify dependencies are extracted - Verify output compiles 3. **Validation**: - Remove manual type definitions from mock_test.zig - Import actual generated file - All tests should still pass ## Success Criteria ✅ Parser detects 5 missing types from SDL_gpu.h ✅ Parser extracts types from dependency headers ✅ Generated code compiles standalone ✅ All tests pass without manual type definitions ✅ Single output file contains all needed types ## Time Estimate - Phase 1 (Type Collection): 1-2 hours - Phase 2 (Include Parsing): 1 hour - Phase 3 (Type Extraction): 2-3 hours - Phase 4 (Integration): 1-2 hours - Phase 5 (Code Gen Enhancement): 1 hour - Testing & Refinement: 2 hours **Total: 8-11 hours** ## Advantages of This Approach 1. **Simple**: Single output file, no module management 2. **Fast**: Only parse dependency headers when needed 3. **Minimal**: Only extract required types 4. **Robust**: Zig handles duplicate definitions 5. **Maintainable**: Clear separation in output ## Open Questions 1. Should we cache parsed dependency headers? - **Answer**: Yes, parse once, extract many types 2. How to handle nested dependencies (Type A needs Type B)? - **Answer**: Recursive extraction, track visited types 3. Should dependencies go at top or bottom of file? - **Answer**: Top, before primary declarations use them 4. What about #define constants? - **Answer**: Skip for now, out of scope ## Next Steps 1. Implement TypeCollector 2. Implement include parsing 3. Implement type extraction 4. Wire together in main 5. Test with SDL_gpu.h 6. Update documentation --- Ready to implement? This plan provides: - Clear phases with time estimates - Concrete code examples - Handles edge cases - Simple single-file output - Full testing strategy