# Dependency Resolution Flow - Technical Deep Dive ## Overview This document traces the complete flow from parser entry point through dependency resolution to final output generation. ## Flow Diagram ``` main() ↓ Parse Primary Header (SDL_gpu.h) ↓ Analyze Dependencies ↓ Extract Missing Types ↓ Combine Declarations ↓ Generate Output ``` ## Detailed Step-by-Step Flow ### Phase 1: Parser Entry Point **File**: `src/parser.zig::main()` ```zig pub fn main() !void { // 1. Setup var gpa = std.heap.GeneralPurposeAllocator(.{}){}; const allocator = gpa.allocator(); // 2. Parse command line arguments const header_path = args[1]; var output_file: ?[]const u8 = null; var mock_output_file: ?[]const u8 = null; // 3. Read the primary header file const source = try std.fs.cwd().readFileAlloc( allocator, header_path, 10 * 1024 * 1024 ); defer allocator.free(source); ``` **Inputs**: - Command line: `zig build run -- SDL_gpu.h --output=gpu.zig` - Header file contents read into memory **Outputs**: - `source`: []const u8 - Full header file content - `header_path`: []const u8 - Path for finding dependency headers --- ### Phase 2: Primary Header Parsing **File**: `src/parser.zig::main()` continued ```zig // 4. Parse declarations from primary header var scanner = patterns.Scanner.init(allocator, source); const decls = try scanner.scan(); // decls is now: []Declaration containing: // - 13 opaque types (GPUDevice, GPUTexture, etc.) // - 24 enums // - 35 structs // - 3 flags // - 94 functions ``` **Process**: 1. `Scanner.init()` creates scanner with allocator and source 2. `scanner.scan()` iterates through source line by line 3. Tries each pattern: opaque, enum, struct, flags, function 4. Builds array of `Declaration` union variants 5. Each declaration owns its strings (allocated from scanner's allocator) **Outputs**: - `decls`: []Declaration - Array of 169 declarations from SDL_gpu.h --- ### Phase 3: Dependency Analysis Entry **File**: `src/parser.zig::main()` continued ```zig // 5. Create dependency resolver var resolver = dependency_resolver.DependencyResolver.init(allocator); defer resolver.deinit(); // 6. Analyze declarations to find missing types try resolver.analyze(decls); ``` **What `DependencyResolver.init()` does**: ```zig pub fn init(allocator: Allocator) DependencyResolver { return .{ .allocator = allocator, .referenced_types = std.StringHashMap(void).init(allocator), .defined_types = std.StringHashMap(void).init(allocator), }; } ``` Creates two HashMaps: - `defined_types`: Types defined in primary header - `referenced_types`: Types used in function signatures/struct fields --- ### Phase 4: Type Collection **File**: `src/dependency_resolver.zig::DependencyResolver.analyze()` ```zig pub fn analyze(self: *DependencyResolver, decls: []const Declaration) !void { try self.collectDefinedTypes(decls); // Step 4a try self.collectReferencedTypes(decls); // Step 4b } ``` #### Step 4a: Collect Defined Types ```zig fn collectDefinedTypes(self: *DependencyResolver, decls: []const Declaration) !void { for (decls) |decl| { const type_name = switch (decl) { .opaque_type => |o| o.name, // e.g., "SDL_GPUDevice" .enum_decl => |e| e.name, // e.g., "SDL_GPUPrimitiveType" .struct_decl => |s| s.name, // e.g., "SDL_GPUViewport" .flag_decl => |f| f.name, // e.g., "SDL_GPUTextureUsageFlags" .function_decl => continue, // Functions don't define types }; try self.defined_types.put(type_name, {}); } } ``` **Result**: `defined_types` HashMap contains: ``` SDL_GPUDevice -> {} SDL_GPUTexture -> {} SDL_GPUViewport -> {} SDL_GPUPrimitiveType -> {} ... (166 more entries) ``` #### Step 4b: Collect Referenced Types ```zig fn collectReferencedTypes(self: *DependencyResolver, decls: []const Declaration) !void { for (decls) |decl| { switch (decl) { .function_decl => |func| { // Scan return type try self.scanType(func.return_type); // Scan each parameter type for (func.params) |param| { try self.scanType(param.type_name); } }, .struct_decl => |struct_decl| { // Scan each field type for (struct_decl.fields) |field| { try self.scanType(field.type_name); } }, else => {}, } } } ``` **Example**: Function signature processing ```c // C function: bool SDL_WindowSupportsGPUSwapchain(SDL_GPUDevice *device, SDL_Window *window) // Parser sees: .function_decl = { .return_type = "bool", .params = [ { .type_name = "SDL_GPUDevice *" }, { .type_name = "SDL_Window *" } ] } ``` **For each type string, calls `scanType()`**: --- ### Phase 5: Type Extraction & Normalization **File**: `src/dependency_resolver.zig::scanType()` ```zig fn scanType(self: *DependencyResolver, type_str: []const u8) !void { // Extract base type from decorated string const base_type = extractBaseType(type_str); if (base_type.len > 0 and isSDLType(base_type)) { // Only add if not already present (deduplicate) if (!self.referenced_types.contains(base_type)) { // Must own the string (type_str may be freed) const owned = try self.allocator.dupe(u8, base_type); try self.referenced_types.put(owned, {}); } } } ``` #### Example: Type Extraction Process **Input**: `"SDL_Window *"` **Step-by-step through `extractBaseType()`**: ```zig fn extractBaseType(type_str: []const u8) []const u8 { var result = "SDL_Window *"; // Loop 1: Remove leading qualifiers result = std.mem.trim(u8, result, " \t"); // "SDL_Window *" // No leading "const", "?", "*", etc. // Loop 2: Remove trailing qualifiers result = std.mem.trim(u8, result, " \t"); // "SDL_Window *" // Check trailing "*" if (std.mem.endsWith(u8, result, "*")) { result = result[0..result.len-1]; // "SDL_Window " continue; } result = std.mem.trim(u8, result, " \t"); // "SDL_Window" return "SDL_Window"; } ``` **Output**: `"SDL_Window"` (clean type name) **More Examples**: ``` "?*SDL_GPUDevice" -> "SDL_GPUDevice" "*const SDL_Rect" -> "SDL_Rect" "SDL_GPUBuffer *const *" -> "SDL_GPUBuffer" "[*c]const u8" -> "u8" "SDL_FColor" -> "SDL_FColor" ``` #### SDL Type Detection ```zig fn isSDLType(type_str: []const u8) bool { // Check for SDL_ prefix if (std.mem.startsWith(u8, type_str, "SDL_")) { return true; } // Check known Zig-ified names const known_types = [_][]const u8{ "Window", "Rect", "FColor", "FlipMode", "PropertiesID", "Surface", ... }; for (known_types) |known| { if (std.mem.eql(u8, type_str, known)) { return true; } } return false; // Primitive type like "bool", "u32" } ``` **Result**: `referenced_types` HashMap contains: ``` SDL_Window -> {} SDL_Rect -> {} SDL_FColor -> {} SDL_FlipMode -> {} SDL_PropertiesID -> {} SDL_GPUShaderFormat -> {} ``` --- ### Phase 6: Missing Type Calculation **File**: `src/parser.zig::main()` continued ```zig // 7. Get missing types (referenced but not defined) const missing_types = try resolver.getMissingTypes(allocator); defer { for (missing_types) |t| allocator.free(t); allocator.free(missing_types); } ``` **File**: `src/dependency_resolver.zig::getMissingTypes()` ```zig pub fn getMissingTypes(self: *DependencyResolver, allocator: Allocator) ![][]const u8 { var missing = std.ArrayList([]const u8){}; var it = self.referenced_types.keyIterator(); while (it.next()) |key| { // Check if type is NOT in defined_types if (!self.defined_types.contains(key.*)) { // This is a missing type - need to find it try missing.append(allocator, try allocator.dupe(u8, key.*)); } } return try missing.toOwnedSlice(allocator); } ``` **Logic**: ``` referenced_types = {SDL_Window, SDL_Rect, SDL_FColor, ...} defined_types = {SDL_GPUDevice, SDL_GPUTexture, ...} missing_types = referenced_types - defined_types = {SDL_Window, SDL_Rect, SDL_FColor, SDL_FlipMode, SDL_PropertiesID, SDL_GPUShaderFormat} ``` **Output**: Array of 6 strings (owned by caller) --- ### Phase 7: Include Header Parsing **File**: `src/parser.zig::main()` continued ```zig if (missing_types.len > 0) { // 8. Parse #include directives from source const includes = try dependency_resolver.parseIncludes(allocator, source); defer { for (includes) |inc| allocator.free(inc); allocator.free(includes); } ``` **File**: `src/dependency_resolver.zig::parseIncludes()` ```zig pub fn parseIncludes(allocator: Allocator, source: []const u8) ![][]const u8 { var includes = std.ArrayList([]const u8){}; var lines = std.mem.splitScalar(u8, source, '\n'); while (lines.next()) |line| { const trimmed = std.mem.trim(u8, line, " \t\r"); // Match: #include if (std.mem.startsWith(u8, trimmed, "#include ")) |end| { const header_name = trimmed[after_open..][0..end]; try includes.append(allocator, try allocator.dupe(u8, header_name)); } } } return try includes.toOwnedSlice(allocator); } ``` **Example**: From SDL_gpu.h header: ```c #include #include #include #include #include #include ``` **Output**: Array of strings: ``` ["SDL_stdinc.h", "SDL_pixels.h", "SDL_properties.h", "SDL_rect.h", "SDL_surface.h", "SDL_video.h"] ``` --- ### Phase 8: Dependency Type Extraction **File**: `src/parser.zig::main()` continued ```zig // 9. Determine header directory const header_dir = std.fs.path.dirname(header_path) orelse "."; // e.g., "../SDL/include/SDL3" var dependency_decls = std.ArrayList(patterns.Declaration){}; defer { for (dependency_decls.items) |dep_decl| { freeDeclDeep(allocator, dep_decl); } dependency_decls.deinit(allocator); } // 10. For each missing type, search dependency headers for (missing_types) |missing_type| { var found = false; // Try each included header for (includes) |include| { // 10a. Build full path const dep_path = try std.fs.path.join( allocator, &[_][]const u8{ header_dir, include } ); defer allocator.free(dep_path); // e.g., "../SDL/include/SDL3/SDL_pixels.h" // 10b. Read dependency header const dep_source = std.fs.cwd().readFileAlloc( allocator, dep_path, 10 * 1024 * 1024 ) catch continue; // Skip if can't read defer allocator.free(dep_source); // 10c. Extract type from this header if (try dependency_resolver.extractTypeFromHeader( allocator, dep_source, missing_type )) |dep_decl| { try dependency_decls.append(allocator, dep_decl); std.debug.print(" ✓ Found {s} in {s}\n", .{missing_type, include}); found = true; break; // Found it, stop searching } } if (!found) { std.debug.print(" ⚠ Warning: Could not find {s}\n", .{missing_type}); } } ``` **Search Algorithm**: ``` For missing_type "SDL_Window": Try SDL_stdinc.h -> Not found Try SDL_pixels.h -> Not found Try SDL_properties.h -> Not found Try SDL_rect.h -> Not found Try SDL_surface.h -> Not found Try SDL_video.h -> FOUND! ✓ ``` --- ### Phase 9: Type Extraction from Header **File**: `src/dependency_resolver.zig::extractTypeFromHeader()` ```zig pub fn extractTypeFromHeader( allocator: Allocator, header_source: []const u8, type_name: []const u8, // e.g., "SDL_Window" ) !?Declaration { // 1. Parse the entire dependency header var scanner = patterns.Scanner.init(allocator, header_source); const all_decls = try scanner.scan(); defer { for (all_decls) |decl| { freeDeclaration(allocator, decl); } allocator.free(all_decls); } // 2. Search for matching type for (all_decls) |decl| { const decl_name = switch (decl) { .opaque_type => |o| o.name, .enum_decl => |e| e.name, .struct_decl => |s| s.name, .flag_decl => |f| f.name, else => continue, }; // 3. Found it! if (std.mem.eql(u8, decl_name, type_name)) { // 4. Deep clone so caller owns it return try cloneDeclaration(allocator, decl); } } return null; // Not found in this header } ``` **Example**: Searching SDL_video.h for SDL_Window 1. Parse SDL_video.h → 50+ declarations 2. Iterate through all declarations 3. Find: `.opaque_type = { .name = "SDL_Window", ... }` 4. Clone the declaration (deep copy all strings) 5. Return the clone 6. Free all the temporary declarations from parsing **Cloning Process**: ```zig fn cloneDeclaration(allocator: Allocator, decl: Declaration) !Declaration { return switch (decl) { .opaque_type => |o| .{ .opaque_type = .{ .name = try allocator.dupe(u8, o.name), // Own the string .doc_comment = if (o.doc_comment) |doc| try allocator.dupe(u8, doc) else null, }, }, // ... similar for enum, struct, flags }; } ``` **Why clone?** The parsed declarations from `scanner.scan()` are freed after this function returns. We need owned copies that live until code generation. --- ### Phase 10: Declaration Combining **File**: `src/parser.zig::main()` continued ```zig // 11. Combine dependency declarations with primary var all_decls = std.ArrayList(patterns.Declaration){}; defer all_decls.deinit(allocator); // IMPORTANT: Dependencies FIRST! try all_decls.appendSlice(allocator, dependency_decls.items); try all_decls.appendSlice(allocator, decls); ``` **Result**: Combined array ``` all_decls = [ // Dependencies (4 items) { .struct_decl = SDL_FColor }, { .enum_decl = SDL_FlipMode }, { .struct_decl = SDL_Rect }, { .opaque_type = SDL_Window }, // Primary header (169 items) { .opaque_type = SDL_GPUDevice }, { .enum_decl = SDL_GPUPrimitiveType }, ... (167 more) ] ``` **Why dependencies first?** Types must be defined before they're used. Since primary header references dependency types, dependencies must come first. --- ### Phase 11: Code Generation **File**: `src/parser.zig::main()` continued ```zig // 12. Generate Zig code from all declarations const output = try codegen.CodeGen.generate(allocator, all_decls.items); defer allocator.free(output); ``` **File**: `src/codegen.zig::CodeGen.generate()` (simplified) ```zig pub fn generate(allocator: Allocator, decls: []const Declaration) ![]const u8 { var buf = std.ArrayList(u8){}; // Header try buf.appendSlice(allocator, "pub const c = @import(\"c.zig\").c;\n\n"); // Generate each declaration for (decls) |decl| { switch (decl) { .opaque_type => |o| { try buf.appendSlice(allocator, "pub const "); try buf.appendSlice(allocator, stripSDLPrefix(o.name)); try buf.appendSlice(allocator, " = opaque {};\n"); }, .struct_decl => |s| { try generateStruct(allocator, &buf, s); }, // ... other types } } return try buf.toOwnedSlice(allocator); } ``` **Output** (excerpt): ```zig pub const c = @import("c.zig").c; pub const FColor = extern struct { r: f32, g: f32, b: f32, a: f32, }; pub const Window = opaque {}; pub const GPUDevice = opaque { pub inline fn windowSupportsGPU( gpudevice: *GPUDevice, window: ?*Window, // ✓ Window is defined above! ) bool { return c.SDL_WindowSupportsGPUDevice(gpudevice, window); } }; ``` --- ### Phase 12: AST Validation & Formatting **File**: `src/parser.zig::main()` continued ```zig // 13. Parse generated code as Zig AST const output_z = try allocator.dupeZ(u8, output); defer allocator.free(output_z); var ast = try std.zig.Ast.parse(allocator, output_z, .zig); defer ast.deinit(allocator); // 14. Check for syntax errors if (ast.errors.len > 0) { std.debug.print("\nError: {d} syntax errors\n", .{ast.errors.len}); for (ast.errors) |err| { const loc = ast.tokenLocation(0, err.token); std.debug.print(" Line {d}: {s}\n", .{ loc.line + 1, @tagName(err.tag) }); } return error.InvalidSyntax; } // 15. Format using Zig's formatter const formatted_output = try ast.renderAlloc(allocator); defer allocator.free(formatted_output); ``` **Why validate?** Catch codegen bugs early. If generated code doesn't parse, we know immediately. **Why format?** Zig's formatter ensures consistent style, proper indentation, and canonical formatting. --- ### Phase 13: Output Writing **File**: `src/parser.zig::main()` continued ```zig // 16. Write to file or stdout if (output_file) |file_path| { try std.fs.cwd().writeFile(.{ .sub_path = file_path, .data = formatted_output, }); std.debug.print("Generated: {s}\n", .{file_path}); } else { _ = try std.posix.write(std.posix.STDOUT_FILENO, formatted_output); } ``` --- ## Memory Management Flow ### Allocations 1. **Primary header source**: Freed at end of main() 2. **Primary declarations**: Freed at end of main() (with deep free) 3. **Dependency resolver HashMaps**: Freed in resolver.deinit() 4. **HashMap keys** (in referenced_types): Freed in resolver.deinit() 5. **Missing types array**: Freed explicitly after use 6. **Includes array**: Freed explicitly after use 7. **Dependency header sources**: Freed immediately after extraction 8. **Temporary parsed declarations**: Freed immediately in extractTypeFromHeader() 9. **Cloned dependency declarations**: Freed at end of scope 10. **Generated output**: Freed after writing 11. **Formatted output**: Freed after writing ### Ownership Rules - **Scanner owns strings** during parsing (from its allocator) - **Cloned declarations own strings** after extraction (allocated explicitly) - **HashMap owns keys** in referenced_types (duped when inserted) - **Caller owns result** of getMissingTypes(), parseIncludes() --- ## Error Handling Flow ### Errors That Fail ```zig // Fatal errors - exit immediately - File not found (primary header) - Out of memory - Invalid syntax in generated code (optional) ``` ### Errors That Warn ```zig // Warnings - continue execution - Dependency header not readable → continue with next header - Type not found in any header → print warning, continue - Struct parsing errors → generate partial output ``` ### Example Error Flow ``` Parse SDL_gpu.h ↓ Missing type: SDL_Window ↓ Try SDL_pixels.h → catch FileNotFound → continue Try SDL_video.h → Success! → break ↓ Missing type: SDL_Unknown ↓ Try all headers → Not found → print warning ↓ Continue with partial results ``` --- ## Performance Characteristics ### Time Complexity - Primary parsing: O(n) where n = source lines - Type collection: O(d) where d = declarations - Missing type detection: O(r) where r = referenced types - Type extraction: O(h × d) where h = headers, d = declarations per header - Overall: O(n + d + r + h×d) ≈ O(n) for typical cases ### Space Complexity - Primary declarations: O(d) - Dependency declarations: O(m) where m = missing types - HashMaps: O(t) where t = total unique types - Peak memory: ~2-5MB for SDL_gpu.h ### Optimization Points 1. **Cache parsed headers** - Currently re-parse for each missing type 2. **Early exit** - Stop searching after finding type 3. **String interning** - Deduplicate type name strings 4. **Lazy loading** - Only parse dependencies if missing types detected --- ## Testing the Flow ### Unit Test Example ```zig test "complete dependency flow" { const source = \\typedef struct SDL_Type SDL_Type; \\extern void SDL_Func(SDL_External *param); ; // Phase 1: Parse var scanner = Scanner.init(allocator, source); const decls = try scanner.scan(); // Phase 2: Analyze var resolver = DependencyResolver.init(allocator); defer resolver.deinit(); try resolver.analyze(decls); // Phase 3: Get missing const missing = try resolver.getMissingTypes(allocator); defer allocator.free(missing); // Verify: SDL_External is missing try testing.expectEqual(@as(usize, 1), missing.len); try testing.expectEqualStrings("SDL_External", missing[0]); } ``` ### Integration Test ```bash # Create test header with dependency echo 'typedef struct Dep Dep;' > dep.h echo '#include "dep.h"' > main.h echo 'void func(Dep *d);' >> main.h # Parse with dependency resolution zig build run -- main.h --output=out.zig # Verify output contains Dep grep 'pub const Dep' out.zig ``` --- ## Summary The dependency resolution flow is: 1. **Parse** primary header → get declarations 2. **Analyze** declarations → find referenced vs defined types 3. **Calculate** missing = referenced - defined 4. **Extract** #include directives from source 5. **Search** dependency headers for missing types 6. **Clone** found declarations (deep copy) 7. **Combine** dependency + primary declarations 8. **Generate** Zig code with all types 9. **Validate** and format using Zig AST 10. **Output** to file or stdout Each phase has clear inputs/outputs, proper memory management, and graceful error handling.