Backlog/lib/sdl3/parser/DEPENDENCY_FLOW.md

Dependency Resolution Flow - Technical Deep Dive

Overview

This document traces the complete flow from parser entry point through dependency resolution to final output generation.

Flow Diagram

main()
  ↓
  Parse Primary Header (SDL_gpu.h)
  ↓
  Analyze Dependencies
  ↓
  Extract Missing Types
  ↓
  Combine Declarations
  ↓
  Generate Output

Detailed Step-by-Step Flow

Phase 1: Parser Entry Point

File: src/parser.zig::main()

pub fn main() !void {
    // 1. Setup
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    const allocator = gpa.allocator();
    
    // 2. Parse command line arguments
    const header_path = args[1];
    var output_file: ?[]const u8 = null;
    var mock_output_file: ?[]const u8 = null;
    
    // 3. Read the primary header file
    const source = try std.fs.cwd().readFileAlloc(
        allocator, 
        header_path, 
        10 * 1024 * 1024
    );
    defer allocator.free(source);

Inputs:

• Command line: zig build run -- SDL_gpu.h --output=gpu.zig
• Header file contents read into memory

Outputs:

• source: []const u8 - Full header file content
• header_path: []const u8 - Path for finding dependency headers

---

Phase 2: Primary Header Parsing

File: src/parser.zig::main() continued

    // 4. Parse declarations from primary header
    var scanner = patterns.Scanner.init(allocator, source);
    const decls = try scanner.scan();
    
    // decls is now: []Declaration containing:
    // - 13 opaque types (GPUDevice, GPUTexture, etc.)
    // - 24 enums
    // - 35 structs
    // - 3 flags
    // - 94 functions

Process:

1. Scanner.init() creates scanner with allocator and source
2. scanner.scan() iterates through source line by line
3. Tries each pattern: opaque, enum, struct, flags, function
4. Builds array of Declaration union variants
5. Each declaration owns its strings (allocated from scanner's allocator)

Outputs:

• decls: []Declaration - Array of 169 declarations from SDL_gpu.h

---

Phase 3: Dependency Analysis Entry

File: src/parser.zig::main() continued

    // 5. Create dependency resolver
    var resolver = dependency_resolver.DependencyResolver.init(allocator);
    defer resolver.deinit();
    
    // 6. Analyze declarations to find missing types
    try resolver.analyze(decls);

What DependencyResolver.init() does:

pub fn init(allocator: Allocator) DependencyResolver {
    return .{
        .allocator = allocator,
        .referenced_types = std.StringHashMap(void).init(allocator),
        .defined_types = std.StringHashMap(void).init(allocator),
    };
}

Creates two HashMaps:

• defined_types: Types defined in primary header
• referenced_types: Types used in function signatures/struct fields

---

Phase 4: Type Collection

File: src/dependency_resolver.zig::DependencyResolver.analyze()

pub fn analyze(self: *DependencyResolver, decls: []const Declaration) !void {
    try self.collectDefinedTypes(decls);      // Step 4a
    try self.collectReferencedTypes(decls);   // Step 4b
}

Step 4a: Collect Defined Types

fn collectDefinedTypes(self: *DependencyResolver, decls: []const Declaration) !void {
    for (decls) |decl| {
        const type_name = switch (decl) {
            .opaque_type => |o| o.name,      // e.g., "SDL_GPUDevice"
            .enum_decl => |e| e.name,        // e.g., "SDL_GPUPrimitiveType"
            .struct_decl => |s| s.name,      // e.g., "SDL_GPUViewport"
            .flag_decl => |f| f.name,        // e.g., "SDL_GPUTextureUsageFlags"
            .function_decl => continue,      // Functions don't define types
        };
        try self.defined_types.put(type_name, {});
    }
}

Result: defined_types HashMap contains:

SDL_GPUDevice -> {}
SDL_GPUTexture -> {}
SDL_GPUViewport -> {}
SDL_GPUPrimitiveType -> {}
... (166 more entries)

Step 4b: Collect Referenced Types

fn collectReferencedTypes(self: *DependencyResolver, decls: []const Declaration) !void {
    for (decls) |decl| {
        switch (decl) {
            .function_decl => |func| {
                // Scan return type
                try self.scanType(func.return_type);
                // Scan each parameter type
                for (func.params) |param| {
                    try self.scanType(param.type_name);
                }
            },
            .struct_decl => |struct_decl| {
                // Scan each field type
                for (struct_decl.fields) |field| {
                    try self.scanType(field.type_name);
                }
            },
            else => {},
        }
    }
}

Example: Function signature processing

// C function:
bool SDL_WindowSupportsGPUSwapchain(SDL_GPUDevice *device, SDL_Window *window)

// Parser sees:
.function_decl = {
    .return_type = "bool",
    .params = [
        { .type_name = "SDL_GPUDevice *" },
        { .type_name = "SDL_Window *" }
    ]
}

For each type string, calls scanType():

---

Phase 5: Type Extraction & Normalization

File: src/dependency_resolver.zig::scanType()

fn scanType(self: *DependencyResolver, type_str: []const u8) !void {
    // Extract base type from decorated string
    const base_type = extractBaseType(type_str);
    
    if (base_type.len > 0 and isSDLType(base_type)) {
        // Only add if not already present (deduplicate)
        if (!self.referenced_types.contains(base_type)) {
            // Must own the string (type_str may be freed)
            const owned = try self.allocator.dupe(u8, base_type);
            try self.referenced_types.put(owned, {});
        }
    }
}

Example: Type Extraction Process

Input: "SDL_Window *"

Step-by-step through extractBaseType():

fn extractBaseType(type_str: []const u8) []const u8 {
    var result = "SDL_Window *";
    
    // Loop 1: Remove leading qualifiers
    result = std.mem.trim(u8, result, " \t");  // "SDL_Window *"
    // No leading "const", "?", "*", etc.
    
    // Loop 2: Remove trailing qualifiers
    result = std.mem.trim(u8, result, " \t");  // "SDL_Window *"
    
    // Check trailing "*"
    if (std.mem.endsWith(u8, result, "*")) {
        result = result[0..result.len-1];       // "SDL_Window "
        continue;
    }
    
    result = std.mem.trim(u8, result, " \t");   // "SDL_Window"
    
    return "SDL_Window";
}

Output: "SDL_Window" (clean type name)

More Examples:

"?*SDL_GPUDevice"         -> "SDL_GPUDevice"
"*const SDL_Rect"         -> "SDL_Rect"
"SDL_GPUBuffer *const *"  -> "SDL_GPUBuffer"
"[*c]const u8"            -> "u8"
"SDL_FColor"              -> "SDL_FColor"

SDL Type Detection

fn isSDLType(type_str: []const u8) bool {
    // Check for SDL_ prefix
    if (std.mem.startsWith(u8, type_str, "SDL_")) {
        return true;
    }
    
    // Check known Zig-ified names
    const known_types = [_][]const u8{
        "Window", "Rect", "FColor", "FlipMode", 
        "PropertiesID", "Surface", ...
    };
    
    for (known_types) |known| {
        if (std.mem.eql(u8, type_str, known)) {
            return true;
        }
    }
    
    return false;  // Primitive type like "bool", "u32"
}

Result: referenced_types HashMap contains:

SDL_Window -> {}
SDL_Rect -> {}
SDL_FColor -> {}
SDL_FlipMode -> {}
SDL_PropertiesID -> {}
SDL_GPUShaderFormat -> {}

---

Phase 6: Missing Type Calculation

File: src/parser.zig::main() continued

    // 7. Get missing types (referenced but not defined)
    const missing_types = try resolver.getMissingTypes(allocator);
    defer {
        for (missing_types) |t| allocator.free(t);
        allocator.free(missing_types);
    }

File: src/dependency_resolver.zig::getMissingTypes()

pub fn getMissingTypes(self: *DependencyResolver, allocator: Allocator) ![][]const u8 {
    var missing = std.ArrayList([]const u8){};
    
    var it = self.referenced_types.keyIterator();
    while (it.next()) |key| {
        // Check if type is NOT in defined_types
        if (!self.defined_types.contains(key.*)) {
            // This is a missing type - need to find it
            try missing.append(allocator, try allocator.dupe(u8, key.*));
        }
    }
    
    return try missing.toOwnedSlice(allocator);
}

Logic:

referenced_types = {SDL_Window, SDL_Rect, SDL_FColor, ...}
defined_types = {SDL_GPUDevice, SDL_GPUTexture, ...}

missing_types = referenced_types - defined_types
              = {SDL_Window, SDL_Rect, SDL_FColor, SDL_FlipMode, 
                 SDL_PropertiesID, SDL_GPUShaderFormat}

Output: Array of 6 strings (owned by caller)

---

Phase 7: Include Header Parsing

File: src/parser.zig::main() continued

    if (missing_types.len > 0) {
        // 8. Parse #include directives from source
        const includes = try dependency_resolver.parseIncludes(allocator, source);
        defer {
            for (includes) |inc| allocator.free(inc);
            allocator.free(includes);
        }

File: src/dependency_resolver.zig::parseIncludes()

pub fn parseIncludes(allocator: Allocator, source: []const u8) ![][]const u8 {
    var includes = std.ArrayList([]const u8){};
    
    var lines = std.mem.splitScalar(u8, source, '\n');
    while (lines.next()) |line| {
        const trimmed = std.mem.trim(u8, line, " \t\r");
        
        // Match: #include <SDL3/SDL_something.h>
        if (std.mem.startsWith(u8, trimmed, "#include <SDL3/")) {
            const after_open = "#include <SDL3/".len;
            if (std.mem.indexOf(u8, trimmed[after_open..], ">")) |end| {
                const header_name = trimmed[after_open..][0..end];
                try includes.append(allocator, try allocator.dupe(u8, header_name));
            }
        }
    }
    
    return try includes.toOwnedSlice(allocator);
}

Example: From SDL_gpu.h header:

#include <SDL3/SDL_stdinc.h>
#include <SDL3/SDL_pixels.h>
#include <SDL3/SDL_properties.h>
#include <SDL3/SDL_rect.h>
#include <SDL3/SDL_surface.h>
#include <SDL3/SDL_video.h>

Output: Array of strings:

["SDL_stdinc.h", "SDL_pixels.h", "SDL_properties.h", 
 "SDL_rect.h", "SDL_surface.h", "SDL_video.h"]

---

Phase 8: Dependency Type Extraction

File: src/parser.zig::main() continued

        // 9. Determine header directory
        const header_dir = std.fs.path.dirname(header_path) orelse ".";
        // e.g., "../SDL/include/SDL3"
        
        var dependency_decls = std.ArrayList(patterns.Declaration){};
        defer {
            for (dependency_decls.items) |dep_decl| {
                freeDeclDeep(allocator, dep_decl);
            }
            dependency_decls.deinit(allocator);
        }
        
        // 10. For each missing type, search dependency headers
        for (missing_types) |missing_type| {
            var found = false;
            
            // Try each included header
            for (includes) |include| {
                // 10a. Build full path
                const dep_path = try std.fs.path.join(
                    allocator,
                    &[_][]const u8{ header_dir, include }
                );
                defer allocator.free(dep_path);
                // e.g., "../SDL/include/SDL3/SDL_pixels.h"
                
                // 10b. Read dependency header
                const dep_source = std.fs.cwd().readFileAlloc(
                    allocator,
                    dep_path,
                    10 * 1024 * 1024
                ) catch continue;  // Skip if can't read
                defer allocator.free(dep_source);
                
                // 10c. Extract type from this header
                if (try dependency_resolver.extractTypeFromHeader(
                    allocator, 
                    dep_source, 
                    missing_type
                )) |dep_decl| {
                    try dependency_decls.append(allocator, dep_decl);
                    std.debug.print("  ✓ Found {s} in {s}\n", 
                        .{missing_type, include});
                    found = true;
                    break;  // Found it, stop searching
                }
            }
            
            if (!found) {
                std.debug.print("  ⚠ Warning: Could not find {s}\n", 
                    .{missing_type});
            }
        }

Search Algorithm:

For missing_type "SDL_Window":
  Try SDL_stdinc.h     -> Not found
  Try SDL_pixels.h     -> Not found
  Try SDL_properties.h -> Not found
  Try SDL_rect.h       -> Not found
  Try SDL_surface.h    -> Not found
  Try SDL_video.h      -> FOUND! ✓

---

Phase 9: Type Extraction from Header

File: src/dependency_resolver.zig::extractTypeFromHeader()

pub fn extractTypeFromHeader(
    allocator: Allocator,
    header_source: []const u8,
    type_name: []const u8,  // e.g., "SDL_Window"
) !?Declaration {
    // 1. Parse the entire dependency header
    var scanner = patterns.Scanner.init(allocator, header_source);
    const all_decls = try scanner.scan();
    defer {
        for (all_decls) |decl| {
            freeDeclaration(allocator, decl);
        }
        allocator.free(all_decls);
    }
    
    // 2. Search for matching type
    for (all_decls) |decl| {
        const decl_name = switch (decl) {
            .opaque_type => |o| o.name,
            .enum_decl => |e| e.name,
            .struct_decl => |s| s.name,
            .flag_decl => |f| f.name,
            else => continue,
        };
        
        // 3. Found it!
        if (std.mem.eql(u8, decl_name, type_name)) {
            // 4. Deep clone so caller owns it
            return try cloneDeclaration(allocator, decl);
        }
    }
    
    return null;  // Not found in this header
}

Example: Searching SDL_video.h for SDL_Window

1. Parse SDL_video.h → 50+ declarations
2. Iterate through all declarations
3. Find: .opaque_type = { .name = "SDL_Window", ... }
4. Clone the declaration (deep copy all strings)
5. Return the clone
6. Free all the temporary declarations from parsing

Cloning Process:

fn cloneDeclaration(allocator: Allocator, decl: Declaration) !Declaration {
    return switch (decl) {
        .opaque_type => |o| .{
            .opaque_type = .{
                .name = try allocator.dupe(u8, o.name),  // Own the string
                .doc_comment = if (o.doc_comment) |doc| 
                    try allocator.dupe(u8, doc) else null,
            },
        },
        // ... similar for enum, struct, flags
    };
}

Why clone? The parsed declarations from scanner.scan() are freed after this function returns. We need owned copies that live until code generation.

---

Phase 10: Declaration Combining

File: src/parser.zig::main() continued

        // 11. Combine dependency declarations with primary
        var all_decls = std.ArrayList(patterns.Declaration){};
        defer all_decls.deinit(allocator);
        
        // IMPORTANT: Dependencies FIRST!
        try all_decls.appendSlice(allocator, dependency_decls.items);
        try all_decls.appendSlice(allocator, decls);

Result: Combined array

all_decls = [
    // Dependencies (4 items)
    { .struct_decl = SDL_FColor },
    { .enum_decl = SDL_FlipMode },
    { .struct_decl = SDL_Rect },
    { .opaque_type = SDL_Window },
    
    // Primary header (169 items)
    { .opaque_type = SDL_GPUDevice },
    { .enum_decl = SDL_GPUPrimitiveType },
    ... (167 more)
]

Why dependencies first? Types must be defined before they're used. Since primary header references dependency types, dependencies must come first.

---

Phase 11: Code Generation

File: src/parser.zig::main() continued

        // 12. Generate Zig code from all declarations
        const output = try codegen.CodeGen.generate(allocator, all_decls.items);
        defer allocator.free(output);

File: src/codegen.zig::CodeGen.generate() (simplified)

pub fn generate(allocator: Allocator, decls: []const Declaration) ![]const u8 {
    var buf = std.ArrayList(u8){};
    
    // Header
    try buf.appendSlice(allocator, "pub const c = @import(\"c.zig\").c;\n\n");
    
    // Generate each declaration
    for (decls) |decl| {
        switch (decl) {
            .opaque_type => |o| {
                try buf.appendSlice(allocator, "pub const ");
                try buf.appendSlice(allocator, stripSDLPrefix(o.name));
                try buf.appendSlice(allocator, " = opaque {};\n");
            },
            .struct_decl => |s| {
                try generateStruct(allocator, &buf, s);
            },
            // ... other types
        }
    }
    
    return try buf.toOwnedSlice(allocator);
}

Output (excerpt):

pub const c = @import("c.zig").c;

pub const FColor = extern struct {
    r: f32,
    g: f32,
    b: f32,
    a: f32,
};

pub const Window = opaque {};

pub const GPUDevice = opaque {
    pub inline fn windowSupportsGPU(
        gpudevice: *GPUDevice,
        window: ?*Window,  // ✓ Window is defined above!
    ) bool {
        return c.SDL_WindowSupportsGPUDevice(gpudevice, window);
    }
};

---

Phase 12: AST Validation & Formatting

File: src/parser.zig::main() continued

        // 13. Parse generated code as Zig AST
        const output_z = try allocator.dupeZ(u8, output);
        defer allocator.free(output_z);
        
        var ast = try std.zig.Ast.parse(allocator, output_z, .zig);
        defer ast.deinit(allocator);
        
        // 14. Check for syntax errors
        if (ast.errors.len > 0) {
            std.debug.print("\nError: {d} syntax errors\n", .{ast.errors.len});
            for (ast.errors) |err| {
                const loc = ast.tokenLocation(0, err.token);
                std.debug.print("  Line {d}: {s}\n", 
                    .{ loc.line + 1, @tagName(err.tag) });
            }
            return error.InvalidSyntax;
        }
        
        // 15. Format using Zig's formatter
        const formatted_output = try ast.renderAlloc(allocator);
        defer allocator.free(formatted_output);

Why validate? Catch codegen bugs early. If generated code doesn't parse, we know immediately.

Why format? Zig's formatter ensures consistent style, proper indentation, and canonical formatting.

---

Phase 13: Output Writing

File: src/parser.zig::main() continued

        // 16. Write to file or stdout
        if (output_file) |file_path| {
            try std.fs.cwd().writeFile(.{
                .sub_path = file_path,
                .data = formatted_output,
            });
            std.debug.print("Generated: {s}\n", .{file_path});
        } else {
            _ = try std.posix.write(std.posix.STDOUT_FILENO, formatted_output);
        }

---

Memory Management Flow

Allocations

1. Primary header source: Freed at end of main()
2. Primary declarations: Freed at end of main() (with deep free)
3. Dependency resolver HashMaps: Freed in resolver.deinit()
4. HashMap keys (in referenced_types): Freed in resolver.deinit()
5. Missing types array: Freed explicitly after use
6. Includes array: Freed explicitly after use
7. Dependency header sources: Freed immediately after extraction
8. Temporary parsed declarations: Freed immediately in extractTypeFromHeader()
9. Cloned dependency declarations: Freed at end of scope
10. Generated output: Freed after writing
11. Formatted output: Freed after writing

Ownership Rules

• Scanner owns strings during parsing (from its allocator)
• Cloned declarations own strings after extraction (allocated explicitly)
• HashMap owns keys in referenced_types (duped when inserted)
• Caller owns result of getMissingTypes(), parseIncludes()

---

Error Handling Flow

Errors That Fail

// Fatal errors - exit immediately
- File not found (primary header)
- Out of memory
- Invalid syntax in generated code (optional)

Errors That Warn

// Warnings - continue execution
- Dependency header not readable → continue with next header
- Type not found in any header → print warning, continue
- Struct parsing errors → generate partial output

Example Error Flow

Parse SDL_gpu.h
  ↓
Missing type: SDL_Window
  ↓
Try SDL_pixels.h → catch FileNotFound → continue
Try SDL_video.h → Success! → break
  ↓
Missing type: SDL_Unknown
  ↓
Try all headers → Not found → print warning
  ↓
Continue with partial results

---

Performance Characteristics

Time Complexity

• Primary parsing: O(n) where n = source lines
• Type collection: O(d) where d = declarations
• Missing type detection: O(r) where r = referenced types
• Type extraction: O(h × d) where h = headers, d = declarations per header
• Overall: O(n + d + r + h×d) ≈ O(n) for typical cases

Space Complexity

• Primary declarations: O(d)
• Dependency declarations: O(m) where m = missing types
• HashMaps: O(t) where t = total unique types
• Peak memory: ~2-5MB for SDL_gpu.h

Optimization Points

1. Cache parsed headers - Currently re-parse for each missing type
2. Early exit - Stop searching after finding type
3. String interning - Deduplicate type name strings
4. Lazy loading - Only parse dependencies if missing types detected

---

Testing the Flow

Unit Test Example

test "complete dependency flow" {
    const source = 
        \\typedef struct SDL_Type SDL_Type;
        \\extern void SDL_Func(SDL_External *param);
    ;
    
    // Phase 1: Parse
    var scanner = Scanner.init(allocator, source);
    const decls = try scanner.scan();
    
    // Phase 2: Analyze
    var resolver = DependencyResolver.init(allocator);
    defer resolver.deinit();
    try resolver.analyze(decls);
    
    // Phase 3: Get missing
    const missing = try resolver.getMissingTypes(allocator);
    defer allocator.free(missing);
    
    // Verify: SDL_External is missing
    try testing.expectEqual(@as(usize, 1), missing.len);
    try testing.expectEqualStrings("SDL_External", missing[0]);
}

Integration Test

# Create test header with dependency
echo 'typedef struct Dep Dep;' > dep.h
echo '#include "dep.h"' > main.h
echo 'void func(Dep *d);' >> main.h

# Parse with dependency resolution
zig build run -- main.h --output=out.zig

# Verify output contains Dep
grep 'pub const Dep' out.zig

---

Summary

The dependency resolution flow is:

1. Parse primary header → get declarations
2. Analyze declarations → find referenced vs defined types
3. Calculate missing = referenced - defined
4. Extract #include directives from source
5. Search dependency headers for missing types
6. Clone found declarations (deep copy)
7. Combine dependency + primary declarations
8. Generate Zig code with all types
9. Validate and format using Zig AST
10. Output to file or stdout

Each phase has clear inputs/outputs, proper memory management, and graceful error handling.