Backlog/lib/sdl3/parser/docs/architecture.md

Architecture

The SDL3 Parser is a multi-stage pipeline that transforms C header declarations into idiomatic Zig code.

Pipeline Overview

┌─────────────┐
│ C Header    │
│ (SDL_gpu.h) │
└──────┬──────┘
       │
       v
┌─────────────────────────────────────────┐
│  Stage 1: Lexical Scanning (Scanner)    │
│  - Read source file                     │
│  - Skip whitespace & comments           │
│  - Extract doc comments                 │
└──────┬──────────────────────────────────┘
       │
       v
┌─────────────────────────────────────────┐
│  Stage 2: Pattern Matching              │
│  - scanOpaque()                         │
│  - scanEnum()                           │
│  - scanStruct()                         │
│  - scanFlagTypedef()                    │
│  - scanFunction()                       │
└──────┬──────────────────────────────────┘
       │
       v
┌─────────────────────────────────────────┐
│  Stage 3: Naming Conversion             │
│  - detectCommonPrefix()                 │
│  - enumValueToZig()                     │
│  - typeNameToZig()                      │
│  - functionNameToZig()                  │
└──────┬──────────────────────────────────┘
       │
       v
┌─────────────────────────────────────────┐
│  Stage 4: Code Generation               │
│  - Generate type declarations           │
│  - Generate inline functions            │
│  - Add proper casts & annotations       │
└──────┬──────────────────────────────────┘
       │
       v
┌─────────────┐
│ Zig Code    │
│ (gpu.zig)   │
└─────────────┘

Components

1. Scanner (patterns.zig)

Purpose: Tokenize and extract C declarations from source.

Key Functions:

• scan() - Main entry point, returns array of declarations
• scanOpaque() - Matches typedef struct X X;
• scanEnum() - Matches typedef enum { ... } X;
• scanStruct() - Matches typedef struct { ... } X;
• scanFlagTypedef() - Matches typedef Uint32 XFlags; + #define lines
• scanFunction() - Matches extern SDL_DECLSPEC ... SDLCALL X(...);

Key Helpers:

• skipWhitespace() - Skip whitespace/newlines (critical for flag parsing)
• peekDocComment() - Extract /** ... */ documentation
• readBracedBlock() - Read { ... } blocks with nesting support

Data Structures:

pub const Declaration = union(enum) {
    opaque_type: OpaqueType,
    enum_decl: EnumDecl,
    struct_decl: StructDecl,
    flag_decl: FlagDecl,
    function_decl: FunctionDecl,
};

2. Naming (naming.zig)

Purpose: Convert C naming conventions to Zig idioms.

Key Algorithm - "First Underscore Rule":

// Input: SDL_GPU_PRIMITIVETYPE_TRIANGLELIST
// 1. Strip prefix: PRIMITIVETYPE_TRIANGLELIST
// 2. Find first underscore at position 13
// 3. Split: PRIMITIVETYPE + TRIANGLELIST
// 4. Convert: primitivetype + Trianglelist
// 5. Result: primitivetypeTrianglelist

Key Functions:

• detectCommonPrefix() - Returns SDL_GPU_ or SDL_ (NOT type name)
• enumValueToZig() - Applies first underscore rule
• typeNameToZig() - Strips SDL prefix: SDL_GPUDevice → GPUDevice
• functionNameToZig() - Lowercases leading acronyms: SDL_CreateGPUDevice → createGPUDevice

Rationale for First Underscore:

• Prevents invalid identifiers starting with numbers (2d → texturetype2d)
• Preserves semantic meaning (type + value)
• Handles multi-word values correctly (2D_ARRAY → 2dArray)

3. Types (types.zig)

Purpose: Map C types to Zig types.

Type Mappings:

C Type          →  Zig Type
─────────────────────────────────
bool            →  bool
int             →  c_int
unsigned int    →  c_uint
float           →  f32
double          →  f64
char *          →  [*:0]const u8
void *          →  ?*anyopaque
const T *       →  *const T
T *             →  *T
Uint32          →  u32
Sint64          →  i64

Cast Types:

• .ptr_cast - For pointer conversions
• .bit_cast - For flag/enum conversions
• .int_from_enum - For enum to int
• .enum_from_int - For int to enum

4. CodeGen (codegen.zig)

Purpose: Generate final Zig code with proper formatting.

Generation Strategy:

Opaque Types:

pub const GPUDevice = opaque {};

Enums:

pub const GPUPrimitiveType = enum(c_int) {
    primitivetypeTrianglelist,
    primitivetypeTrianglestrip,
};

Flags (Packed Structs):

pub const GPUTextureUsageFlags = packed struct(u32) {
    textureusageSampler: bool = false,
    textureusageColorTarget: bool = false,
    // ... more flags
    pad0: u24 = 0,  // Calculated padding
    rsvd: bool = false,  // Reserved bit
};

Functions (Inline Wrappers):

pub inline fn createGPUDevice(debug_mode: bool) ?*GPUDevice {
    return @ptrCast(c.SDL_CreateGPUDevice(debug_mode));
}

Why Inline Functions?

• Zero overhead (inlined away at compile time)
• Type-safe wrappers around C calls
• Automatic cast insertion
• Better error messages

Critical Implementation Details

Flag Parsing Bug Fix

Problem: After reading typedef Uint32 SDL_GPUTextureUsageFlags;, scanner position is at newline. Calling matchPrefix("#define ") immediately fails.

Solution: Call skipWhitespace() before checking for #define statements.

// In scanFlagTypedef()
var flags = try std.ArrayList(FlagValue).initCapacity(self.allocator, 10);

self.skipWhitespace();  // <-- CRITICAL: Skip newlines

while (!self.isAtEnd()) {
    if (!self.matchPrefix("#define ")) break;
    // ... parse flag
}

Invalid Identifier Fix

Problem: Using "last underscore" rule on SDL_GPU_TEXTURETYPE_2D_ARRAY splits as:

• Type: TEXTURETYPE_2D
• Value: ARRAY
• Result: texturetype2dArray ✓ Valid but wrong semantics

Using "last underscore" on SDL_GPU_SAMPLECOUNT_1 splits as:

• Type: SAMPLECOUNT
• Value: 1
• Result: samplecount1 ✓ But "first underscore" gives same result

The key insight: Always use first underscore after prefix. This keeps type name intact and prevents semantic errors.

Memory Management

Allocation Points:

1. Source file read (readFileAlloc)
2. Declaration storage (ArrayList)
3. String duplication (allocator.dupe)
4. Doc comments (allocator.dupe)

Cleanup Strategy:

• Use arena allocator in tests (automatic cleanup)
• Manual cleanup in main with defer blocks
• Free doc comments in declaration cleanup
• Free pending_doc_comment when skipping lines

GPA Verification:

zig build run -- SDL_gpu.h 2>&1 | grep -i leak
# Output: (empty = no leaks)

Performance Characteristics

• Time Complexity: O(n) where n = source file size
• Memory: O(d) where d = number of declarations
• Typical Parse Time: <500ms for SDL_gpu.h (169 declarations)
• Memory Usage: ~5MB peak for SDL_gpu.h

Extension Points

To add support for new C patterns:

1. Add pattern matcher in patterns.zig:

   fn scanNewPattern(self: *Scanner) !?NewDecl { ... }
   

2. Add naming converter in naming.zig:

   pub fn newPatternToZig(c_name: []const u8) []const u8 { ... }
   

3. Add code generator in codegen.zig:

   fn writeNewPattern(self: *CodeGen, decl: NewDecl) !void { ... }
   

4. Add to Declaration union:

   pub const Declaration = union(enum) {
       // ... existing
       new_pattern: NewDecl,
   };
   

Testing Strategy

Unit Tests: Test individual components in isolation

• Scanner tests: Verify pattern matching
• Naming tests: Verify conversion rules
• CodeGen tests: Verify output formatting

Integration Tests: Test complete pipeline

• Parse real SDL3 headers
• Verify output compiles
• Check declaration counts

Regression Tests (planned):

• Golden file comparison
• Detect unintended changes

See Test Harness Plan for future testing infrastructure.