Backlog/lib/sdl3/parser/docs/architecture.md

# Architecture

The SDL3 Parser is a multi-stage pipeline that transforms C header declarations into idiomatic Zig code.

## Pipeline Overview

```
┌─────────────┐
│ C Header    │
│ (SDL_gpu.h) │
└──────┬──────┘
       │
       v
┌─────────────────────────────────────────┐
│  Stage 1: Lexical Scanning (Scanner)    │
│  - Read source file                     │
│  - Skip whitespace & comments           │
│  - Extract doc comments                 │
└──────┬──────────────────────────────────┘
       │
       v
┌─────────────────────────────────────────┐
│  Stage 2: Pattern Matching              │
│  - scanOpaque()                         │
│  - scanEnum()                           │
│  - scanStruct()                         │
│  - scanFlagTypedef()                    │
│  - scanFunction()                       │
└──────┬──────────────────────────────────┘
       │
       v
┌─────────────────────────────────────────┐
│  Stage 3: Naming Conversion             │
│  - detectCommonPrefix()                 │
│  - enumValueToZig()                     │
│  - typeNameToZig()                      │
│  - functionNameToZig()                  │
└──────┬──────────────────────────────────┘
       │
       v
┌─────────────────────────────────────────┐
│  Stage 4: Code Generation               │
│  - Generate type declarations           │
│  - Generate inline functions            │
│  - Add proper casts & annotations       │
└──────┬──────────────────────────────────┘
       │
       v
┌─────────────┐
│ Zig Code    │
│ (gpu.zig)   │
└─────────────┘
```

## Components

### 1. Scanner (patterns.zig)

**Purpose**: Tokenize and extract C declarations from source.

**Key Functions**:
- `scan()` - Main entry point, returns array of declarations
- `scanOpaque()` - Matches `typedef struct X X;`
- `scanEnum()` - Matches `typedef enum { ... } X;`
- `scanStruct()` - Matches `typedef struct { ... } X;`
- `scanFlagTypedef()` - Matches `typedef Uint32 XFlags;` + `#define` lines
- `scanFunction()` - Matches `extern SDL_DECLSPEC ... SDLCALL X(...);`

**Key Helpers**:
- `skipWhitespace()` - Skip whitespace/newlines (critical for flag parsing)
- `peekDocComment()` - Extract `/** ... */` documentation
- `readBracedBlock()` - Read `{ ... }` blocks with nesting support

**Data Structures**:
```zig
pub const Declaration = union(enum) {
    opaque_type: OpaqueType,
    enum_decl: EnumDecl,
    struct_decl: StructDecl,
    flag_decl: FlagDecl,
    function_decl: FunctionDecl,
};
```

### 2. Naming (naming.zig)

**Purpose**: Convert C naming conventions to Zig idioms.

**Key Algorithm - "First Underscore Rule"**:

```zig
// Input: SDL_GPU_PRIMITIVETYPE_TRIANGLELIST
// 1. Strip prefix: PRIMITIVETYPE_TRIANGLELIST
// 2. Find first underscore at position 13
// 3. Split: PRIMITIVETYPE + TRIANGLELIST
// 4. Convert: primitivetype + Trianglelist
// 5. Result: primitivetypeTrianglelist
```

**Key Functions**:
- `detectCommonPrefix()` - Returns `SDL_GPU_` or `SDL_` (NOT type name)
- `enumValueToZig()` - Applies first underscore rule
- `typeNameToZig()` - Strips SDL prefix: `SDL_GPUDevice` → `GPUDevice`
- `functionNameToZig()` - Lowercases leading acronyms: `SDL_CreateGPUDevice` → `createGPUDevice`

**Rationale for First Underscore**:
- Prevents invalid identifiers starting with numbers (`2d` → `texturetype2d`)
- Preserves semantic meaning (type + value)
- Handles multi-word values correctly (`2D_ARRAY` → `2dArray`)

### 3. Types (types.zig)

**Purpose**: Map C types to Zig types.

**Type Mappings**:
```zig
C Type          →  Zig Type
─────────────────────────────────
bool            →  bool
int             →  c_int
unsigned int    →  c_uint
float           →  f32
double          →  f64
char *          →  [*:0]const u8
void *          →  ?*anyopaque
const T *       →  *const T
T *             →  *T
Uint32          →  u32
Sint64          →  i64
```

**Cast Types**:
- `.ptr_cast` - For pointer conversions
- `.bit_cast` - For flag/enum conversions
- `.int_from_enum` - For enum to int
- `.enum_from_int` - For int to enum

### 4. CodeGen (codegen.zig)

**Purpose**: Generate final Zig code with proper formatting.

**Generation Strategy**:

**Opaque Types**:
```zig
pub const GPUDevice = opaque {};
```

**Enums**:
```zig
pub const GPUPrimitiveType = enum(c_int) {
    primitivetypeTrianglelist,
    primitivetypeTrianglestrip,
};
```

**Flags (Packed Structs)**:
```zig
pub const GPUTextureUsageFlags = packed struct(u32) {
    textureusageSampler: bool = false,
    textureusageColorTarget: bool = false,
    // ... more flags
    pad0: u24 = 0,  // Calculated padding
    rsvd: bool = false,  // Reserved bit
};
```

**Functions (Inline Wrappers)**:
```zig
pub inline fn createGPUDevice(debug_mode: bool) ?*GPUDevice {
    return @ptrCast(c.SDL_CreateGPUDevice(debug_mode));
}
```

**Why Inline Functions?**
- Zero overhead (inlined away at compile time)
- Type-safe wrappers around C calls
- Automatic cast insertion
- Better error messages

## Critical Implementation Details

### Flag Parsing Bug Fix

**Problem**: After reading `typedef Uint32 SDL_GPUTextureUsageFlags;`, scanner position is at newline. Calling `matchPrefix("#define ")` immediately fails.

**Solution**: Call `skipWhitespace()` before checking for `#define` statements.

```zig
// In scanFlagTypedef()
var flags = try std.ArrayList(FlagValue).initCapacity(self.allocator, 10);

self.skipWhitespace();  // <-- CRITICAL: Skip newlines

while (!self.isAtEnd()) {
    if (!self.matchPrefix("#define ")) break;
    // ... parse flag
}
```

### Invalid Identifier Fix

**Problem**: Using "last underscore" rule on `SDL_GPU_TEXTURETYPE_2D_ARRAY` splits as:
- Type: `TEXTURETYPE_2D`
- Value: `ARRAY`
- Result: `texturetype2dArray` ✓ Valid but wrong semantics

Using "last underscore" on `SDL_GPU_SAMPLECOUNT_1` splits as:
- Type: `SAMPLECOUNT`
- Value: `1`
- Result: `samplecount1` ✓ But "first underscore" gives same result

The key insight: **Always use first underscore after prefix**. This keeps type name intact and prevents semantic errors.

### Memory Management

**Allocation Points**:
1. Source file read (`readFileAlloc`)
2. Declaration storage (`ArrayList`)
3. String duplication (`allocator.dupe`)
4. Doc comments (`allocator.dupe`)

**Cleanup Strategy**:
- Use arena allocator in tests (automatic cleanup)
- Manual cleanup in main with defer blocks
- Free doc comments in declaration cleanup
- Free pending_doc_comment when skipping lines

**GPA Verification**:
```bash
zig build run -- SDL_gpu.h 2>&1 | grep -i leak
# Output: (empty = no leaks)
```

## Performance Characteristics

- **Time Complexity**: O(n) where n = source file size
- **Memory**: O(d) where d = number of declarations
- **Typical Parse Time**: <500ms for SDL_gpu.h (169 declarations)
- **Memory Usage**: ~5MB peak for SDL_gpu.h

## Extension Points

To add support for new C patterns:

1. **Add pattern matcher** in `patterns.zig`:
   ```zig
   fn scanNewPattern(self: *Scanner) !?NewDecl { ... }
   ```

2. **Add naming converter** in `naming.zig`:
   ```zig
   pub fn newPatternToZig(c_name: []const u8) []const u8 { ... }
   ```

3. **Add code generator** in `codegen.zig`:
   ```zig
   fn writeNewPattern(self: *CodeGen, decl: NewDecl) !void { ... }
   ```

4. **Add to Declaration union**:
   ```zig
   pub const Declaration = union(enum) {
       // ... existing
       new_pattern: NewDecl,
   };
   ```

## Testing Strategy

**Unit Tests**: Test individual components in isolation
- Scanner tests: Verify pattern matching
- Naming tests: Verify conversion rules
- CodeGen tests: Verify output formatting

**Integration Tests**: Test complete pipeline
- Parse real SDL3 headers
- Verify output compiles
- Check declaration counts

**Regression Tests** (planned):
- Golden file comparison
- Detect unintended changes

See [Test Harness Plan](../TEST_HARNESS_PLAN_V2.md) for future testing infrastructure.