Backlog/lib/sdl3/research/zig-skills.md

622 lines
21 KiB
Markdown

# Zig 0.15 Skills and Gotchas
This document tracks common issues, syntax changes, and gotchas encountered when working with Zig 0.15 during the SDL3 parser implementation.
## Work Summary
Successfully implemented a fully functional SDL3 C header parser in Zig that converts SDL3 C headers into idiomatic Zig bindings. The parser uses a simplified text-matching approach (no full C parser) and successfully handles:
- **Opaque types**: `typedef struct SDL_GPUDevice SDL_GPUDevice;``pub const GPUDevice = opaque {};`
- **Enums**: C enums with value detection → Zig enums with camelCase values
- **Structs**: C structs with field parsing → Zig extern structs with converted types
- **Flags**: C typedef + #define flags → Zig packed structs
- **Functions**: C function declarations → Zig inline wrapper functions with proper casts
**Key Achievements**:
- 4 core modules: patterns.zig (700+ lines), naming.zig (130+ lines), types.zig (88 lines), codegen.zig (339 lines)
- All 14 tests passing
- Zero memory leaks after fixes
- Successfully parses test headers and generates valid Zig code
- Proper handling of SDL naming conventions (SDL_GPUDevice → GPUDevice, SDL_CreateGPUDevice → createGPUDevice)
**Major Bugs Fixed**:
1. Memory leaks from readLine() allocations - fixed with defer
2. Function name conversion bug (gPUSupportsShaderFormats) - fixed acronym lowercasing logic
3. Enum/struct parsing broken - fixed matchPrefix + readLine interaction
4. Brace characters appearing in parsed values - added brace line filtering
## Build System Changes
### root_module vs root_source_file
**Issue**: In Zig 0.15, the build system API changed from `root_source_file` to `root_module`.
**Old (pre-0.15)**:
```zig
const parser_exe = b.addExecutable(.{
.name = "sdl-parser",
.root_source_file = b.path("parser.zig"),
.target = target,
.optimize = optimize,
});
```
**New (0.15+)**:
```zig
const parser_exe = b.addExecutable(.{
.name = "sdl-parser",
.root_module = b.createModule(.{
.root_source_file = b.path("parser.zig"),
.target = target,
.optimize = optimize,
}),
});
```
**Solution**: Use `root_module = b.createModule(.{...})` instead of passing fields directly.
## ArrayList API Changes
### Allocator Required for All Methods
**Issue**: `ArrayList` methods now require passing the allocator explicitly, not just at initialization.
**Old**:
```zig
var list = std.ArrayList(u8).init(allocator);
try list.append('x');
const slice = list.toOwnedSlice();
list.deinit();
```
**New (0.15+)**:
```zig
var list = try std.ArrayList(u8).initCapacity(allocator, initial_capacity);
try list.append(allocator, 'x');
const slice = try list.toOwnedSlice(allocator);
list.deinit(allocator);
```
**Solution**: Pass allocator to `append()`, `toOwnedSlice()`, `deinit()`, and use `initCapacity()` instead of `init()`.
**Files affected**:
- `naming.zig`: All ArrayList operations
- `patterns.zig`: String building and result accumulation
## Reserved Keywords
### opaque is Reserved
**Issue**: `opaque` is a reserved keyword in Zig and cannot be used as an identifier.
**Error**:
```zig
// ❌ This fails
fn scanOpaque() !?OpaqueType {
if (condition) |opaque| { // ERROR: 'opaque' is an identifier
return opaque;
}
}
// ❌ This also fails
fn writeOpaque(self: *Self, opaque: OpaqueType) !void {
^~~~~~ // ERROR: expected '{', found ':'
}
// ❌ Even in tests
test "opaque type" {
const opaque = OpaqueType{...}; // ERROR: expected 'an identifier', found 'opaque'
}
```
**Solution**: Use alternative names like `opaque_type`, `opaque_decl`, `opaque_val`:
```zig
// ✅ Correct
fn scanOpaque() !?OpaqueType {
if (condition) |opaque_decl| {
return opaque_decl;
}
}
fn writeOpaque(self: *Self, opaque_type: OpaqueType) !void {
// ...
}
test "opaque type" {
const opaque_type = OpaqueType{...};
}
```
**Files affected**:
- `patterns.zig:92`: Capture variable renamed to `opaque_decl`
- `codegen.zig:44`: Capture variable renamed to `opaque_decl`
- `codegen.zig:53`: Parameter renamed to `opaque_type`
- `codegen.zig:247`: Test variable renamed to `opaque_type`
## Compiler Strictness
### Unused Variables are Errors
**Issue**: Zig 0.15 treats unused variables as compilation errors, not warnings.
**Error**:
```zig
const value = getSomething(); // ERROR: unused local variable
```
**Solutions**:
1. Use the variable
2. Assign to `_` if intentionally unused:
```zig
_ = getSomething();
```
3. Prefix with underscore for parameters:
```zig
fn callback(_unused: u32) void {}
```
### Error Unions Must be Handled
**Issue**: Functions returning error unions must have errors explicitly handled.
**Error**:
```zig
const line = self.readLine(); // ERROR: error union not handled
// readLine() returns ![]const u8
```
**Solution**: Use `try` or explicit error handling:
```zig
const line = try self.readLine(); // ✅ Correct
// Or handle explicitly
const line = self.readLine() catch |err| {
return err;
};
```
**Files affected**:
- `patterns.zig`: All `readLine()` calls needed `try`
## Memory Management
### Arena Allocator for Tests
**Issue**: Using direct allocation in tests can cause memory leak errors that are hard to track.
**Problem**:
```zig
test "something" {
var gpa = std.testing.allocator;
const name = try gpa.alloc(u8, 10);
// ... use name ...
// If test fails before freeing, GPA reports leak
}
```
**Solution**: Use arena allocator for test allocations:
```zig
test "something" {
var arena = std.heap.ArenaAllocator.init(std.testing.allocator);
defer arena.deinit();
const allocator = arena.allocator();
const name = try allocator.alloc(u8, 10);
// ... use name ...
// Arena automatically frees everything on deinit
}
```
**Files affected**:
- `patterns.zig`: Test for opaque typedef uses arena
## Type System
### Const Pointer to Array vs Slice
**Issue**: `&array` creates a const pointer to array, not a slice.
**Error**:
```zig
const decls = [_]Declaration{...};
function(&decls); // ERROR: expected '[]Declaration', found '*const [1]Declaration'
```
**Solution**: Use array slice syntax `&array` for mutable or `decls[0..]` for explicit slice:
```zig
const decls = [_]Declaration{...};
function(decls[0..]); // ✅ Explicit slice
// Or if function accepts const:
function(&decls); // Works if function parameter is '[]const Declaration'
```
## Parser Implementation Challenges
### matchPrefix() Consumes Input
**Problem**: After calling `matchPrefix("typedef enum ")`, the scanner position has moved past the prefix. Calling `readLine()` immediately after will read from the new position, not from the start of the line.
**Example**:
```zig
// Source: "typedef enum SDL_GPUPrimitiveType {"
if (self.matchPrefix("typedef enum ")) { // pos moves to after "typedef enum "
const line = try self.readLine(); // Reads "SDL_GPUPrimitiveType {"
// BUG: Trying to tokenize expecting "typedef enum name"
var iter = std.mem.tokenizeScalar(u8, line, ' ');
_ = iter.next(); // Expects "typedef" - NOT THERE
_ = iter.next(); // Expects "enum" - NOT THERE
const name = iter.next(); // Gets "SDL_GPUPrimitiveType" but after skipping non-existent tokens
}
```
**Solution**: Don't use `readLine()` after `matchPrefix()`. Instead, scan forward to find landmarks (like opening brace), extract what you need from the source slice directly:
```zig
if (self.matchPrefix("typedef enum ")) {
// Find the opening brace
const name_start = self.pos;
while (self.pos < self.source.len and self.source[self.pos] != '{') {
self.pos += 1;
}
// Extract name from the slice between matchPrefix and '{'
const name_slice = std.mem.trim(u8, self.source[name_start..self.pos], " \t\n\r");
var iter = std.mem.tokenizeScalar(u8, name_slice, ' ');
const name = iter.next() orelse return null;
// Now we're positioned at '{', which is what readBracedBlock() expects
const body = try self.readBracedBlock();
}
```
**Files affected**:
- `patterns.zig:scanEnum()` - Fixed to scan for '{' instead of using readLine()
- `patterns.zig:scanStruct()` - Applied same fix
### readBracedBlock() Returns Full Source Including Braces
**Problem**: `readBracedBlock()` returns the entire source from current position to the end of the closing brace, including the braces themselves and any typedef name after the closing brace.
**Example**:
```zig
// Source at position: "{ VALUE1, VALUE2 } TypeName;"
const body = try self.readBracedBlock();
// body = "{ VALUE1, VALUE2 } TypeName;"
// ^ ^^^^^^^^^^ - includes closing brace and typedef name
```
When splitting by newlines and parsing, you get lines like:
- `"{"`
- `"VALUE1,"`
- `"VALUE2"`
- `"} TypeName;"`
**Solution**: Filter out lines that start with braces when parsing the body:
```zig
var lines = std.mem.splitScalar(u8, body, '\n');
while (lines.next()) |line| {
const trimmed = std.mem.trim(u8, line, " \t\r");
if (trimmed.len == 0) continue;
if (std.mem.startsWith(u8, trimmed, "//")) continue;
if (std.mem.startsWith(u8, trimmed, "/*")) continue;
if (std.mem.startsWith(u8, trimmed, "{")) continue; // ← Filter opening brace
if (std.mem.startsWith(u8, trimmed, "}")) continue; // ← Filter closing brace + typedef name
// Now parse the actual content
if (try self.parseEnumValue(trimmed)) |value| {
try values.append(self.allocator, value);
}
}
```
**Files affected**:
- `patterns.zig:scanEnum()` - Added brace filtering for enum values
- `patterns.zig:parseStructField()` - Added brace filtering for struct fields
### Memory Leaks from Temporary Allocations
**Problem**: Functions that allocate memory (like `readLine()`) return owned slices that must be freed. In loops, forgetting to free these causes memory leaks.
**Example**:
```zig
// BUG: Memory leak
while (!self.isAtEnd()) {
const line = try self.readLine(); // Allocates
try func_text.appendSlice(self.allocator, line);
// line is never freed - LEAK!
if (std.mem.indexOfScalar(u8, line, ';')) |_| break;
}
```
**Solution**: Use `defer` to ensure allocation is freed even if loop breaks early:
```zig
while (!self.isAtEnd()) {
const line = try self.readLine();
defer self.allocator.free(line); // ← Always freed when scope exits
try func_text.appendSlice(self.allocator, line);
if (std.mem.indexOfScalar(u8, line, ';')) |_| break;
}
```
**Files affected**:
- `patterns.zig:scanFunction()` - Added defer for readLine() calls
### Leading Acronym Lowercasing in Function Names
**Problem**: Simply lowercasing the first character of a function name doesn't work well with leading acronyms.
**Example**:
- `SDL_GPUSupportsShaderFormats` → strip "SDL_" → `GPUSupportsShaderFormats`
- Lowercase first char → `gPUSupportsShaderFormats` ❌ (should be `gpuSupportsShaderFormats`)
**Solution**: Lowercase the entire leading acronym (consecutive uppercase letters) until you hit a lowercase letter that starts a new word:
```zig
pub fn functionNameToZig(c_name: []const u8, allocator: Allocator) ![]const u8 {
const without_prefix = stripSDLPrefix(c_name);
var result = try allocator.dupe(u8, without_prefix);
// Lowercase leading acronyms (e.g., "GPUSupports" -> "gpuSupports")
var i: usize = 0;
while (i < result.len and std.ascii.isUpper(result[i])) : (i += 1) {
// If we have at least 2 uppercase letters and the next char is lowercase,
// we've found the end of the acronym (e.g., "GPUs" -> "gpu" + "Supports")
if (i > 0 and i + 1 < result.len and std.ascii.isLower(result[i + 1])) {
// Don't lowercase this last uppercase letter - it starts the next word
break;
}
result[i] = std.ascii.toLower(result[i]);
}
return result;
}
```
**Results**:
- `GPUSupportsShaderFormats``gpuSupportsShaderFormats`
- `CreateGPUDevice``createGPUDevice`
- `GPUTextureFormat``gpuTextureFormat`
**Files affected**:
- `naming.zig:functionNameToZig()` - Implemented acronym lowercasing logic
## Best Practices
1. **Always handle error unions**: Use `try` or explicit `catch`
2. **Avoid reserved keywords**: Check keyword list before naming variables
3. **Use arena allocators in tests**: Simplifies cleanup and prevents leak errors
4. **Pass allocators explicitly**: Don't assume methods have access to allocator
5. **Prefer explicit slices**: Use `array[0..]` instead of relying on coercion
6. **Check build API docs**: Build system API changes between versions
7. **Use defer for cleanup**: Especially in loops where early breaks can skip cleanup
8. **Don't mix matchPrefix with readLine**: Scanner position management requires care
9. **Filter implementation details from parsed data**: Braces, keywords, etc. aren't part of semantic content
## Common Error Messages
| Error Message | Likely Cause | Solution |
|--------------|--------------|----------|
| `expected 'an identifier', found 'opaque'` | Using reserved keyword | Rename variable |
| `expected type '[]T', found '*const [N]T'` | Array vs slice mismatch | Use `array[0..]` |
| `error union not handled` | Missing `try` or `catch` | Add error handling |
| `unused local variable` | Variable declared but not used | Use it or assign to `_` |
| `expected 2 arguments, found 1` | ArrayList API change | Pass allocator explicitly |
| `root_source_file` field doesn't exist | Old build API | Use `root_module` |
### Const Arrays in Tests Must Be Mutable for Slicing
**Issue**: When passing array slices to functions, the array must be declared with `var` not `const`, even if you're not modifying the array itself.
**Error**:
```zig
test "generate opaque type" {
const decls = [_]Declaration{...};
const output = try CodeGen.generate(allocator, decls[0..]);
// ERROR: expected type '[]Declaration', found '*const [1]Declaration'
// ERROR: cast discards const qualifier
}
```
**Solution**: Declare the array with `var`:
```zig
test "generate opaque type" {
var decls = [_]Declaration{...}; // ✅ Use var
const output = try CodeGen.generate(allocator, decls[0..]);
}
```
**Files affected**:
- `codegen.zig`: All test arrays changed from `const` to `var`
## Standard Library API Changes
### ArrayList.writer() Requires Allocator
**Issue**: The `writer()` method on ArrayList now requires an allocator parameter.
**Error**:
```zig
var list = std.ArrayList(u8).initCapacity(allocator, 256);
try list.writer().print("Hello", .{});
// ERROR: member function expected 1 argument(s), found 0
```
**Solution**: Pass the allocator to `writer()`:
```zig
var list = std.ArrayList(u8).initCapacity(allocator, 256);
try list.writer(allocator).print("Hello", .{}); // ✅ Pass allocator
```
**Files affected**:
- `codegen.zig`: All `writer()` calls updated to pass allocator
### std.io.getStdOut() Moved
**Issue**: `std.io.getStdOut()` no longer exists in Zig 0.15.
**Error**:
```zig
const stdout = std.io.getStdOut().writer();
// ERROR: root source file struct 'Io' has no member named 'getStdOut'
```
**Solution**: Use `std.posix.write()` with `std.posix.STDOUT_FILENO`:
```zig
// Old way (doesn't work):
const stdout = std.io.getStdOut().writer();
try stdout.writeAll(output);
// New way (Zig 0.15):
_ = try std.posix.write(std.posix.STDOUT_FILENO, output);
```
**Files affected**:
- `parser.zig:105`: Changed to use `std.posix.write()`
## Type Inference Issues
### Comptime Type Inference with Runtime Values
**Issue**: When using if-else chains with runtime string comparisons, the compiler may try to infer types as comptime when they should be runtime.
**Error**:
```zig
const type_bits = if (std.mem.eql(u8, underlying_type, "u8"))
8
else if (std.mem.eql(u8, underlying_type, "u16"))
16
else
32;
// ERROR: value with comptime-only type 'comptime_int' depends on runtime control flow
```
**Solution**: Explicitly annotate the type:
```zig
const type_bits: u32 = if (std.mem.eql(u8, underlying_type, "u8"))
8
else if (std.mem.eql(u8, underlying_type, "u16"))
16
else
32; // ✅ Explicit type annotation
```
**Files affected**:
- `codegen.zig:148`: Added explicit `u32` type annotation
### Bit Shift Operand Type Mismatch
**Issue**: Bit shift operations require the right operand to be exactly the right type for the shift amount.
**Error**:
```zig
var bit: u6 = 0;
while (bit < 32) : (bit += 1) {
if (val == (@as(u32, 1) << bit)) return bit;
// ERROR: expected type 'u5', found 'u6'
// NOTE: unsigned 5-bit int cannot represent all possible unsigned 6-bit values
}
```
**Explanation**: Shifting a `u32` requires a shift amount of type `u5` (since 2^5 = 32 bits). A `u6` can represent values 0-63, but only 0-31 are valid shift amounts for `u32`.
**Solution**: Cast the shift amount to the correct type:
```zig
var bit: u6 = 0;
while (bit < 32) : (bit += 1) {
if (val == (@as(u32, 1) << @as(u5, @intCast(bit)))) return bit; // ✅ Cast to u5
}
```
**Files affected**:
- `codegen.zig:238`: Added `@as(u5, @intCast(bit))` cast
## Complete List of Compilation Errors Encountered
During the SDL3 parser implementation, we encountered the following compilation errors in order:
1. **`root_source_file` field doesn't exist in build.zig**
- File: `build.zig`
- Fix: Changed to `root_module = b.createModule(...)`
2. **`init` expects 2 arguments, found 1 (ArrayList API)**
- Files: `naming.zig`, `patterns.zig`
- Fix: Changed to `initCapacity(allocator, capacity)` and passed allocator to all methods
3. **`expected 'an identifier', found 'opaque'` (reserved keyword)**
- Files: `patterns.zig:92`, `codegen.zig:44`, `codegen.zig:53`, `codegen.zig:247`
- Fix: Renamed all `opaque` variables to `opaque_decl` or `opaque_type`
4. **Error union not handled for `readLine()`**
- File: `patterns.zig`
- Fix: Added `try` keyword before all `readLine()` calls
5. **Memory leak in test (GPA reported leak)**
- File: `patterns.zig` test
- Fix: Changed to use arena allocator
6. **`expected type '[]Declaration', found '*const [1]Declaration'`**
- File: `codegen.zig` tests
- Fix: Changed arrays from `const` to `var` and used `array[0..]` syntax
7. **`writer()` member function expected 1 argument(s), found 0**
- File: `codegen.zig` (multiple locations)
- Fix: Passed allocator to all `writer()` calls: `writer(allocator)`
8. **`value with comptime-only type 'comptime_int' depends on runtime control flow`**
- File: `codegen.zig:148`
- Fix: Added explicit type annotation: `const type_bits: u32 = ...`
9. **`expected type 'u5', found 'u6'` (bit shift operand)**
- File: `codegen.zig:238`
- Fix: Cast shift amount: `@as(u5, @intCast(bit))`
10. **`getCastType` expects 1 argument, found 2**
- File: `codegen.zig:249`
- Fix: Removed allocator parameter, `getCastType()` returns enum not string
11. **`std.io.getStdOut()` - no member named 'getStdOut'**
- File: `parser.zig:105`
- Fix: Changed to `std.posix.write(std.posix.STDOUT_FILENO, output)`
## Best Practices
1. **Always handle error unions**: Use `try` or explicit `catch`
2. **Avoid reserved keywords**: Check keyword list before naming variables
3. **Use arena allocators in tests**: Simplifies cleanup and prevents leak errors
4. **Pass allocators explicitly**: Don't assume methods have access to allocator
5. **Prefer explicit slices**: Use `array[0..]` instead of relying on coercion
6. **Check build API docs**: Build system API changes between versions
7. **Annotate types when using runtime conditions**: Avoid comptime inference issues
8. **Match bit shift operand types**: Use correct size for shift amounts (u5 for u32, etc.)
9. **Use `var` for arrays that need slicing**: Even if you don't modify the array itself
## Common Error Messages
| Error Message | Likely Cause | Solution |
|--------------|--------------|----------|
| `expected 'an identifier', found 'opaque'` | Using reserved keyword | Rename variable |
| `expected type '[]T', found '*const [N]T'` | Array vs slice mismatch | Use `array[0..]` |
| `error union not handled` | Missing `try` or `catch` | Add error handling |
| `unused local variable` | Variable declared but not used | Use it or assign to `_` |
| `expected 2 arguments, found 1` | ArrayList API change | Pass allocator explicitly |
| `root_source_file` field doesn't exist | Old build API | Use `root_module` |
| `member function expected 1 argument(s), found 0` | Missing allocator for ArrayList methods | Pass allocator to method |
| `no member named 'getStdOut'` | Moved/renamed std lib function | Use `std.posix.write()` |
| `comptime-only type depends on runtime control flow` | Missing explicit type annotation | Add `: TypeName` annotation |
| `expected type 'u5', found 'u6'` | Shift operand type mismatch | Cast to correct shift type |
| `cast discards const qualifier` | Trying to get mutable slice from const array | Declare array with `var` |
## Resources
- Zig 0.15 Release Notes: https://ziglang.org/download/0.15.0/release-notes.html
- Build System API: https://ziglang.org/documentation/master/std/#std.Build
- ArrayList API: https://ziglang.org/documentation/master/std/#std.ArrayList
- Reserved Keywords: https://ziglang.org/documentation/master/#Keywords