50 KiB
SDL3 Header Parser & Zig Binding Generator
Overview
SDL3's C headers are highly regular and well-structured, making them ideal candidates for automated parsing and Zig binding generation. This document outlines the architecture and implementation plan for a parser that will extract type and function information from SDL3 headers and generate idiomatic Zig bindings.
Current State
The lib/sdl3/src/ directory contains hand-maintained Zig bindings for SDL3. These bindings demonstrate the target output format that our generator should produce. Key files include:
gpu.zig- Comprehensive GPU API bindings (good reference implementation)video.zig,events.zig,init.zig- Other module bindingsc.zig- Direct C imports
Goals
- Parse all 85 SDL3 headers in
SDL/include/SDL3/ - Extract complete type information: enums, flags, structs, opaque types, functions
- Generate idiomatic Zig bindings matching the style of existing hand-written bindings
- Preserve documentation from C headers in generated Zig files
- Support incremental updates when SDL3 headers change
SDL3 Header Patterns
1. Opaque Types
C Pattern:
/**
* An opaque handle representing a GPU device.
*
* \since This struct is available since SDL 3.2.0.
*
* \sa SDL_CreateGPUDevice
* \sa SDL_DestroyGPUDevice
*/
typedef struct SDL_GPUDevice SDL_GPUDevice;
Zig Output:
pub const GPUDevice = opaque {
// Methods will be added here
};
2. Enumerations
C Pattern:
/**
* Specifies the primitive topology of a graphics pipeline.
*
* \since This enum is available since SDL 3.2.0.
*
* \sa SDL_CreateGPUGraphicsPipeline
*/
typedef enum SDL_GPUPrimitiveType
{
SDL_GPU_PRIMITIVETYPE_TRIANGLELIST, /**< A series of separate triangles. */
SDL_GPU_PRIMITIVETYPE_TRIANGLESTRIP, /**< A series of connected triangles. */
SDL_GPU_PRIMITIVETYPE_LINELIST, /**< A series of separate lines. */
SDL_GPU_PRIMITIVETYPE_LINESTRIP, /**< A series of connected lines. */
SDL_GPU_PRIMITIVETYPE_POINTLIST /**< A series of separate points. */
} SDL_GPUPrimitiveType;
Zig Output:
pub const GPUPrimitiveType = enum(c_int) {
primitivetypeTrianglelist, //*< A series of separate triangles. */
primitivetypeTrianglestrip, //*< A series of connected triangles. */
primitivetypeLinelist, //*< A series of separate lines. */
primitivetypeLinestrip, //*< A series of connected lines. */
primitivetypePointlist, //*< A series of separate points. */
};
Naming Convention:
- Remove
SDL_GPU_prefix - Convert to camelCase starting with lowercase
- Example:
SDL_GPU_PRIMITIVETYPE_TRIANGLELIST→primitivetypeTrianglelist
3. Flag Types (Bitmasks)
C Pattern:
typedef Uint32 SDL_GPUTextureUsageFlags;
#define SDL_GPU_TEXTUREUSAGE_SAMPLER (1u << 0) /**< Texture supports sampling. */
#define SDL_GPU_TEXTUREUSAGE_COLOR_TARGET (1u << 1) /**< Texture is a color render target. */
#define SDL_GPU_TEXTUREUSAGE_DEPTH_STENCIL_TARGET (1u << 2) /**< Texture is a depth stencil target. */
#define SDL_GPU_TEXTUREUSAGE_GRAPHICS_STORAGE_READ (1u << 3) /**< Texture supports storage reads in graphics stages. */
#define SDL_GPU_TEXTUREUSAGE_COMPUTE_STORAGE_READ (1u << 4) /**< Texture supports storage reads in the compute stage. */
#define SDL_GPU_TEXTUREUSAGE_COMPUTE_STORAGE_WRITE (1u << 5) /**< Texture supports storage writes in the compute stage. */
#define SDL_GPU_TEXTUREUSAGE_COMPUTE_STORAGE_SIMULTANEOUS_READ_WRITE (1u << 6) /**< Texture supports reads and writes in the same compute shader. */
Zig Output:
pub const GPUTextureUsageFlags = packed struct(u32) {
textureusageSampler: bool = false,
textureusageColorTarget: bool = false,
textureusageDepthStencilTarget: bool = false,
textureusageGraphicsStorageRead: bool = false,
textureusageComputeStorageRead: bool = false,
textureusageComputeStorageWrite: bool = false,
textureusageComputeStorageSimultaneousReadWrite: bool = false,
pad0: u24 = 0,
rsvd: bool = false,
};
Naming Convention:
- Remove common prefix (e.g.,
SDL_GPU_TEXTUREUSAGE_) - Convert to camelCase starting with lowercase
- Add padding fields to reach the backing integer size (u32, u64, etc.)
- Add
rsvdfield as the high bit for future expansion
4. Structures
C Pattern:
/**
* A structure specifying the parameters of a graphics pipeline viewport.
*
* \since This struct is available since SDL 3.2.0.
*
* \sa SDL_SetGPUViewport
*/
typedef struct SDL_GPUViewport
{
float x; /**< The left offset of the viewport. */
float y; /**< The top offset of the viewport. */
float w; /**< The width of the viewport. */
float h; /**< The height of the viewport. */
float min_depth; /**< The minimum depth of the viewport. */
float max_depth; /**< The maximum depth of the viewport. */
} SDL_GPUViewport;
Zig Output:
pub const GPUViewport = extern struct {
x: f32, // The left offset of the viewport.
y: f32, // The top offset of the viewport.
w: f32, // The width of the viewport.
h: f32, // The height of the viewport.
min_depth: f32, // The minimum depth of the viewport.
max_depth: f32, // The maximum depth of the viewport.
};
Naming Convention:
- Keep field names as-is (already snake_case)
- Convert C types to Zig equivalents:
float→f32double→f64Uint8→u8Uint16→u16Uint32→u32Uint64→u64Sint8→i8Sint16→i16Sint32→i32Sint64→i64bool/SDL_bool→boolsize_t→usizeint→c_intchar→u8(for single chars) or[*c]const u8(for strings)void*→?*anyopaque(if nullable) or*anyopaque(if non-null)const char*→[*c]const u8T*(opaque pointer) →*Tconst T*(opaque pointer) →*const TT**(out parameter) →[*c]*T
5. Constants & Large Enums
Some enums in SDL3 have many values and are better represented as individual constants in Zig.
C Pattern:
typedef enum SDL_EventType
{
SDL_EVENT_FIRST = 0, /**< Unused (do not remove) */
/* Application events */
SDL_EVENT_QUIT = 0x100, /**< User-requested quit */
SDL_EVENT_TERMINATING = 0x101, /**< OS is terminating the app */
// ... many more values
} SDL_EventType;
Zig Output (Individual Constants):
pub const first: u32 = 0;
pub const quit: u32 = 256;
pub const terminating: u32 = 257;
// ... many more constants
Design Decision:
- Large enums (>20 values) that serve as constant collections → individual constants
- Small enums that represent a closed set of values → Zig enum
- Configuration: Mark certain enums for constant expansion in config
6. Functions
C Pattern:
/**
* Create a GPU context.
*
* \param format_flags a bitflag indicating which shader formats the app can
* provide.
* \param debug_mode enable debug mode properties and validations.
* \param name the preferred GPU driver, or NULL to let SDL pick the optimal
* driver.
* \returns a GPU context on success, or NULL on failure; call SDL_GetError()
* for more information.
*
* \since This function is available since SDL 3.2.0.
*
* \sa SDL_GetGPUDriver
* \sa SDL_DestroyGPUDevice
* \sa SDL_GPUSupportsShaderFormats
*/
extern SDL_DECLSPEC SDL_GPUDevice * SDLCALL SDL_CreateGPUDevice(
SDL_GPUShaderFormat format_flags,
bool debug_mode,
const char *name);
Zig Output (Free Function):
// SDL_CreateGPUDevice
pub inline fn createGPUDevice(format_flags: GPUShaderFormat, debug_mode: bool, name: [*c]const u8) *GPUDevice {
return @ptrCast(c.SDL_CreateGPUDevice(@bitCast(format_flags), debug_mode, name));
}
Zig Output (Method on Opaque Type):
pub const GPUDevice = opaque {
// SDL_DestroyGPUDevice
pub inline fn destroyGPUDevice(device: *GPUDevice) void {
c.SDL_DestroyGPUDevice(@ptrCast(device));
}
};
Function Classification Rules:
- Functions taking an opaque type pointer as the first parameter → method on that type
- Functions that create an opaque type → free function (constructor)
- All other functions → free functions
Naming Convention:
- Remove
SDL_prefix - Convert to camelCase
- Example:
SDL_CreateGPUDevice→createGPUDevice - For methods, keep the full name but it will be called as
device.destroyGPUDevice(device)
Cast Handling in Generated Code:
The wrapper functions need to insert appropriate casts:
-
Opaque pointers: Use
@ptrCastc.SDL_DestroyGPUDevice(@ptrCast(device)) -
Enums: Use
@intFromEnum(Zig → C) or@enumFromInt(C → Zig)// Zig to C c.SDL_Function(@intFromEnum(my_enum)) // C to Zig return @enumFromInt(c.SDL_Function()) -
Flags (packed structs): Use
@bitCastc.SDL_CreateDevice(@bitCast(format_flags)) -
Primitive types: Usually no cast needed, but may use
@bitCastfor same-size conversionsc.SDL_Function(@bitCast(my_u32)) -
Return values:
- Opaque pointers:
@ptrCastthe result - Enums:
@enumFromIntthe result - Flags:
@bitCastthe result - Primitives: direct return
- Opaque pointers:
Module Dependencies & Header Relationships
SDL3 headers have dependencies on each other:
SDL_stdinc.h # Base types (Uint32, Sint32, etc.)
↓
SDL_error.h # Error handling
↓
SDL_properties.h # Properties system
↓
SDL_video.h # Video/window system
↓
SDL_gpu.h # GPU rendering (depends on video for SDL_Window)
Parsing Strategy:
- Parse all headers into a unified AST first
- Build type dependency graph
- Resolve cross-header type references
- Generate modules in dependency order
- Add imports between generated modules as needed
Generated Module Structure:
// gpu.zig
pub const c = @import("c.zig").c;
pub const video = @import("video.zig"); // If needed
// Use video types
pub const Window = video.Window;
Simplified Zig Parser Architecture
Key Insight: SDL3 Headers Are EXTREMELY Regular
After analyzing the actual SDL3 headers, they follow very simple patterns:
- Opaque types:
typedef struct SDL_Foo SDL_Foo;- Single line! - Enums:
typedef enum SDL_Foo { ... } SDL_Foo;- Braces are balanced - Structs:
typedef struct SDL_Foo { ... } SDL_Foo;- Same as enums - Flags:
typedef Uint32 SDL_FooFlags;+#define SDL_FOO_*lines following - Functions:
extern SDL_DECLSPEC Type SDLCALL SDL_Name(...);- May span lines
We don't need:
- ❌ Full lexer/tokenizer
- ❌ Recursive descent parser
- ❌ Abstract Syntax Tree
- ❌ Symbol tables
- ❌ Type resolution
- ❌ Semantic analysis
- ❌ Following #includes or system headers
- ❌ Complex preprocessor
We DO need:
- ✅ Line-by-line reader with brace tracking
- ✅ Simple pattern matching (regex or string matching)
- ✅ Extract pattern data into structs
- ✅ Direct code generation
Simplified Pipeline
C Header File
↓
[1. Pattern Scanner] → Extract Declarations
↓ (opaque, enum, struct, flags, function)
[2. Data Extraction] → Simple Structs
↓ (name, fields, values, etc.)
[3. Code Generator] → Zig Source Code
↓
Generated Bindings
Complexity Reduction: ~1/5th the original complexity!
Simplified Module Structure
parser.zig - Main Entry Point & Scanner
The main file that does pattern scanning and code generation.
const std = @import("std");
const patterns = @import("patterns.zig");
const codegen = @import("codegen.zig");
pub fn main() !void {
var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator);
defer arena.deinit();
const allocator = arena.allocator();
// 1. Parse command line arguments
const args = try std.process.argsAlloc(allocator);
// 2. Read header file
const source = try std.fs.cwd().readFileAlloc(allocator, header_path, 10 * 1024 * 1024);
// 3. Scan for patterns
const declarations = try patterns.scan(allocator, source);
// 4. Generate Zig code
const output = try codegen.generate(allocator, declarations);
// 5. Write output
try std.fs.cwd().writeFile(output_path, output);
}
Responsibilities:
- Command-line argument parsing
- File I/O
- Call scanner and generator
- Memory management (arena allocator)
patterns.zig - Pattern Scanner
Scans the C header and extracts declarations using simple pattern matching.
pub const Declaration = union(enum) {
opaque_type: OpaqueType,
enum_decl: EnumDecl,
struct_decl: StructDecl,
flag_decl: FlagDecl,
function_decl: FunctionDecl,
};
pub const OpaqueType = struct {
name: []const u8, // SDL_GPUDevice
doc_comment: ?[]const u8, // /** ... */
};
pub const EnumDecl = struct {
name: []const u8, // SDL_GPUPrimitiveType
values: []EnumValue, // List of enum values
doc_comment: ?[]const u8,
};
pub const EnumValue = struct {
name: []const u8, // SDL_GPU_PRIMITIVETYPE_TRIANGLELIST
value: ?[]const u8, // Optional explicit value
comment: ?[]const u8, // Inline comment
};
pub const StructDecl = struct {
name: []const u8, // SDL_GPUViewport
fields: []FieldDecl,
doc_comment: ?[]const u8,
};
pub const FieldDecl = struct {
name: []const u8, // x
type_name: []const u8, // float
comment: ?[]const u8,
};
pub const FlagDecl = struct {
name: []const u8, // SDL_GPUTextureUsageFlags
underlying_type: []const u8, // Uint32
flags: []FlagValue,
doc_comment: ?[]const u8,
};
pub const FlagValue = struct {
name: []const u8, // SDL_GPU_TEXTUREUSAGE_SAMPLER
value: []const u8, // (1u << 0)
comment: ?[]const u8,
};
pub const FunctionDecl = struct {
name: []const u8, // SDL_CreateGPUDevice
return_type: []const u8, // SDL_GPUDevice *
params: []ParamDecl,
doc_comment: ?[]const u8,
};
pub const ParamDecl = struct {
name: []const u8, // format_flags
type_name: []const u8, // SDL_GPUShaderFormat
};
pub const Scanner = struct {
source: []const u8,
pos: usize,
pub fn init(source: []const u8) Scanner {
return .{ .source = source, .pos = 0 };
}
pub fn scan(self: *Scanner, allocator: Allocator) ![]Declaration {
var decls = std.ArrayList(Declaration).init(allocator);
while (!self.isAtEnd()) {
if (try self.scanOpaque()) |opaque| {
try decls.append(.{ .opaque_type = opaque });
} else if (try self.scanEnum()) |enum_| {
try decls.append(.{ .enum_decl = enum_ });
} else if (try self.scanStruct()) |struct_| {
try decls.append(.{ .struct_decl = struct_ });
} else if (try self.scanFlags()) |flags| {
try decls.append(.{ .flag_decl = flags });
} else if (try self.scanFunction()) |func| {
try decls.append(.{ .function_decl = func });
} else {
self.skipLine();
}
}
return decls.toOwnedSlice();
}
fn scanOpaque(self: *Scanner) !?OpaqueType {
// Look for: typedef struct SDL_Foo SDL_Foo;
if (self.matchLine("typedef struct ")) {
// Extract name from "SDL_Foo SDL_Foo;"
// ...
}
return null;
}
fn scanEnum(self: *Scanner) !?EnumDecl {
// Look for: typedef enum SDL_Foo
// Then collect until } SDL_Foo;
// ...
}
fn scanStruct(self: *Scanner) !?StructDecl {
// Same as enum but for structs
// ...
}
fn scanFlags(self: *Scanner) !?FlagDecl {
// Look for: typedef Uint32 SDL_FooFlags;
// Then collect following #define lines
// ...
}
fn scanFunction(self: *Scanner) !?FunctionDecl {
// Look for: extern SDL_DECLSPEC Type SDLCALL SDL_Name(...);
// May span multiple lines
// ...
}
// Utility functions
fn matchLine(self: *Scanner, prefix: []const u8) bool { }
fn readUntil(self: *Scanner, terminator: u8) []const u8 { }
fn readBraced(self: *Scanner) []const u8 { } // Read {...}
fn extractDocComment(self: *Scanner) ?[]const u8 { }
fn skipLine(self: *Scanner) void { }
fn isAtEnd(self: *Scanner) bool { }
};
Strategy:
- Simple line-by-line scanning
- Pattern matching with
std.mem.startsWith - Brace counting for
{...}blocks - Store raw strings, parse during generation
naming.zig - Name Conversion
Simple string manipulation for name conversion.
pub fn stripPrefix(name: []const u8, prefix: []const u8) []const u8 {
if (std.mem.startsWith(u8, name, prefix)) {
return name[prefix.len..];
}
return name;
}
pub fn typeNameToZig(c_name: []const u8) []const u8 {
// SDL_GPUDevice -> GPUDevice (just strip SDL_)
return stripPrefix(c_name, "SDL_");
}
pub fn functionNameToZig(c_name: []const u8, allocator: Allocator) ![]const u8 {
// SDL_CreateGPUDevice -> createGPUDevice
const without_prefix = stripPrefix(c_name, "SDL_");
return lowerFirstChar(without_prefix, allocator);
}
pub fn enumValueToZig(c_name: []const u8, prefix: []const u8, allocator: Allocator) ![]const u8 {
// SDL_GPU_PRIMITIVETYPE_TRIANGLELIST -> primitivetypeTrianglelist
const without_prefix = stripPrefix(c_name, prefix);
return toLowerCamelCase(without_prefix, allocator);
}
pub fn detectCommonPrefix(names: []const []const u8) []const u8 {
// Find longest common prefix
// SDL_GPU_PRIMITIVETYPE_TRIANGLELIST, SDL_GPU_PRIMITIVETYPE_LINESTRIP
// -> SDL_GPU_PRIMITIVETYPE_
}
fn lowerFirstChar(s: []const u8, allocator: Allocator) ![]const u8 {
var result = try allocator.dupe(u8, s);
if (result.len > 0) result[0] = std.ascii.toLower(result[0]);
return result;
}
fn toLowerCamelCase(s: []const u8, allocator: Allocator) ![]const u8 {
// Convert SCREAMING_SNAKE to lowerCamelCase
// Handle SDL3's conventions
}
Convention Rules:
| C Pattern | Zig Pattern | Example |
|---|---|---|
SDL_FooBar (type) |
FooBar |
SDL_GPUDevice → GPUDevice |
SDL_FooBar (function) |
fooBar |
SDL_CreateGPUDevice → createGPUDevice |
SDL_FOO_BAR_BAZ (enum) |
fooBarBaz |
SDL_GPU_PRIMITIVETYPE_TRIANGLELIST → primitivetypeTrianglelist |
SDL_FOO_BAR (flag) |
fooBar |
SDL_GPU_TEXTUREUSAGE_SAMPLER → textureusageSampler |
types.zig - Type Conversion
Simple string-based type conversion (no need to parse types fully).
pub fn convertType(c_type: []const u8) []const u8 {
// Simple table lookup and string replacement
if (std.mem.eql(u8, c_type, "void")) return "void";
if (std.mem.eql(u8, c_type, "bool")) return "bool";
if (std.mem.eql(u8, c_type, "SDL_bool")) return "bool";
if (std.mem.eql(u8, c_type, "float")) return "f32";
if (std.mem.eql(u8, c_type, "double")) return "f64";
if (std.mem.eql(u8, c_type, "char")) return "u8";
if (std.mem.eql(u8, c_type, "int")) return "c_int";
if (std.mem.eql(u8, c_type, "Uint8")) return "u8";
if (std.mem.eql(u8, c_type, "Uint16")) return "u16";
if (std.mem.eql(u8, c_type, "Uint32")) return "u32";
if (std.mem.eql(u8, c_type, "Uint64")) return "u64";
if (std.mem.eql(u8, c_type, "Sint8")) return "i8";
if (std.mem.eql(u8, c_type, "Sint16")) return "i16";
if (std.mem.eql(u8, c_type, "Sint32")) return "i32";
if (std.mem.eql(u8, c_type, "Sint64")) return "i64";
if (std.mem.eql(u8, c_type, "size_t")) return "usize";
// Pointers - simple pattern matching
if (std.mem.eql(u8, c_type, "const char *")) return "[*c]const u8";
if (std.mem.eql(u8, c_type, "void *")) return "?*anyopaque";
// SDL types - just strip SDL_ prefix
if (std.mem.startsWith(u8, c_type, "SDL_")) {
// SDL_GPUDevice * -> *GPUDevice
// SDL_GPUTextureFormat -> GPUTextureFormat
// Handle pointers and const
}
return c_type; // fallback
}
Strategy:
- Table lookup for primitives
- Pattern matching for pointers
- String replacement for SDL types
- No need to fully parse - SDL types are very regular!
codegen.zig - Code Generation
Direct code generation from extracted declarations.
pub const CodeGen = struct {
decls: []Declaration,
allocator: Allocator,
output: std.ArrayList(u8),
pub fn generate(allocator: Allocator, decls: []Declaration) ![]const u8 {
var gen = CodeGen{
.decls = decls,
.allocator = allocator,
.output = std.ArrayList(u8).init(allocator),
};
try gen.writeHeader();
// Generate each declaration
for (decls) |decl| {
switch (decl) {
.opaque_type => |opaque| try gen.writeOpaque(opaque),
.enum_decl => |enum_| try gen.writeEnum(enum_),
.struct_decl => |struct_| try gen.writeStruct(struct_),
.flag_decl => |flags| try gen.writeFlags(flags),
.function_decl => |func| try gen.writeFunction(func),
}
}
return gen.output.toOwnedSlice();
}
fn writeHeader(self: *CodeGen) !void {
try self.output.appendSlice("pub const c = @import(\"c.zig\").c;\n\n");
}
fn writeOpaque(self: *CodeGen, opaque: OpaqueType) !void {
// pub const GPUDevice = opaque {};
try self.output.writer().print("pub const {s} = opaque {{}};\n\n", .{
naming.typeNameToZig(opaque.name),
});
}
fn writeEnum(self: *CodeGen, enum_: EnumDecl) !void {
const zig_name = naming.typeNameToZig(enum_.name);
try self.output.writer().print("pub const {s} = enum(c_int) {{\n", .{zig_name});
const prefix = naming.detectCommonPrefix(/* enum values */);
for (enum_.values) |value| {
const zig_value = try naming.enumValueToZig(value.name, prefix, self.allocator);
if (value.comment) |comment| {
try self.output.writer().print(" {s}, // {s}\n", .{ zig_value, comment });
} else {
try self.output.writer().print(" {s},\n", .{zig_value});
}
}
try self.output.appendSlice("};\n\n");
}
fn writeStruct(self: *CodeGen, struct_: StructDecl) !void {
const zig_name = naming.typeNameToZig(struct_.name);
try self.output.writer().print("pub const {s} = extern struct {{\n", .{zig_name});
for (struct_.fields) |field| {
const zig_type = types.convertType(field.type_name);
if (field.comment) |comment| {
try self.output.writer().print(" {s}: {s}, // {s}\n", .{
field.name, zig_type, comment,
});
} else {
try self.output.writer().print(" {s}: {s},\n", .{ field.name, zig_type });
}
}
try self.output.appendSlice("};\n\n");
}
fn writeFlags(self: *CodeGen, flags: FlagDecl) !void {
// pub const GPUTextureUsageFlags = packed struct(u32) {
// textureusageSampler: bool = false,
// ...
// };
// Calculate padding, generate fields
}
fn writeFunction(self: *CodeGen, func: FunctionDecl) !void {
// Determine if it's a method or free function
const is_method = isMethod(func);
if (is_method) {
// Will be added to opaque type later (need second pass)
} else {
// Free function
const zig_name = try naming.functionNameToZig(func.name, self.allocator);
// Generate: pub inline fn createGPUDevice(...) ... { c.SDL_CreateGPUDevice(...); }
}
}
};
fn isMethod(func: FunctionDecl) bool {
// Check if first parameter is an opaque type
if (func.params.len > 0) {
const first_param_type = func.params[0].type_name;
// Check if it's one of the opaque types
return std.mem.startsWith(u8, first_param_type, "SDL_GPU") and
std.mem.endsWith(u8, first_param_type, " *");
}
return false;
}
Strategy:
- Direct string generation (no templates needed!)
- Two passes: types first, then group methods with opaque types
- Simple
std.fmt.formatfor code generation - No complex AST traversal
Note: Config can be added later if needed, but start without it for simplicity.
Memory Management
Arena allocation for simplicity:
pub fn main() !void {
var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator);
defer arena.deinit();
const allocator = arena.allocator();
// Everything freed at once when done
}
Testing
Simple integration tests:
test "scan opaque typedef" {
const source = "typedef struct SDL_GPUDevice SDL_GPUDevice;";
var scanner = patterns.Scanner.init(source);
const decls = try scanner.scan(std.testing.allocator);
try std.testing.expectEqual(@as(usize, 1), decls.len);
try std.testing.expect(decls[0] == .opaque_type);
}
test "generate enum" {
const enum_decl = EnumDecl{
.name = "SDL_GPUPrimitiveType",
.values = &[_]EnumValue{
.{ .name = "SDL_GPU_PRIMITIVETYPE_TRIANGLELIST", .value = null, .comment = null },
},
.doc_comment = null,
};
const output = try codegen.generateEnum(enum_decl, std.testing.allocator);
// Check output matches expected Zig code
}
Performance
Expected:
- Parse all 85 headers: < 1 second
- Memory: < 50 MB
- Single-threaded (sufficient for this workload)
Parser Architecture
Phase 1: Lexical Analysis & Preprocessing
Input: Raw C header files Output: Token stream
Tasks:
- Remove
SDL_begin_code.h/SDL_close_code.hincludes (these are preprocessor magic) - Strip out platform-specific
#ifdefblocks (or handle multiple platform variants) - Expand or track
#definemacros (especially for flag values) - Tokenize the remaining C code
- Handle multi-line comments and documentation blocks
Challenges:
- C preprocessor complexity
- Platform-specific code paths
- Macro expansion for flag definitions
Approach:
- Use a simple regex-based preprocessor for well-known patterns
- Or use libclang Python bindings for robust parsing
- Focus on public API headers only (skip internal
_c.hfiles)
Phase 2: Syntax Analysis & AST Building
Input: Token stream Output: Abstract Syntax Tree (AST)
AST Node Types:
OpaqueType- opaque struct typedefsEnum- enum definitions with valuesFlagType- flag typedef + associated definesStruct- struct definitionsFunction- function declarationsComment- documentation blocks
Key Information to Extract:
For each type/function:
- Full C name (e.g.,
SDL_GPUDevice) - Zig name (e.g.,
GPUDevice) - Documentation comment
- Source location (file, line number)
- Related items (
\sareferences) - Version info (
\since)
For functions:
- Return type
- Parameter names and types
- Which opaque type it belongs to (if any)
- Const/pointer qualifiers
For enums:
- Each enumerant name and value
- Inline comments for each value
For flags:
- Each flag name and bit position
- Backing integer type
For structs:
- Each field name and type
- Inline comments for each field
- Padding requirements
Phase 3: Semantic Analysis
Input: Raw AST Output: Enriched AST with relationships
Tasks:
-
Type Resolution:
- Resolve all type references to their definitions
- Handle forward declarations
- Build type dependency graph
-
Function Classification:
- Identify which functions are methods vs. free functions
- Group methods by opaque type
- Detect constructor/destructor patterns
-
Documentation Processing:
- Parse Doxygen tags (
\param,\returns,\sa,\since) - Build cross-reference map
- Extract and clean inline comments
- Parse Doxygen tags (
-
Naming Convention Application:
- Convert SDL names to Zig names
- Detect and handle naming collisions
- Generate consistent camelCase names
-
Module Organization:
- Determine which Zig file each definition belongs to
- Based on C header name (e.g.,
SDL_gpu.h→gpu.zig) - Handle cross-module dependencies
Phase 4: Code Generation
Input: Enriched AST Output: Zig source files
Generation Strategy:
- Header:
pub const c = @import("c.zig").c;
pub const PropertiesID = u32;
// Other common imports/aliases
-
Type Definitions (Order matters!):
- First: Flag types (no dependencies)
- Second: Enums (no dependencies)
- Third: Opaque types (empty declarations)
- Fourth: Structs (may reference above types)
-
Free Functions:
- After all types
- Grouped by category
-
Opaque Type Methods:
- Fill in method definitions in opaque types
- Maintain consistent ordering
Code Generation Templates:
For each AST node type, we need a template. Examples:
Enum Template:
pub const {ZigName} = enum(c_int) {
{for each value}
{zigValueName}, //{inline comment}
{end for}
};
Flag Template:
pub const {ZigName} = packed struct({backingType}) {
{for each flag}
{zigFlagName}: bool = false,
{end for}
{padding fields}
rsvd: bool = false,
};
Struct Template:
pub const {ZigName} = extern struct {
{for each field}
{fieldName}: {zigType}, // {inline comment}
{end for}
};
Free Function Template:
// {C function name}
pub inline fn {zigFuncName}({params}) {returnType} {
{function body with casts}
}
Method Template:
// {C function name}
pub inline fn {zigMethodName}({params}) {returnType} {
c.{cFuncName}({casts and calls});
}
Phase 5: Validation & Testing
Input: Generated Zig files Output: Validated, compilable bindings
Validation Steps:
-
Compilation Test:
- Run
zig buildon generated files - Ensure no syntax errors
- Check type correctness
- Run
-
API Completeness:
- Compare generated API surface with C headers
- Ensure no functions/types are missing
- Check for extra/duplicate definitions
-
Comparison with Hand-Written:
- Diff generated
gpu.zigwith existingsrc/gpu.zig - Verify naming conventions match
- Check structure and organization
- Diff generated
-
Cross-Reference Validation:
- Verify all type references are resolvable
- Check method ownership is correct
- Ensure no circular dependencies
-
Documentation Check:
- Verify comments are preserved
- Check for formatting issues
- Validate cross-references
Recommended Implementation Milestones
Current Status: ✅ Hello world implemented (parser.zig lists all 85 headers)
Milestone 1: Pattern Scanner (2-3 days)
Goal: Extract declarations from SDL_gpu.h
Scope:
- Implement
patterns.zigwithScannerstruct - Scan for opaque typedefs (simple one-line pattern)
- Scan for enums (track braces)
- Scan for structs (track braces)
- Store declarations in simple structs
Deliverable:
patterns.zigthat extracts opaque, enum, and struct from SDL_gpu.h- Basic tests for each pattern type
Complexity: Low - just string matching and brace counting
Milestone 2: Code Generation (2-3 days)
Goal: Generate Zig code for extracted declarations
Scope:
- Implement
codegen.zig - Implement
naming.zigfor name conversion - Implement
types.zigfor type conversion - Generate opaque types
- Generate enums
- Generate structs
Deliverable:
- Generated Zig code for subset of SDL_gpu.h
- Code compiles with
zig build - Matches hand-written style
Complexity: Low - direct string generation
Milestone 3: Flags and Functions (2-3 days)
Goal: Complete SDL_gpu.h parsing
Scope:
- Add flag scanning (typedef + #define lines)
- Add function scanning
- Classify functions (method vs. free function)
- Generate flag types
- Generate functions and methods
Deliverable:
- Complete
gpu.ziggeneration - All types and functions included
- Compiles and matches hand-written version
Complexity: Medium - function classification logic
Milestone 4: Multi-Header Support (1-2 days)
Goal: Generalize to other headers
Scope:
- Test on SDL_video.h, SDL_events.h, SDL_init.h
- Handle any new patterns
- Fix bugs
- Add integration tests
Deliverable:
- Parser handles all common SDL3 patterns
- Generate bindings for multiple headers
- All generated code compiles
Complexity: Low - SDL headers are very consistent
Total Time Estimate
2-3 weeks of focused work (vs. 6-8 weeks with complex architecture)
Key Simplifications:
- No lexer/tokenizer (line-by-line scanning)
- No AST (direct data extraction)
- No semantic analysis (simple pattern matching)
- No complex type system (string conversion)
Implementation Plan
Stage 1: Prototype Parser (Week 1-2)
Goal: Parse SDL_gpu.h and generate gpu.zig
Tasks:
- Choose parsing approach (libclang vs. custom parser)
- Implement basic token scanner
- Parse enum definitions
- Parse flag definitions
- Parse struct definitions
- Parse opaque types
- Parse function signatures
Deliverable: Working parser for SDL_gpu.h
Stage 2: Code Generator (Week 2-3)
Goal: Generate gpu.zig from parsed data
Tasks:
- Implement naming convention rules
- Build type dependency resolver
- Create code generation templates
- Implement function classification
- Add method grouping logic
- Generate initial gpu.zig
Deliverable: Generated gpu.zig that compiles
Stage 3: Refinement (Week 3-4)
Goal: Match hand-written gpu.zig quality
Tasks:
- Compare generated vs. hand-written
- Fix naming mismatches
- Improve comment formatting
- Adjust code organization
- Handle edge cases
- Add manual override system for special cases
Deliverable: Generated gpu.zig identical to hand-written version
Stage 4: Generalization (Week 4-6)
Goal: Parse all 85 SDL3 headers
Tasks:
- Test parser on other headers (video, events, init, etc.)
- Handle new patterns not seen in gpu.h
- Implement cross-header type resolution
- Add module dependency management
- Handle platform-specific code
- Create configuration system for header selection
Deliverable: Parser that handles all SDL3 headers
Stage 5: Integration & Automation (Week 6-7)
Goal: Integrate into build system
Tasks:
- Create Zig build step for code generation
- Add header change detection
- Implement incremental regeneration
- Add validation step to build
- Create documentation generator
- Write user guide
Deliverable: Automated, maintainable system
Technical Decisions
Parser Implementation
Option A: libclang bindings (C or Zig)
- ✅ Robust, handles all C syntax
- ✅ Proper preprocessor support
- ✅ Battle-tested
- ❌ External dependency
- ❌ Slower
- ❌ Overkill for regular headers
- ❌ Complex API
Option B: Custom Zig parser
- ✅ Lightweight and fast
- ✅ Tailored to SDL patterns
- ✅ Easy to debug and modify
- ✅ No external dependencies
- ✅ Compiles to single binary
- ✅ Same language as target (Zig → Zig)
- ✅ Can share types with generated code
- ✅ Strong type safety during parsing
- ❌ Need to handle C syntax edge cases
- ❌ Manual preprocessor handling
Decision: Option B (custom Zig parser).
Rationale: SDL3 headers are extremely regular. A custom Zig parser lets us exploit this regularity and generate better Zig code. We can handle the preprocessor with simple pattern matching for the common cases. Using Zig gives us strong typing, safety, and performance, and results in a single-binary tool with no external dependencies.
Language Choice
Zig - Perfect for this task:
- Strong type system helps model C syntax accurately
- Excellent string processing with
std.mem - Arena allocators simplify AST memory management
- Fast compilation and execution
- Same language as output (Zig → Zig)
- Can share common types between parser and generated code
- Single binary deployment
- No runtime dependencies
File Organization
lib/sdl3/
├── parser/ # Simple parser implementation (Zig)
│ ├── build.zig # Parser build script
│ ├── parser.zig # Main entry point & CLI
│ ├── patterns.zig # Pattern scanner (core logic)
│ ├── codegen.zig # Code generator
│ ├── naming.zig # Name conversion utilities
│ └── types.zig # Type conversion utilities
├── src/ # Hand-written & final bindings (target)
│ ├── gpu.zig # Reference implementation (1,198 lines)
│ ├── video.zig
│ ├── events.zig
│ └── ...
├── SDL/ # SDL3 submodule
│ └── include/SDL3/ # Source C headers (85 files)
└── research/
└── sdl-header-parser.md # This file
Total Modules: 4 (vs. 10+ in over-engineered approach) Total Complexity: ~1/5th of original plan
Configuration System
Support manual overrides for edge cases:
config.py:
# Functions that should be free functions despite taking opaque pointer first
FREE_FUNCTIONS = [
'SDL_SomeSpecialCase',
]
# Custom type mappings
TYPE_OVERRIDES = {
'SDL_bool': 'bool',
'void*': '*anyopaque',
}
# Headers to skip
SKIP_HEADERS = [
'SDL_test_*.h', # Test framework
'SDL_oldnames.h', # Deprecated
]
# Custom naming rules
NAMING_OVERRIDES = {
'SDL_bool': 'bool',
}
Parsing Challenges & Solutions
Challenge 1: Multi-line Declarations
C allows declarations to span multiple lines:
extern SDL_DECLSPEC SDL_GPUDevice * SDLCALL
SDL_CreateGPUDevice(
SDL_GPUShaderFormat format_flags,
bool debug_mode,
const char *name);
Solution: Normalize whitespace before parsing, treat newlines as spaces inside declarations.
Challenge 2: Documentation Comment Association
Comments must be correctly associated with the following declaration:
/**
* Creates a device.
*/
typedef struct SDL_GPUDevice SDL_GPUDevice; // This gets the comment
Solution: Track the "pending comment" and attach it to the next declaration.
Challenge 3: Macro Values
Flag values use macros:
#define SDL_GPU_TEXTUREUSAGE_SAMPLER (1u << 0)
Solution: Evaluate simple expressions (bit shifts, arithmetic) during parsing.
Challenge 4: Nested Structs
SDL rarely uses these, but they can appear:
typedef struct SDL_Foo {
struct {
int x, y;
} point;
} SDL_Foo;
Solution: Flatten or generate anonymous struct types as needed.
Challenge 5: Function Pointers in Structs
typedef struct SDL_Foo {
void (*callback)(void *userdata);
} SDL_Foo;
Solution: Convert to Zig function pointer syntax:
callback: ?*const fn (userdata: ?*anyopaque) callconv(.C) void,
Challenge 6: Forward Declarations
typedef struct SDL_Surface SDL_Surface; // Forward declaration
// ... later ...
typedef struct SDL_Surface {
// actual definition
} SDL_Surface;
Solution: Track forward declarations, replace with full definition when found.
Challenge 7: Conditional Compilation
#ifdef SDL_PLATFORM_WIN32
typedef HWND SDL_WindowHandle;
#else
typedef void* SDL_WindowHandle;
#endif
Solution: Either:
- Parse all branches and generate conditional Zig code
- Use platform-specific configuration
- Default to most general case
Edge Cases & Special Handling
1. Properties API
SDL3 has a properties system with string constants:
#define SDL_PROP_GPU_DEVICE_CREATE_DEBUGMODE_BOOLEAN "SDL.gpu.device.create.debugmode"
These should be preserved as string constants in Zig.
2. Callbacks
Function pointer types need special handling:
typedef void (*SDL_SomeCallback)(void *userdata);
Map to Zig function pointers:
pub const SomeCallback = *const fn (userdata: ?*anyopaque) callconv(.C) void;
3. Union Types
SDL uses unions in some places:
typedef union SDL_Event {
Uint32 type;
SDL_WindowEvent window;
} SDL_Event;
Map to Zig extern unions:
pub const Event = extern union {
type: u32,
window: WindowEvent,
};
4. Variadic Functions
Some SDL functions are variadic (e.g., SDL_Log). These should be marked appropriately or wrapped.
5. Platform-Specific Types
Handle with conditional compilation:
pub const WindowsHandle = if (builtin.os.tag == .windows) *c.HWND else *anyopaque;
6. Anonymous Structs/Enums
These rarely appear in SDL3 public headers but should be handled if encountered.
Success Criteria
The parser/generator is successful when:
- ✅ All 85 SDL3 headers can be parsed without errors
- ✅ Generated Zig code compiles without warnings
- ✅ Generated API is 100% complete (no missing functions/types)
- ✅ Generated code matches hand-written style
- ✅ Documentation is preserved and readable
- ✅ Build time is reasonable (<5 seconds for full regeneration)
- ✅ Integration tests pass with generated bindings
- ✅ Code is maintainable and well-documented
Future Enhancements
Phase 2 Features
- Multi-language support: Generate bindings for other languages
- Documentation generation: Create API documentation from parsed data
- Test generation: Auto-generate basic API tests
- Type-safe wrappers: Generate higher-level Zig wrappers with better error handling
- Backwards compatibility: Handle multiple SDL versions
Open Questions
-
Preprocessor handling: How much preprocessor complexity do we need to support?
- Answer: Start simple, expand as needed
-
Manual overrides: How do we handle cases where generated code isn't quite right?
- Answer: Configuration file + ability to exclude certain items from generation
-
Version tracking: How do we track which SDL version we're generating for?
- Answer: Parse version from SDL_version.h, embed in generated files
-
Breaking changes: What happens when SDL API changes?
- Answer: Regenerate, review diff, update override config if needed
-
Testing strategy: How do we test the generated bindings?
- Answer: Compile tests + comparison with hand-written + integration tests
Example Workflow
Here's how the parser would be used in practice:
# Build the parser
$ cd lib/sdl3/parser
$ zig build
# Test it lists headers correctly
$ zig build run -- ../SDL/include/SDL3
SDL3 Header Parser
==================
Scanning headers in: ../SDL/include/SDL3
[1] SDL_gpu.h
[2] SDL_video.h
...
Total headers found: 85
# Parse a single header (future)
$ zig build run -- ../SDL/include/SDL3/SDL_gpu.h --output ../src/gpu.zig
# Parse all headers (future)
$ zig build run -- ../SDL/include/SDL3 --output-dir ../src
# Compare with hand-written
$ diff ../src/gpu.zig ../src/gpu.zig.backup
# Build and test the generated bindings
$ cd ../.. && zig build test
Recommended Development Workflow:
- Implement lexer with comprehensive tests
- Implement syntax parser for basic patterns (enum, struct, function)
- Implement code generator for those patterns
- Test on subset of SDL_gpu.h (see "Recommended First Milestone")
- Iterate until output matches hand-written bindings
- Add semantic analysis (type resolution, function classification)
- Extend to full SDL_gpu.h
- Generalize to other headers one by one
- Add config system for edge cases
- Integrate into build system for automatic regeneration
Troubleshooting Guide
Problem: Generated code doesn't compile
Possible Causes:
- Type conversion is wrong (check C type → Zig type mapping)
- Cast is missing or incorrect (check @ptrCast, @bitCast usage)
- Missing import (check module dependencies)
- Struct field alignment issue (use
extern struct)
Solution:
- Compare with hand-written version
- Check
zig builderror message carefully - Verify the C type in header matches assumption
Problem: Parser fails to extract a declaration
Possible Causes:
- Multi-line declaration not handled
- Unexpected syntax/formatting
- Preprocessor directive interfering
- Comment breaking parser
Solution:
- Print the problematic line with context
- Check for unusual formatting
- Simplify the declaration in a test case
- Add special handling for this pattern
Problem: Generated function doesn't match hand-written
Possible Causes:
- Function classification is wrong (method vs. free function)
- Parameter types differ
- Cast strategy differs
- Naming convention mismatch
Solution:
- Review function classification rules
- Check parameter type conversions
- Verify cast strategy for each type
- Update naming rules in config
Problem: Flag structure has wrong padding
Possible Causes:
- Bit positions calculated incorrectly
- Backing type size wrong (u32 vs u64)
- Missing flags
Solution:
- Verify all flag definitions are found
- Check bit position extraction
- Ensure padding calculation accounts for all bits
- Validate backing type matches typedef
Problem: Cross-reference types not found
Possible Causes:
- Type defined in different header
- Forward declaration not resolved
- Module dependency missing
Solution:
- Parse dependent headers first
- Build complete type database
- Add explicit imports in generated code
- Check type dependency graph
References
- SDL3 repository: https://github.com/libsdl-org/SDL
- SDL3 headers:
lib/sdl3/SDL/include/SDL3/ - Existing bindings:
lib/sdl3/src/ - Zig documentation: https://ziglang.org/documentation/master/
- libclang Python: https://libclang.readthedocs.io/
- C11 Standard: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf
- Zig Language Reference: https://ziglang.org/documentation/master/
Next Steps
Current Status: ✅ Hello world implemented
Immediate Next Steps (This Week)
-
Implement
patterns.zig(2-3 days)- Create simple scanner that reads line-by-line
- Match pattern:
typedef struct SDL_Foo SDL_Foo;→ opaque - Match pattern:
typedef enum SDL_Foo {...} SDL_Foo;→ enum - Match pattern:
typedef struct SDL_Foo {...} SDL_Foo;→ struct - Store in simple structs (no complex AST)
-
Implement
codegen.zig+ helpers (2-3 days)- Create
naming.zigfor name conversion (SDL_GPUDevice → GPUDevice) - Create
types.zigfor type conversion (Uint32 → u32, float → f32) - Generate Zig code directly from extracted data
- Test on subset of SDL_gpu.h
- Create
-
Add flags and functions (2-3 days)
- Parse flag typedefs + #define sequences
- Parse function declarations
- Classify as method or free function (check first param)
- Generate complete gpu.zig
Following Week
-
Test on other headers (1-2 days)
- Try SDL_video.h, SDL_events.h
- Fix any new patterns
- Handle edge cases
-
Polish and integrate (1-2 days)
- Clean up code
- Add tests
- Update build system
- Document usage
Total: 2-3 weeks to complete parser
Key Takeaways
- SDL3 headers are EXTREMELY regular - Perfect for simple pattern matching
- Don't over-engineer - Text transformation is sufficient, no need for full parser
- Start small - Get pattern matching working for one header first
- Use hand-written as reference - The existing gpu.zig shows exactly what we want
- Iterate quickly - Scan, generate, compile, compare, fix, repeat
- Line-by-line scanning works - No need for tokenizer/lexer
- Direct generation is simpler - No need for AST, just extract and generate
- Simple pattern matching -
typedef struct SDL_Foo SDL_Foo;is a one-line pattern - Brace counting is enough - Track
{and}for multi-line declarations - String conversion for types - Table lookup, no need to parse type expressions
- Function classification is simple - Check if first param is opaque type
- Implementation time: 2-3 weeks (vs. 6-8 weeks for over-engineered approach)
Conclusion
This plan provides a comprehensive roadmap for creating an SDL3 header parser and Zig binding generator. The regular structure of SDL3 headers makes this an ideal project for automated code generation. By following this plan, we can create maintainable, high-quality Zig bindings that stay synchronized with SDL3 development.
The project is feasible because:
- SDL3 headers are EXTREMELY regular - Simple pattern matching works
- We have excellent reference implementations - Hand-written bindings show target output
- Text transformation is sufficient - No need for complex parsing
- The scope is well-defined - 85 headers, 5-6 simple patterns
- Zig provides excellent tooling - String manipulation, arena allocation, fast compilation
- No external dependencies - Single binary, easy integration
With a simplified approach using pattern matching instead of full parsing, this can be completed in 2-3 weeks of focused work (vs. 6-8 weeks for over-engineered approach). The result will be a maintainable system that generates high-quality Zig bindings automatically.
Advantages of Simplified Approach
- Simplicity: ~500 lines of code vs. 2000+ for full parser
- Speed: Faster to implement and faster to execute
- Maintainability: Easy to understand and modify
- Reliability: Less code = fewer bugs
- Sufficiency: SDL3 headers don't need full C parsing
- Quick iteration: Changes are fast to test
Simplified Parser Flow
C Header File
↓
┌──────────────────────────────────┐
│ patterns.zig (Scanner) │
│ Line-by-line pattern matching │
│ - typedef struct SDL_Foo... │
│ - typedef enum SDL_Foo {... │
│ - typedef Uint32 SDL_Flags; │
│ - extern SDL_DECLSPEC... │
└──────────────────┬───────────────┘
│
▼
Simple Struct Data
(name, fields, values, etc.)
│
▼
┌──────────────────────────────────┐
│ codegen.zig │
│ Direct string generation │
│ + naming.zig (name conversion) │
│ + types.zig (type conversion) │
└──────────────────┬───────────────┘
│
▼
gpu.zig (output)
Complexity: Very Low - just pattern matching and string generation!