peterino
/
Backlog
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity

dev/sdl3-parser #1

Open
peterino wants to merge 51 commits from dev/sdl3-parser into dev/variant-engine
pull from: dev/sdl3-parser
merge into: peterino:dev/variant-engine
Conversation 0 Commits 51 Files Changed 105 +61 -306
3 changed files with 61 additions and 306 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 002ceb891a - Show all commits

4
lib/sdl3/parser/README.md vendored
View File

@ -6,6 +6,7 @@ A Zig tool that automatically generates idiomatic Zig bindings from SDL3 C heade
✅ **Automatic Dependency Resolution** - Detects and extracts missing types from included headers
✅ **Multi-Field Struct Parsing** - Handles compact C syntax like `int x, y;`
✅ **JSON Output** - Export structured JSON representation of all parsed types
✅ **Type Conversion** - Converts C types to idiomatic Zig types
✅ **Method Organization** - Groups functions as methods on opaque types
✅ **Mock Generation** - Creates C stub implementations for testing
@ -29,6 +30,9 @@ zig build run -- ../SDL/include/SDL3/SDL_gpu.h --output=gpu.zig
# Generate with C mocks for testing
zig build run -- ../SDL/include/SDL3/SDL_gpu.h --output=gpu.zig --mocks=gpu_mock.c
# Generate JSON representation
zig build run -- ../SDL/include/SDL3/SDL_gpu.h --generate-json=gpu.json
```
### Example Output

57
lib/sdl3/parser/docs/API_REFERENCE.md vendored
View File

@ -40,6 +40,14 @@ Examples:
--mocks=test/mocks.c
```
**`--generate-json=<file>`** - Generate JSON representation of parsed types
Examples:
```bash
--generate-json=gpu.json
--generate-json=api/types.json
```
## Output Formats
### Zig Bindings (Default)
@ -81,6 +89,55 @@ void SDL_DestroyGPUDevice(SDL_GPUDevice *device) {
**Use Case**: Testing without real SDL implementation
### JSON Output (Optional)
Generated when `--generate-json` is specified:
```json
{
"header": "SDL_gpu.h",
"opaque_types": [
{"name": "SDL_GPUDevice"}
],
"typedefs": [
{"name": "SDL_PropertiesID", "underlying_type": "Uint32"}
],
"enums": [
{
"name": "SDL_GPUPrimitiveType",
"values": [
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLELIST"},
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLESTRIP"}
]
}
],
"structs": [
{
"name": "SDL_GPUViewport",
"fields": [
{"name": "x", "type": "float"},
{"name": "y", "type": "float"}
]
}
],
"functions": [
{
"name": "SDL_CreateGPUDevice",
"return_type": "SDL_GPUDevice*",
"parameters": [
{"name": "debug_mode", "type": "bool"}
]
}
]
}
```
**Use Cases**:
- API documentation generation
- Schema validation
- Cross-language binding generation
- Type introspection tools
## Build System Integration
### In build.zig

306
lib/sdl3/parser/docs/JSON_OUTPUT_PLAN.md vendored
View File

@ -1,306 +0,0 @@
# JSON Output Implementation Plan
## Goal
Add a `--generate-json` flag to the parser that outputs all parsed declarations (types, enums, functions, etc.) as a structured JSON file for external tooling and analysis.
## Design Decisions
### 1. JSON Schema Design
```json
{
"header": "SDL_gpu.h",
"parsed_at": "2026-01-22T23:23:35Z",
"declarations": {
"opaque_types": [
{
"name": "SDL_GPUDevice",
"doc_comment": "/**\n * Opaque handle to a GPU device\n */"
}
],
"typedefs": [
{
"name": "SDL_PropertiesID",
"underlying_type": "Uint32",
"doc_comment": "..."
}
],
"function_pointers": [
{
"name": "SDL_TimerCallback",
"return_type": "Uint32",
"params": [
{"name": "userdata", "type": "void *"},
{"name": "timerID", "type": "SDL_TimerID"}
],
"doc_comment": "..."
}
],
"enums": [
{
"name": "SDL_GPUPrimitiveType",
"values": [
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLELIST", "value": "0", "comment": "..."},
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLESTRIP", "value": null, "comment": "..."}
],
"doc_comment": "..."
}
],
"structs": [
{
"name": "SDL_GPUViewport",
"fields": [
{"name": "x", "type": "float", "comment": "..."},
{"name": "y", "type": "float", "comment": "..."}
],
"doc_comment": "..."
}
],
"unions": [
{
"name": "SDL_Event",
"fields": [
{"name": "type", "type": "Uint32", "comment": "..."}
],
"doc_comment": "..."
}
],
"flags": [
{
"name": "SDL_GPUTextureUsageFlags",
"underlying_type": "Uint32",
"flags": [
{"name": "SDL_GPU_TEXTUREUSAGE_SAMPLER", "value": "(1u << 0)", "comment": "..."}
],
"doc_comment": "..."
}
],
"functions": [
{
"name": "SDL_CreateGPUDevice",
"return_type": "SDL_GPUDevice *",
"params": [
{"name": "format_flags", "type": "SDL_GPUShaderFormat"}
],
"doc_comment": "..."
}
]
},
"statistics": {
"total_declarations": 150,
"opaque_types": 5,
"typedefs": 10,
"function_pointers": 3,
"enums": 15,
"structs": 25,
"unions": 2,
"flags": 10,
"functions": 80
}
}
```
### 2. Command Line Interface
```bash
# Output JSON to stdout
parser SDL_gpu.h --generate-json
# Output JSON to file
parser SDL_gpu.h --generate-json=output.json
# Combine with other outputs
parser SDL_gpu.h --output=gpu.zig --generate-json=gpu.json
```
### 3. Implementation Strategy
#### Phase 1: Create JSON Serializer Module
- Create `src/json_output.zig`
- Implement serialization functions for each declaration type
- Handle proper escaping of strings (especially doc comments with quotes/newlines)
- Use `std.json.stringify` for structured output
#### Phase 2: Update Command Line Parsing
- Add `--generate-json` and `--generate-json=<file>` flag parsing in `parser.zig`
- Store flag in configuration structure
#### Phase 3: Integrate with Main Parser Flow
- After `scanner.scan()` and dependency resolution
- Before or after Zig code generation
- Call JSON serializer with full declaration list
#### Phase 4: Testing
- Test with multiple SDL headers
- Verify JSON is valid and well-formed
- Test edge cases: empty comments, special characters, null values
- Validate against JSON schema
## Implementation Details
### Module Structure (`src/json_output.zig`)
```zig
const std = @import("std");
const patterns = @import("patterns.zig");
const Allocator = std.mem.Allocator;
pub fn writeJson(
allocator: Allocator,
writer: anytype,
header_name: []const u8,
decls: []const patterns.Declaration,
) !void {
// Write JSON structure
}
fn writeOpaqueType(writer: anytype, opaque: patterns.OpaqueType) !void;
fn writeTypedef(writer: anytype, typedef: patterns.TypedefDecl) !void;
fn writeFunctionPointer(writer: anytype, func_ptr: patterns.FunctionPointerDecl) !void;
fn writeEnum(writer: anytype, enum_decl: patterns.EnumDecl) !void;
fn writeStruct(writer: anytype, struct_decl: patterns.StructDecl) !void;
fn writeUnion(writer: anytype, union_decl: patterns.UnionDecl) !void;
fn writeFlags(writer: anytype, flags: patterns.FlagDecl) !void;
fn writeFunction(writer: anytype, func: patterns.FunctionDecl) !void;
fn escapeString(allocator: Allocator, str: []const u8) ![]u8;
```
### Updates to `parser.zig`
```zig
// Add after argument parsing
var json_output_file: ?[]const u8 = null;
for (args[2..]) |arg| {
// ... existing flags ...
const json_prefix = "--generate-json";
if (std.mem.eql(u8, arg, json_prefix)) {
json_output_file = ""; // stdout
} else if (std.mem.startsWith(u8, arg, json_prefix ++ "=")) {
json_output_file = arg[(json_prefix.len + 1)..];
}
}
// Add after dependency resolution
if (json_output_file) |json_file| {
std.debug.print("Generating JSON output...\n", .{});
if (json_file.len == 0) {
// Write to stdout
const stdout = std.io.getStdOut().writer();
try json_output.writeJson(allocator, stdout, header_path, decls);
} else {
// Write to file
const file = try std.fs.cwd().createFile(json_file, .{});
defer file.close();
const writer = file.writer();
try json_output.writeJson(allocator, writer, header_path, decls);
std.debug.print("JSON written to: {s}\n", .{json_file});
}
}
```
## Edge Cases to Handle
1. **Null/Optional Fields**: doc_comment, enum values, field comments
2. **String Escaping**: Quotes, newlines, backslashes in doc comments
3. **Special Characters**: Unicode in comments or identifiers
4. **Empty Arrays**: Structs with no fields, enums with no values
5. **Large Output**: Efficient writing without loading entire JSON in memory
6. **Mixed Output**: Ensure JSON doesn't interfere with stderr debug output
## Success Criteria
- [ ] Can parse any SDL header and output valid JSON
- [ ] JSON validates against standard JSON parsers (jq, Python json module)
- [ ] All declaration types are represented
- [ ] Doc comments are preserved with proper escaping
- [ ] Statistics section is accurate
- [ ] Can output to both stdout and file
- [ ] Works alongside existing --output and --mocks flags
- [ ] No memory leaks in JSON generation path
## Testing Plan
```bash
# Test basic functionality
./parser ../SDL/include/SDL3/SDL_gpu.h --generate-json | jq .
# Test with file output
./parser ../SDL/include/SDL3/SDL_gpu.h --generate-json=gpu.json
cat gpu.json | jq '.statistics'
# Test combined with Zig output
./parser ../SDL/include/SDL3/SDL_video.h --output=video.zig --generate-json=video.json
# Validate JSON structure
python3 -m json.tool gpu.json > /dev/null && echo "Valid JSON"
# Test edge cases
./parser test_small.h --generate-json | jq '.declarations.functions[0].doc_comment'
```
## Future Enhancements (Not in Scope)
- JSON Schema file generation
- Filtering by declaration type (e.g., only functions)
- Dependency graph in JSON format
- Diff mode between two JSON outputs
- Machine-readable error format
## Iteration Notes
### Iteration 1 Considerations:
- Should we include dependency information in JSON?
- **Decision**: No, keep it simple. Focus on declarations only.
- Should we include source location (line numbers)?
- **Decision**: Future enhancement. Not in initial scope.
- Should JSON output be pretty-printed or compact?
- **Decision**: Pretty-printed with 2-space indentation for readability.
- Error handling: What if JSON write fails partway through?
- **Decision**: Write to temporary file first, rename on success. For stdout, fail fast.
### Iteration 2 Review:
Looking at the plan again:
**Strengths:**
- Clear JSON schema design
- Comprehensive edge case handling
- Good testing plan
- Realistic scope
**Potential Issues:**
- Need to handle timestamp generation (use std.time)
- Should verify that nested JSON writing doesn't cause stack overflow
- Consider buffering for large outputs
- Add validation that string escaping handles all C comment styles
**Refinements:**
- Add buffered writer wrapper for performance
- Use `std.json.writeStream` if available in Zig 0.14
- Add --json-pretty flag to control formatting
- Document that all strings are UTF-8 encoded
### Final Confidence Assessment:
✅ **High Confidence Areas:**
- JSON schema design is complete and covers all declaration types
- Integration points are well-defined
- Testing approach is thorough
⚠️ **Medium Confidence Areas:**
- String escaping complexity (especially multi-line doc comments)
- Performance with very large headers
- Error recovery during JSON generation
✅ **Ready to Implement:**
The plan is comprehensive and actionable. We should proceed with implementation.
## Implementation Order
1. Create `src/json_output.zig` with basic structure
2. Implement individual serialization functions
3. Add command line flag parsing
4. Integrate into main parser flow
5. Test with SDL_gpu.h (known good header)
6. Test with SDL_video.h (larger header)
7. Test edge cases and error conditions
8. Update documentation