dev/sdl3-parser #1
|
|
@ -6,6 +6,7 @@ A Zig tool that automatically generates idiomatic Zig bindings from SDL3 C heade
|
||||||
|
|
||||||
✅ **Automatic Dependency Resolution** - Detects and extracts missing types from included headers
|
✅ **Automatic Dependency Resolution** - Detects and extracts missing types from included headers
|
||||||
✅ **Multi-Field Struct Parsing** - Handles compact C syntax like `int x, y;`
|
✅ **Multi-Field Struct Parsing** - Handles compact C syntax like `int x, y;`
|
||||||
|
✅ **JSON Output** - Export structured JSON representation of all parsed types
|
||||||
✅ **Type Conversion** - Converts C types to idiomatic Zig types
|
✅ **Type Conversion** - Converts C types to idiomatic Zig types
|
||||||
✅ **Method Organization** - Groups functions as methods on opaque types
|
✅ **Method Organization** - Groups functions as methods on opaque types
|
||||||
✅ **Mock Generation** - Creates C stub implementations for testing
|
✅ **Mock Generation** - Creates C stub implementations for testing
|
||||||
|
|
@ -29,6 +30,9 @@ zig build run -- ../SDL/include/SDL3/SDL_gpu.h --output=gpu.zig
|
||||||
|
|
||||||
# Generate with C mocks for testing
|
# Generate with C mocks for testing
|
||||||
zig build run -- ../SDL/include/SDL3/SDL_gpu.h --output=gpu.zig --mocks=gpu_mock.c
|
zig build run -- ../SDL/include/SDL3/SDL_gpu.h --output=gpu.zig --mocks=gpu_mock.c
|
||||||
|
|
||||||
|
# Generate JSON representation
|
||||||
|
zig build run -- ../SDL/include/SDL3/SDL_gpu.h --generate-json=gpu.json
|
||||||
```
|
```
|
||||||
|
|
||||||
### Example Output
|
### Example Output
|
||||||
|
|
|
||||||
|
|
@ -40,6 +40,14 @@ Examples:
|
||||||
--mocks=test/mocks.c
|
--mocks=test/mocks.c
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**`--generate-json=<file>`** - Generate JSON representation of parsed types
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
```bash
|
||||||
|
--generate-json=gpu.json
|
||||||
|
--generate-json=api/types.json
|
||||||
|
```
|
||||||
|
|
||||||
## Output Formats
|
## Output Formats
|
||||||
|
|
||||||
### Zig Bindings (Default)
|
### Zig Bindings (Default)
|
||||||
|
|
@ -81,6 +89,55 @@ void SDL_DestroyGPUDevice(SDL_GPUDevice *device) {
|
||||||
|
|
||||||
**Use Case**: Testing without real SDL implementation
|
**Use Case**: Testing without real SDL implementation
|
||||||
|
|
||||||
|
### JSON Output (Optional)
|
||||||
|
|
||||||
|
Generated when `--generate-json` is specified:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"header": "SDL_gpu.h",
|
||||||
|
"opaque_types": [
|
||||||
|
{"name": "SDL_GPUDevice"}
|
||||||
|
],
|
||||||
|
"typedefs": [
|
||||||
|
{"name": "SDL_PropertiesID", "underlying_type": "Uint32"}
|
||||||
|
],
|
||||||
|
"enums": [
|
||||||
|
{
|
||||||
|
"name": "SDL_GPUPrimitiveType",
|
||||||
|
"values": [
|
||||||
|
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLELIST"},
|
||||||
|
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLESTRIP"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"structs": [
|
||||||
|
{
|
||||||
|
"name": "SDL_GPUViewport",
|
||||||
|
"fields": [
|
||||||
|
{"name": "x", "type": "float"},
|
||||||
|
{"name": "y", "type": "float"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"functions": [
|
||||||
|
{
|
||||||
|
"name": "SDL_CreateGPUDevice",
|
||||||
|
"return_type": "SDL_GPUDevice*",
|
||||||
|
"parameters": [
|
||||||
|
{"name": "debug_mode", "type": "bool"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Use Cases**:
|
||||||
|
- API documentation generation
|
||||||
|
- Schema validation
|
||||||
|
- Cross-language binding generation
|
||||||
|
- Type introspection tools
|
||||||
|
|
||||||
## Build System Integration
|
## Build System Integration
|
||||||
|
|
||||||
### In build.zig
|
### In build.zig
|
||||||
|
|
|
||||||
|
|
@ -1,306 +0,0 @@
|
||||||
# JSON Output Implementation Plan
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
Add a `--generate-json` flag to the parser that outputs all parsed declarations (types, enums, functions, etc.) as a structured JSON file for external tooling and analysis.
|
|
||||||
|
|
||||||
## Design Decisions
|
|
||||||
|
|
||||||
### 1. JSON Schema Design
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"header": "SDL_gpu.h",
|
|
||||||
"parsed_at": "2026-01-22T23:23:35Z",
|
|
||||||
"declarations": {
|
|
||||||
"opaque_types": [
|
|
||||||
{
|
|
||||||
"name": "SDL_GPUDevice",
|
|
||||||
"doc_comment": "/**\n * Opaque handle to a GPU device\n */"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"typedefs": [
|
|
||||||
{
|
|
||||||
"name": "SDL_PropertiesID",
|
|
||||||
"underlying_type": "Uint32",
|
|
||||||
"doc_comment": "..."
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"function_pointers": [
|
|
||||||
{
|
|
||||||
"name": "SDL_TimerCallback",
|
|
||||||
"return_type": "Uint32",
|
|
||||||
"params": [
|
|
||||||
{"name": "userdata", "type": "void *"},
|
|
||||||
{"name": "timerID", "type": "SDL_TimerID"}
|
|
||||||
],
|
|
||||||
"doc_comment": "..."
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"enums": [
|
|
||||||
{
|
|
||||||
"name": "SDL_GPUPrimitiveType",
|
|
||||||
"values": [
|
|
||||||
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLELIST", "value": "0", "comment": "..."},
|
|
||||||
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLESTRIP", "value": null, "comment": "..."}
|
|
||||||
],
|
|
||||||
"doc_comment": "..."
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"structs": [
|
|
||||||
{
|
|
||||||
"name": "SDL_GPUViewport",
|
|
||||||
"fields": [
|
|
||||||
{"name": "x", "type": "float", "comment": "..."},
|
|
||||||
{"name": "y", "type": "float", "comment": "..."}
|
|
||||||
],
|
|
||||||
"doc_comment": "..."
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"unions": [
|
|
||||||
{
|
|
||||||
"name": "SDL_Event",
|
|
||||||
"fields": [
|
|
||||||
{"name": "type", "type": "Uint32", "comment": "..."}
|
|
||||||
],
|
|
||||||
"doc_comment": "..."
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"flags": [
|
|
||||||
{
|
|
||||||
"name": "SDL_GPUTextureUsageFlags",
|
|
||||||
"underlying_type": "Uint32",
|
|
||||||
"flags": [
|
|
||||||
{"name": "SDL_GPU_TEXTUREUSAGE_SAMPLER", "value": "(1u << 0)", "comment": "..."}
|
|
||||||
],
|
|
||||||
"doc_comment": "..."
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"functions": [
|
|
||||||
{
|
|
||||||
"name": "SDL_CreateGPUDevice",
|
|
||||||
"return_type": "SDL_GPUDevice *",
|
|
||||||
"params": [
|
|
||||||
{"name": "format_flags", "type": "SDL_GPUShaderFormat"}
|
|
||||||
],
|
|
||||||
"doc_comment": "..."
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"statistics": {
|
|
||||||
"total_declarations": 150,
|
|
||||||
"opaque_types": 5,
|
|
||||||
"typedefs": 10,
|
|
||||||
"function_pointers": 3,
|
|
||||||
"enums": 15,
|
|
||||||
"structs": 25,
|
|
||||||
"unions": 2,
|
|
||||||
"flags": 10,
|
|
||||||
"functions": 80
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Command Line Interface
|
|
||||||
```bash
|
|
||||||
# Output JSON to stdout
|
|
||||||
parser SDL_gpu.h --generate-json
|
|
||||||
|
|
||||||
# Output JSON to file
|
|
||||||
parser SDL_gpu.h --generate-json=output.json
|
|
||||||
|
|
||||||
# Combine with other outputs
|
|
||||||
parser SDL_gpu.h --output=gpu.zig --generate-json=gpu.json
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Implementation Strategy
|
|
||||||
|
|
||||||
#### Phase 1: Create JSON Serializer Module
|
|
||||||
- Create `src/json_output.zig`
|
|
||||||
- Implement serialization functions for each declaration type
|
|
||||||
- Handle proper escaping of strings (especially doc comments with quotes/newlines)
|
|
||||||
- Use `std.json.stringify` for structured output
|
|
||||||
|
|
||||||
#### Phase 2: Update Command Line Parsing
|
|
||||||
- Add `--generate-json` and `--generate-json=<file>` flag parsing in `parser.zig`
|
|
||||||
- Store flag in configuration structure
|
|
||||||
|
|
||||||
#### Phase 3: Integrate with Main Parser Flow
|
|
||||||
- After `scanner.scan()` and dependency resolution
|
|
||||||
- Before or after Zig code generation
|
|
||||||
- Call JSON serializer with full declaration list
|
|
||||||
|
|
||||||
#### Phase 4: Testing
|
|
||||||
- Test with multiple SDL headers
|
|
||||||
- Verify JSON is valid and well-formed
|
|
||||||
- Test edge cases: empty comments, special characters, null values
|
|
||||||
- Validate against JSON schema
|
|
||||||
|
|
||||||
## Implementation Details
|
|
||||||
|
|
||||||
### Module Structure (`src/json_output.zig`)
|
|
||||||
|
|
||||||
```zig
|
|
||||||
const std = @import("std");
|
|
||||||
const patterns = @import("patterns.zig");
|
|
||||||
const Allocator = std.mem.Allocator;
|
|
||||||
|
|
||||||
pub fn writeJson(
|
|
||||||
allocator: Allocator,
|
|
||||||
writer: anytype,
|
|
||||||
header_name: []const u8,
|
|
||||||
decls: []const patterns.Declaration,
|
|
||||||
) !void {
|
|
||||||
// Write JSON structure
|
|
||||||
}
|
|
||||||
|
|
||||||
fn writeOpaqueType(writer: anytype, opaque: patterns.OpaqueType) !void;
|
|
||||||
fn writeTypedef(writer: anytype, typedef: patterns.TypedefDecl) !void;
|
|
||||||
fn writeFunctionPointer(writer: anytype, func_ptr: patterns.FunctionPointerDecl) !void;
|
|
||||||
fn writeEnum(writer: anytype, enum_decl: patterns.EnumDecl) !void;
|
|
||||||
fn writeStruct(writer: anytype, struct_decl: patterns.StructDecl) !void;
|
|
||||||
fn writeUnion(writer: anytype, union_decl: patterns.UnionDecl) !void;
|
|
||||||
fn writeFlags(writer: anytype, flags: patterns.FlagDecl) !void;
|
|
||||||
fn writeFunction(writer: anytype, func: patterns.FunctionDecl) !void;
|
|
||||||
|
|
||||||
fn escapeString(allocator: Allocator, str: []const u8) ![]u8;
|
|
||||||
```
|
|
||||||
|
|
||||||
### Updates to `parser.zig`
|
|
||||||
|
|
||||||
```zig
|
|
||||||
// Add after argument parsing
|
|
||||||
var json_output_file: ?[]const u8 = null;
|
|
||||||
|
|
||||||
for (args[2..]) |arg| {
|
|
||||||
// ... existing flags ...
|
|
||||||
const json_prefix = "--generate-json";
|
|
||||||
if (std.mem.eql(u8, arg, json_prefix)) {
|
|
||||||
json_output_file = ""; // stdout
|
|
||||||
} else if (std.mem.startsWith(u8, arg, json_prefix ++ "=")) {
|
|
||||||
json_output_file = arg[(json_prefix.len + 1)..];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add after dependency resolution
|
|
||||||
if (json_output_file) |json_file| {
|
|
||||||
std.debug.print("Generating JSON output...\n", .{});
|
|
||||||
if (json_file.len == 0) {
|
|
||||||
// Write to stdout
|
|
||||||
const stdout = std.io.getStdOut().writer();
|
|
||||||
try json_output.writeJson(allocator, stdout, header_path, decls);
|
|
||||||
} else {
|
|
||||||
// Write to file
|
|
||||||
const file = try std.fs.cwd().createFile(json_file, .{});
|
|
||||||
defer file.close();
|
|
||||||
const writer = file.writer();
|
|
||||||
try json_output.writeJson(allocator, writer, header_path, decls);
|
|
||||||
std.debug.print("JSON written to: {s}\n", .{json_file});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Edge Cases to Handle
|
|
||||||
|
|
||||||
1. **Null/Optional Fields**: doc_comment, enum values, field comments
|
|
||||||
2. **String Escaping**: Quotes, newlines, backslashes in doc comments
|
|
||||||
3. **Special Characters**: Unicode in comments or identifiers
|
|
||||||
4. **Empty Arrays**: Structs with no fields, enums with no values
|
|
||||||
5. **Large Output**: Efficient writing without loading entire JSON in memory
|
|
||||||
6. **Mixed Output**: Ensure JSON doesn't interfere with stderr debug output
|
|
||||||
|
|
||||||
## Success Criteria
|
|
||||||
|
|
||||||
- [ ] Can parse any SDL header and output valid JSON
|
|
||||||
- [ ] JSON validates against standard JSON parsers (jq, Python json module)
|
|
||||||
- [ ] All declaration types are represented
|
|
||||||
- [ ] Doc comments are preserved with proper escaping
|
|
||||||
- [ ] Statistics section is accurate
|
|
||||||
- [ ] Can output to both stdout and file
|
|
||||||
- [ ] Works alongside existing --output and --mocks flags
|
|
||||||
- [ ] No memory leaks in JSON generation path
|
|
||||||
|
|
||||||
## Testing Plan
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Test basic functionality
|
|
||||||
./parser ../SDL/include/SDL3/SDL_gpu.h --generate-json | jq .
|
|
||||||
|
|
||||||
# Test with file output
|
|
||||||
./parser ../SDL/include/SDL3/SDL_gpu.h --generate-json=gpu.json
|
|
||||||
cat gpu.json | jq '.statistics'
|
|
||||||
|
|
||||||
# Test combined with Zig output
|
|
||||||
./parser ../SDL/include/SDL3/SDL_video.h --output=video.zig --generate-json=video.json
|
|
||||||
|
|
||||||
# Validate JSON structure
|
|
||||||
python3 -m json.tool gpu.json > /dev/null && echo "Valid JSON"
|
|
||||||
|
|
||||||
# Test edge cases
|
|
||||||
./parser test_small.h --generate-json | jq '.declarations.functions[0].doc_comment'
|
|
||||||
```
|
|
||||||
|
|
||||||
## Future Enhancements (Not in Scope)
|
|
||||||
|
|
||||||
- JSON Schema file generation
|
|
||||||
- Filtering by declaration type (e.g., only functions)
|
|
||||||
- Dependency graph in JSON format
|
|
||||||
- Diff mode between two JSON outputs
|
|
||||||
- Machine-readable error format
|
|
||||||
|
|
||||||
## Iteration Notes
|
|
||||||
|
|
||||||
### Iteration 1 Considerations:
|
|
||||||
- Should we include dependency information in JSON?
|
|
||||||
- **Decision**: No, keep it simple. Focus on declarations only.
|
|
||||||
- Should we include source location (line numbers)?
|
|
||||||
- **Decision**: Future enhancement. Not in initial scope.
|
|
||||||
- Should JSON output be pretty-printed or compact?
|
|
||||||
- **Decision**: Pretty-printed with 2-space indentation for readability.
|
|
||||||
- Error handling: What if JSON write fails partway through?
|
|
||||||
- **Decision**: Write to temporary file first, rename on success. For stdout, fail fast.
|
|
||||||
|
|
||||||
### Iteration 2 Review:
|
|
||||||
Looking at the plan again:
|
|
||||||
|
|
||||||
**Strengths:**
|
|
||||||
- Clear JSON schema design
|
|
||||||
- Comprehensive edge case handling
|
|
||||||
- Good testing plan
|
|
||||||
- Realistic scope
|
|
||||||
|
|
||||||
**Potential Issues:**
|
|
||||||
- Need to handle timestamp generation (use std.time)
|
|
||||||
- Should verify that nested JSON writing doesn't cause stack overflow
|
|
||||||
- Consider buffering for large outputs
|
|
||||||
- Add validation that string escaping handles all C comment styles
|
|
||||||
|
|
||||||
**Refinements:**
|
|
||||||
- Add buffered writer wrapper for performance
|
|
||||||
- Use `std.json.writeStream` if available in Zig 0.14
|
|
||||||
- Add --json-pretty flag to control formatting
|
|
||||||
- Document that all strings are UTF-8 encoded
|
|
||||||
|
|
||||||
### Final Confidence Assessment:
|
|
||||||
|
|
||||||
✅ **High Confidence Areas:**
|
|
||||||
- JSON schema design is complete and covers all declaration types
|
|
||||||
- Integration points are well-defined
|
|
||||||
- Testing approach is thorough
|
|
||||||
|
|
||||||
⚠️ **Medium Confidence Areas:**
|
|
||||||
- String escaping complexity (especially multi-line doc comments)
|
|
||||||
- Performance with very large headers
|
|
||||||
- Error recovery during JSON generation
|
|
||||||
|
|
||||||
✅ **Ready to Implement:**
|
|
||||||
The plan is comprehensive and actionable. We should proceed with implementation.
|
|
||||||
|
|
||||||
## Implementation Order
|
|
||||||
|
|
||||||
1. Create `src/json_output.zig` with basic structure
|
|
||||||
2. Implement individual serialization functions
|
|
||||||
3. Add command line flag parsing
|
|
||||||
4. Integrate into main parser flow
|
|
||||||
5. Test with SDL_gpu.h (known good header)
|
|
||||||
6. Test with SDL_video.h (larger header)
|
|
||||||
7. Test edge cases and error conditions
|
|
||||||
8. Update documentation
|
|
||||||
Loading…
Reference in New Issue