Update documentation for JSON output feature
- Add JSON output to feature list in README - Document --generate-json flag in API_REFERENCE - Include JSON output example with use cases - Remove planning document (implementation complete)
This commit is contained in:
parent
b7ec134b0e
commit
002ceb891a
|
|
@ -6,6 +6,7 @@ A Zig tool that automatically generates idiomatic Zig bindings from SDL3 C heade
|
|||
|
||||
✅ **Automatic Dependency Resolution** - Detects and extracts missing types from included headers
|
||||
✅ **Multi-Field Struct Parsing** - Handles compact C syntax like `int x, y;`
|
||||
✅ **JSON Output** - Export structured JSON representation of all parsed types
|
||||
✅ **Type Conversion** - Converts C types to idiomatic Zig types
|
||||
✅ **Method Organization** - Groups functions as methods on opaque types
|
||||
✅ **Mock Generation** - Creates C stub implementations for testing
|
||||
|
|
@ -29,6 +30,9 @@ zig build run -- ../SDL/include/SDL3/SDL_gpu.h --output=gpu.zig
|
|||
|
||||
# Generate with C mocks for testing
|
||||
zig build run -- ../SDL/include/SDL3/SDL_gpu.h --output=gpu.zig --mocks=gpu_mock.c
|
||||
|
||||
# Generate JSON representation
|
||||
zig build run -- ../SDL/include/SDL3/SDL_gpu.h --generate-json=gpu.json
|
||||
```
|
||||
|
||||
### Example Output
|
||||
|
|
|
|||
|
|
@ -40,6 +40,14 @@ Examples:
|
|||
--mocks=test/mocks.c
|
||||
```
|
||||
|
||||
**`--generate-json=<file>`** - Generate JSON representation of parsed types
|
||||
|
||||
Examples:
|
||||
```bash
|
||||
--generate-json=gpu.json
|
||||
--generate-json=api/types.json
|
||||
```
|
||||
|
||||
## Output Formats
|
||||
|
||||
### Zig Bindings (Default)
|
||||
|
|
@ -81,6 +89,55 @@ void SDL_DestroyGPUDevice(SDL_GPUDevice *device) {
|
|||
|
||||
**Use Case**: Testing without real SDL implementation
|
||||
|
||||
### JSON Output (Optional)
|
||||
|
||||
Generated when `--generate-json` is specified:
|
||||
|
||||
```json
|
||||
{
|
||||
"header": "SDL_gpu.h",
|
||||
"opaque_types": [
|
||||
{"name": "SDL_GPUDevice"}
|
||||
],
|
||||
"typedefs": [
|
||||
{"name": "SDL_PropertiesID", "underlying_type": "Uint32"}
|
||||
],
|
||||
"enums": [
|
||||
{
|
||||
"name": "SDL_GPUPrimitiveType",
|
||||
"values": [
|
||||
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLELIST"},
|
||||
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLESTRIP"}
|
||||
]
|
||||
}
|
||||
],
|
||||
"structs": [
|
||||
{
|
||||
"name": "SDL_GPUViewport",
|
||||
"fields": [
|
||||
{"name": "x", "type": "float"},
|
||||
{"name": "y", "type": "float"}
|
||||
]
|
||||
}
|
||||
],
|
||||
"functions": [
|
||||
{
|
||||
"name": "SDL_CreateGPUDevice",
|
||||
"return_type": "SDL_GPUDevice*",
|
||||
"parameters": [
|
||||
{"name": "debug_mode", "type": "bool"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Use Cases**:
|
||||
- API documentation generation
|
||||
- Schema validation
|
||||
- Cross-language binding generation
|
||||
- Type introspection tools
|
||||
|
||||
## Build System Integration
|
||||
|
||||
### In build.zig
|
||||
|
|
|
|||
|
|
@ -1,306 +0,0 @@
|
|||
# JSON Output Implementation Plan
|
||||
|
||||
## Goal
|
||||
Add a `--generate-json` flag to the parser that outputs all parsed declarations (types, enums, functions, etc.) as a structured JSON file for external tooling and analysis.
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### 1. JSON Schema Design
|
||||
```json
|
||||
{
|
||||
"header": "SDL_gpu.h",
|
||||
"parsed_at": "2026-01-22T23:23:35Z",
|
||||
"declarations": {
|
||||
"opaque_types": [
|
||||
{
|
||||
"name": "SDL_GPUDevice",
|
||||
"doc_comment": "/**\n * Opaque handle to a GPU device\n */"
|
||||
}
|
||||
],
|
||||
"typedefs": [
|
||||
{
|
||||
"name": "SDL_PropertiesID",
|
||||
"underlying_type": "Uint32",
|
||||
"doc_comment": "..."
|
||||
}
|
||||
],
|
||||
"function_pointers": [
|
||||
{
|
||||
"name": "SDL_TimerCallback",
|
||||
"return_type": "Uint32",
|
||||
"params": [
|
||||
{"name": "userdata", "type": "void *"},
|
||||
{"name": "timerID", "type": "SDL_TimerID"}
|
||||
],
|
||||
"doc_comment": "..."
|
||||
}
|
||||
],
|
||||
"enums": [
|
||||
{
|
||||
"name": "SDL_GPUPrimitiveType",
|
||||
"values": [
|
||||
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLELIST", "value": "0", "comment": "..."},
|
||||
{"name": "SDL_GPU_PRIMITIVETYPE_TRIANGLESTRIP", "value": null, "comment": "..."}
|
||||
],
|
||||
"doc_comment": "..."
|
||||
}
|
||||
],
|
||||
"structs": [
|
||||
{
|
||||
"name": "SDL_GPUViewport",
|
||||
"fields": [
|
||||
{"name": "x", "type": "float", "comment": "..."},
|
||||
{"name": "y", "type": "float", "comment": "..."}
|
||||
],
|
||||
"doc_comment": "..."
|
||||
}
|
||||
],
|
||||
"unions": [
|
||||
{
|
||||
"name": "SDL_Event",
|
||||
"fields": [
|
||||
{"name": "type", "type": "Uint32", "comment": "..."}
|
||||
],
|
||||
"doc_comment": "..."
|
||||
}
|
||||
],
|
||||
"flags": [
|
||||
{
|
||||
"name": "SDL_GPUTextureUsageFlags",
|
||||
"underlying_type": "Uint32",
|
||||
"flags": [
|
||||
{"name": "SDL_GPU_TEXTUREUSAGE_SAMPLER", "value": "(1u << 0)", "comment": "..."}
|
||||
],
|
||||
"doc_comment": "..."
|
||||
}
|
||||
],
|
||||
"functions": [
|
||||
{
|
||||
"name": "SDL_CreateGPUDevice",
|
||||
"return_type": "SDL_GPUDevice *",
|
||||
"params": [
|
||||
{"name": "format_flags", "type": "SDL_GPUShaderFormat"}
|
||||
],
|
||||
"doc_comment": "..."
|
||||
}
|
||||
]
|
||||
},
|
||||
"statistics": {
|
||||
"total_declarations": 150,
|
||||
"opaque_types": 5,
|
||||
"typedefs": 10,
|
||||
"function_pointers": 3,
|
||||
"enums": 15,
|
||||
"structs": 25,
|
||||
"unions": 2,
|
||||
"flags": 10,
|
||||
"functions": 80
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Command Line Interface
|
||||
```bash
|
||||
# Output JSON to stdout
|
||||
parser SDL_gpu.h --generate-json
|
||||
|
||||
# Output JSON to file
|
||||
parser SDL_gpu.h --generate-json=output.json
|
||||
|
||||
# Combine with other outputs
|
||||
parser SDL_gpu.h --output=gpu.zig --generate-json=gpu.json
|
||||
```
|
||||
|
||||
### 3. Implementation Strategy
|
||||
|
||||
#### Phase 1: Create JSON Serializer Module
|
||||
- Create `src/json_output.zig`
|
||||
- Implement serialization functions for each declaration type
|
||||
- Handle proper escaping of strings (especially doc comments with quotes/newlines)
|
||||
- Use `std.json.stringify` for structured output
|
||||
|
||||
#### Phase 2: Update Command Line Parsing
|
||||
- Add `--generate-json` and `--generate-json=<file>` flag parsing in `parser.zig`
|
||||
- Store flag in configuration structure
|
||||
|
||||
#### Phase 3: Integrate with Main Parser Flow
|
||||
- After `scanner.scan()` and dependency resolution
|
||||
- Before or after Zig code generation
|
||||
- Call JSON serializer with full declaration list
|
||||
|
||||
#### Phase 4: Testing
|
||||
- Test with multiple SDL headers
|
||||
- Verify JSON is valid and well-formed
|
||||
- Test edge cases: empty comments, special characters, null values
|
||||
- Validate against JSON schema
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Module Structure (`src/json_output.zig`)
|
||||
|
||||
```zig
|
||||
const std = @import("std");
|
||||
const patterns = @import("patterns.zig");
|
||||
const Allocator = std.mem.Allocator;
|
||||
|
||||
pub fn writeJson(
|
||||
allocator: Allocator,
|
||||
writer: anytype,
|
||||
header_name: []const u8,
|
||||
decls: []const patterns.Declaration,
|
||||
) !void {
|
||||
// Write JSON structure
|
||||
}
|
||||
|
||||
fn writeOpaqueType(writer: anytype, opaque: patterns.OpaqueType) !void;
|
||||
fn writeTypedef(writer: anytype, typedef: patterns.TypedefDecl) !void;
|
||||
fn writeFunctionPointer(writer: anytype, func_ptr: patterns.FunctionPointerDecl) !void;
|
||||
fn writeEnum(writer: anytype, enum_decl: patterns.EnumDecl) !void;
|
||||
fn writeStruct(writer: anytype, struct_decl: patterns.StructDecl) !void;
|
||||
fn writeUnion(writer: anytype, union_decl: patterns.UnionDecl) !void;
|
||||
fn writeFlags(writer: anytype, flags: patterns.FlagDecl) !void;
|
||||
fn writeFunction(writer: anytype, func: patterns.FunctionDecl) !void;
|
||||
|
||||
fn escapeString(allocator: Allocator, str: []const u8) ![]u8;
|
||||
```
|
||||
|
||||
### Updates to `parser.zig`
|
||||
|
||||
```zig
|
||||
// Add after argument parsing
|
||||
var json_output_file: ?[]const u8 = null;
|
||||
|
||||
for (args[2..]) |arg| {
|
||||
// ... existing flags ...
|
||||
const json_prefix = "--generate-json";
|
||||
if (std.mem.eql(u8, arg, json_prefix)) {
|
||||
json_output_file = ""; // stdout
|
||||
} else if (std.mem.startsWith(u8, arg, json_prefix ++ "=")) {
|
||||
json_output_file = arg[(json_prefix.len + 1)..];
|
||||
}
|
||||
}
|
||||
|
||||
// Add after dependency resolution
|
||||
if (json_output_file) |json_file| {
|
||||
std.debug.print("Generating JSON output...\n", .{});
|
||||
if (json_file.len == 0) {
|
||||
// Write to stdout
|
||||
const stdout = std.io.getStdOut().writer();
|
||||
try json_output.writeJson(allocator, stdout, header_path, decls);
|
||||
} else {
|
||||
// Write to file
|
||||
const file = try std.fs.cwd().createFile(json_file, .{});
|
||||
defer file.close();
|
||||
const writer = file.writer();
|
||||
try json_output.writeJson(allocator, writer, header_path, decls);
|
||||
std.debug.print("JSON written to: {s}\n", .{json_file});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Edge Cases to Handle
|
||||
|
||||
1. **Null/Optional Fields**: doc_comment, enum values, field comments
|
||||
2. **String Escaping**: Quotes, newlines, backslashes in doc comments
|
||||
3. **Special Characters**: Unicode in comments or identifiers
|
||||
4. **Empty Arrays**: Structs with no fields, enums with no values
|
||||
5. **Large Output**: Efficient writing without loading entire JSON in memory
|
||||
6. **Mixed Output**: Ensure JSON doesn't interfere with stderr debug output
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Can parse any SDL header and output valid JSON
|
||||
- [ ] JSON validates against standard JSON parsers (jq, Python json module)
|
||||
- [ ] All declaration types are represented
|
||||
- [ ] Doc comments are preserved with proper escaping
|
||||
- [ ] Statistics section is accurate
|
||||
- [ ] Can output to both stdout and file
|
||||
- [ ] Works alongside existing --output and --mocks flags
|
||||
- [ ] No memory leaks in JSON generation path
|
||||
|
||||
## Testing Plan
|
||||
|
||||
```bash
|
||||
# Test basic functionality
|
||||
./parser ../SDL/include/SDL3/SDL_gpu.h --generate-json | jq .
|
||||
|
||||
# Test with file output
|
||||
./parser ../SDL/include/SDL3/SDL_gpu.h --generate-json=gpu.json
|
||||
cat gpu.json | jq '.statistics'
|
||||
|
||||
# Test combined with Zig output
|
||||
./parser ../SDL/include/SDL3/SDL_video.h --output=video.zig --generate-json=video.json
|
||||
|
||||
# Validate JSON structure
|
||||
python3 -m json.tool gpu.json > /dev/null && echo "Valid JSON"
|
||||
|
||||
# Test edge cases
|
||||
./parser test_small.h --generate-json | jq '.declarations.functions[0].doc_comment'
|
||||
```
|
||||
|
||||
## Future Enhancements (Not in Scope)
|
||||
|
||||
- JSON Schema file generation
|
||||
- Filtering by declaration type (e.g., only functions)
|
||||
- Dependency graph in JSON format
|
||||
- Diff mode between two JSON outputs
|
||||
- Machine-readable error format
|
||||
|
||||
## Iteration Notes
|
||||
|
||||
### Iteration 1 Considerations:
|
||||
- Should we include dependency information in JSON?
|
||||
- **Decision**: No, keep it simple. Focus on declarations only.
|
||||
- Should we include source location (line numbers)?
|
||||
- **Decision**: Future enhancement. Not in initial scope.
|
||||
- Should JSON output be pretty-printed or compact?
|
||||
- **Decision**: Pretty-printed with 2-space indentation for readability.
|
||||
- Error handling: What if JSON write fails partway through?
|
||||
- **Decision**: Write to temporary file first, rename on success. For stdout, fail fast.
|
||||
|
||||
### Iteration 2 Review:
|
||||
Looking at the plan again:
|
||||
|
||||
**Strengths:**
|
||||
- Clear JSON schema design
|
||||
- Comprehensive edge case handling
|
||||
- Good testing plan
|
||||
- Realistic scope
|
||||
|
||||
**Potential Issues:**
|
||||
- Need to handle timestamp generation (use std.time)
|
||||
- Should verify that nested JSON writing doesn't cause stack overflow
|
||||
- Consider buffering for large outputs
|
||||
- Add validation that string escaping handles all C comment styles
|
||||
|
||||
**Refinements:**
|
||||
- Add buffered writer wrapper for performance
|
||||
- Use `std.json.writeStream` if available in Zig 0.14
|
||||
- Add --json-pretty flag to control formatting
|
||||
- Document that all strings are UTF-8 encoded
|
||||
|
||||
### Final Confidence Assessment:
|
||||
|
||||
✅ **High Confidence Areas:**
|
||||
- JSON schema design is complete and covers all declaration types
|
||||
- Integration points are well-defined
|
||||
- Testing approach is thorough
|
||||
|
||||
⚠️ **Medium Confidence Areas:**
|
||||
- String escaping complexity (especially multi-line doc comments)
|
||||
- Performance with very large headers
|
||||
- Error recovery during JSON generation
|
||||
|
||||
✅ **Ready to Implement:**
|
||||
The plan is comprehensive and actionable. We should proceed with implementation.
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. Create `src/json_output.zig` with basic structure
|
||||
2. Implement individual serialization functions
|
||||
3. Add command line flag parsing
|
||||
4. Integrate into main parser flow
|
||||
5. Test with SDL_gpu.h (known good header)
|
||||
6. Test with SDL_video.h (larger header)
|
||||
7. Test edge cases and error conditions
|
||||
8. Update documentation
|
||||
Loading…
Reference in New Issue