From 5e60be2addb1bee1f2cd23d733b77458e882ce3e Mon Sep 17 00:00:00 2001 From: peterino2 Date: Sat, 11 Jul 2026 16:20:04 -0700 Subject: [PATCH] saving --- SKILL.md | 367 +++++++ references/builtins.md | 870 +++++++++++++++ references/c-interop.md | 875 ++++++++++++++++ references/code-review.md | 1452 +++++++++++++++++++++++++ references/comptime.md | 462 ++++++++ references/language.md | 745 +++++++++++++ references/patterns.md | 1455 ++++++++++++++++++++++++++ references/std-allocators.md | 592 +++++++++++ references/std-array-hash-map.md | 233 +++++ references/std-arraylist.md | 165 +++ references/std-ascii.md | 144 +++ references/std-atomic.md | 416 ++++++++ references/std-base64.md | 144 +++ references/std-bit-set.md | 211 ++++ references/std-buf-map.md | 176 ++++ references/std-build.md | 1135 ++++++++++++++++++++ references/std-c.md | 1005 ++++++++++++++++++ references/std-compress.md | 457 ++++++++ references/std-crypto.md | 602 +++++++++++ references/std-debug.md | 470 +++++++++ references/std-enums.md | 300 ++++++ references/std-fmt.md | 556 ++++++++++ references/std-fs.md | 238 +++++ references/std-hash.md | 512 +++++++++ references/std-hashmap.md | 181 ++++ references/std-http.md | 709 +++++++++++++ references/std-io.md | 289 +++++ references/std-json.md | 428 ++++++++ references/std-linked-list.md | 155 +++ references/std-log.md | 259 +++++ references/std-math.md | 411 ++++++++ references/std-mem.md | 315 ++++++ references/std-meta.md | 448 ++++++++ references/std-multi-array-list.md | 137 +++ references/std-net.md | 642 ++++++++++++ references/std-os.md | 628 +++++++++++ references/std-priority-dequeue.md | 183 ++++ references/std-priority-queue.md | 192 ++++ references/std-process.md | 241 +++++ references/std-random.md | 414 ++++++++ references/std-segmented-list.md | 142 +++ references/std-simd.md | 586 +++++++++++ references/std-sort.md | 234 +++++ references/std-static-string-map.md | 223 ++++ references/std-tar.md | 436 ++++++++ references/std-testing.md | 381 +++++++ references/std-thread.md | 199 ++++ references/std-time.md | 128 +++ references/std-treap.md | 171 +++ references/std-tz.md | 237 +++++ references/std-unicode.md | 267 +++++ references/std-uri.md | 484 +++++++++ references/std-zig.md | 724 +++++++++++++ references/std-zip.md | 350 +++++++ references/std-zon.md | 489 +++++++++ references/style-guide.md | 215 ++++ references/zig-0.16-release-notes.md | 642 ++++++++++++ 57 files changed, 25122 insertions(+) create mode 100644 SKILL.md create mode 100644 references/builtins.md create mode 100644 references/c-interop.md create mode 100644 references/code-review.md create mode 100644 references/comptime.md create mode 100644 references/language.md create mode 100644 references/patterns.md create mode 100644 references/std-allocators.md create mode 100644 references/std-array-hash-map.md create mode 100644 references/std-arraylist.md create mode 100644 references/std-ascii.md create mode 100644 references/std-atomic.md create mode 100644 references/std-base64.md create mode 100644 references/std-bit-set.md create mode 100644 references/std-buf-map.md create mode 100644 references/std-build.md create mode 100644 references/std-c.md create mode 100644 references/std-compress.md create mode 100644 references/std-crypto.md create mode 100644 references/std-debug.md create mode 100644 references/std-enums.md create mode 100644 references/std-fmt.md create mode 100644 references/std-fs.md create mode 100644 references/std-hash.md create mode 100644 references/std-hashmap.md create mode 100644 references/std-http.md create mode 100644 references/std-io.md create mode 100644 references/std-json.md create mode 100644 references/std-linked-list.md create mode 100644 references/std-log.md create mode 100644 references/std-math.md create mode 100644 references/std-mem.md create mode 100644 references/std-meta.md create mode 100644 references/std-multi-array-list.md create mode 100644 references/std-net.md create mode 100644 references/std-os.md create mode 100644 references/std-priority-dequeue.md create mode 100644 references/std-priority-queue.md create mode 100644 references/std-process.md create mode 100644 references/std-random.md create mode 100644 references/std-segmented-list.md create mode 100644 references/std-simd.md create mode 100644 references/std-sort.md create mode 100644 references/std-static-string-map.md create mode 100644 references/std-tar.md create mode 100644 references/std-testing.md create mode 100644 references/std-thread.md create mode 100644 references/std-time.md create mode 100644 references/std-treap.md create mode 100644 references/std-tz.md create mode 100644 references/std-unicode.md create mode 100644 references/std-uri.md create mode 100644 references/std-zig.md create mode 100644 references/std-zip.md create mode 100644 references/std-zon.md create mode 100644 references/style-guide.md create mode 100644 references/zig-0.16-release-notes.md diff --git a/SKILL.md b/SKILL.md new file mode 100644 index 0000000..6312fe4 --- /dev/null +++ b/SKILL.md @@ -0,0 +1,367 @@ +--- +name: zig +description: Up-to-date Zig programming language patterns for version 0.16.0. Use when writing, reviewing, or debugging Zig code, working with build.zig/build.zig.zon, translating C headers, or using comptime metaprogramming. Critical for avoiding outdated patterns from training data, especially std.Io ownership, Juicy Main, std.Io.Dir/File/net/process APIs, Io synchronization primitives, removed @Type, @cImport migration, unmanaged containers, and removed std.Thread.Pool. +--- + +# Zig Language Reference (v0.16.0) + +Zig evolves rapidly. Training data contains outdated patterns that cause compilation errors or subtle runtime bugs. This skill documents modern Zig 0.16.0 patterns and points to deeper references when needed. + +Always load **[Zig 0.16 Release Notes Migration Reference](references/zig-0.16-release-notes.md)** for upgrade work, stdlib API questions, C translation changes, I/O changes, or code review involving old Zig examples. It is grounded in the official Zig 0.16.0 release notes: https://ziglang.org/download/0.16.0/release-notes.html + +## Critical: I/O Is an Interface + +Starting in Zig 0.16.0, input/output APIs require a `std.Io` instance. Treat anything that can block, touch the OS, use entropy/time/network/process/file APIs, or introduce nondeterminism as owned by `std.Io`. + +Preferred app entry: + +```zig +const std = @import("std"); + +pub fn main(init: std.process.Init) !void { + const gpa = init.gpa; + const io = init.io; + + const args = try init.minimal.args.toSlice(init.arena.allocator()); + _ = .{ gpa, io, args }; +} +``` + +Preferred test I/O: + +```zig +const io = std.testing.io; +``` + +Temporary adapter only when a boundary cannot yet receive `io`: + +```zig +var threaded: std.Io.Threaded = .init_single_threaded; +const io = threaded.io(); +``` + +General rule: + +- Accept `io: std.Io` where the API performs or later owns I/O. +- Store `io` on long-lived systems, allocators, registries, timers, queues, and clients that need it after initialization. +- Do not create local fallback `Io.Threaded` instances deep in library code unless explicitly approved as a transition shim. +- When converting existing code, prefer threading `io` through signatures over global state. + +## Critical: File, Net, Process, Time, Random + +### Files + +Use `std.Io.Dir` and `std.Io.File`; old `std.fs.File`/`std.fs.Dir` patterns are obsolete. + +```zig +const cwd = std.Io.Dir.cwd(); +const file = try cwd.openFile(io, "data.txt", .{}); +defer file.close(io); + +var buf: [4096]u8 = undefined; +var reader = file.reader(io, &buf); +const bytes = try reader.interface.allocRemaining(gpa, .limited(1024 * 1024)); +defer gpa.free(bytes); +``` + +```zig +const file = try std.Io.Dir.cwd().createFile(io, "out.txt", .{}); +defer file.close(io); + +var buf: [4096]u8 = undefined; +var writer = file.writer(io, &buf); +try writer.interface.print("hello {s}\n", .{"world"}); +try writer.interface.flush(); +``` + +Convenience: + +```zig +const bytes = try std.Io.Dir.cwd().readFileAlloc(io, "data.txt", gpa, .limited(max_size)); +defer gpa.free(bytes); + +try std.Io.Dir.cwd().writeFile(io, .{ + .sub_path = "out.txt", + .data = "hello\n", +}); +``` + +### Reader and Writer + +`std.io` is replaced by `std.Io`. `GenericReader`, `AnyReader`, `GenericWriter`, `AnyWriter`, and `FixedBufferStream` are gone. + +```zig +var r: std.Io.Reader = .fixed("a\nb\n"); +while (try r.takeDelimiter('\n')) |line| { + _ = line; +} + +var out_buf: [256]u8 = undefined; +var w: std.Io.Writer = .fixed(&out_buf); +try w.print("value={d}", .{42}); +const written = w.buffered(); +``` + +Custom format methods use the writer-only signature: + +```zig +pub fn format(self: @This(), writer: *std.Io.Writer) std.Io.Writer.Error!void { + try writer.print("{s}:{d}", .{ self.name, self.value }); +} +``` + +Use `{f}` to invoke format methods. + +### Process + +Use Juicy Main for args/env and `std.process.run` / `std.process.spawn` for child processes. + +```zig +const result = try std.process.run(gpa, io, .{ + .argv = &.{ "git", "status", "--short" }, + .stdout_limit = .limited(64 * 1024), + .stderr_limit = .limited(64 * 1024), +}); +defer gpa.free(result.stdout); +defer gpa.free(result.stderr); +``` + +```zig +var child = try std.process.spawn(io, .{ + .argv = &.{ "tool", "--flag" }, + .stdout = .pipe, + .stderr = .pipe, +}); +defer child.kill(io); +const term = try child.wait(io); +_ = term; +``` + +Current directory: + +```zig +const cwd = try std.process.currentPathAlloc(io, gpa); +defer gpa.free(cwd); +``` + +### Networking and HTTP + +Use `std.Io.net` and pass `io` to high-level clients. + +```zig +var client: std.http.Client = .{ + .allocator = gpa, + .io = io, +}; +defer client.deinit(); +``` + +### Time + +0.16 routes time through `std.Io` types: + +- `std.time.Instant` -> `std.Io.Timestamp` +- `std.time.Timer` -> `std.Io.Timestamp` +- `std.time.timestamp` -> `std.Io.Timestamp.now` + +Applications may centralize timestamp reads behind a shared helper so clock selection and timestamp semantics remain consistent. + +### Random and Entropy + +Do not use `std.crypto.random` directly in new 0.16 code. + +```zig +var seed: [32]u8 = undefined; +io.random(&seed); + +const source: std.Random.IoSource = .{ .io = io }; +const rng = source.interface(); +``` + +## Critical: I/O Synchronization + +Blocking synchronization moved from `std.Thread` to `std.Io` so it cooperates with the runtime backend and cancelation. + +Migration map: + +- `std.Thread.Mutex` -> `std.Io.Mutex` +- `std.Thread.Condition` -> `std.Io.Condition` +- `std.Thread.Semaphore` -> `std.Io.Semaphore` +- `std.Thread.RwLock` -> `std.Io.RwLock` +- `std.Thread.ResetEvent` -> `std.Io.Event` +- `std.Thread.WaitGroup` -> `std.Io.Group` +- `std.Thread.Futex` -> `std.Io.Futex` + +Cancelable lock: + +```zig +try mutex.lock(io); +defer mutex.unlock(io); +``` + +Uncancelable lock for short critical sections or cleanup: + +```zig +mutex.lockUncancelable(io); +defer mutex.unlock(io); +``` + +Do not replace these with custom spin loops. Use std primitives unless a measured, documented special case requires otherwise. + +## Critical: Language and Builtins + +### Removed or Deprecated + +- `@Type` removed. Use `@Int`, `@Struct`, `@Union`, `@Enum`, `@Pointer`, `@Fn`, `@Tuple`, `@Opaque`, or `@EnumLiteral`. +- `@cImport` is deprecated long-term. Prefer `b.addTranslateC(...)` and import `translate_c.createModule()`. +- `async`/`await` keywords remain removed; use `std.Io` task APIs, an application scheduler, or explicit threads. +- `usingnamespace` is removed; explicitly re-export names. +- `@fence` is removed; use stronger atomic orderings or RMW operations. +- `@intFromFloat` is deprecated; use `@trunc` when truncating float to integer. + +### Packed and Extern Layout + +- Packed structs/unions cannot contain pointers. +- Packed unions need unambiguous bit sizes and may use explicit backing integers. +- Extern enum/packed types need explicit backing types. +- ABI-sensitive bindings must keep comptime size/alignment/offset assertions. + +### Vectors and Arrays + +- Runtime vector indexes are forbidden. +- Vectors and arrays no longer support in-memory coercion; convert explicitly. + +### Safer Pointer Returns + +The compiler rejects trivially returning the address of an expired local. When returning pointers, prove ownership and lifetime explicitly. + +## Critical: Containers and Allocators + +Use `.empty` for empty unmanaged containers and pass allocators to methods. + +```zig +var list: std.ArrayList(u32) = .empty; +defer list.deinit(gpa); +try list.append(gpa, 42); +``` + +0.16 container changes: + +- `std.ArrayHashMap`, `std.AutoArrayHashMap`, and `std.StringArrayHashMap` removed. +- Use `std.array_hash_map.Custom`, `std.array_hash_map.Auto`, and `std.array_hash_map.String`. +- `std.PriorityQueue` and `std.PriorityDequeue` no longer store allocators. +- Priority queues use `push`/`pop` terminology and `.empty` initialization. +- `std.SegmentedList` removed. +- `std.heap.ThreadSafe` removed. +- `std.heap.ArenaAllocator` is thread-safe and lock-free. + +## Critical: Build System + +The 0.15 root-module build style remains the baseline: + +```zig +const exe = b.addExecutable(.{ + .name = "app", + .root_module = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, + }), +}); +``` + +Use module-level APIs: + +```zig +exe.root_module.addImport("helper", helper_mod); +exe.root_module.linkSystemLibrary("SDL3", .{}); +exe.root_module.addCSourceFiles(.{ .files = &.{"src/foo.c"} }); +``` + +0.16 additions and removals: + +- `--prominent-compile-errors` removed; use `--error-style minimal`. +- New `--multiline-errors` styles are available. +- `zig build test --test-timeout ` can bound test runtime. +- `Build.makeTempPath` and RemoveDir step removed. +- Use `b.addTempFiles`, `b.addMutateFiles`, `b.tmpPath`, and WriteFile temp/mutate modes. +- Use build-system `addTranslateC` for new translated C bindings. + +## Critical: C Translation and ABI Safety + +Translate-c is now Aro/translate-c based rather than libclang based. It is intended to be non-breaking, but subtle generated-code changes can happen. + +For ABI-sensitive C bindings: + +- Keep translated output under review. +- Preserve size/alignment/offset comptime asserts. +- Compare enum values, bit flags, packed layouts, extern structs, calling conventions, and macro translations. +- Do not dismiss translate-c diffs as cosmetic until ABI checks pass. + +## Application Integration Guidance + +When integrating these patterns into an application: + +- Prefer an explicit `std.Io` parameter; an application-owned accessor is also reasonable where the architecture already provides one. +- New file operations require an `io` parameter at the API boundary. +- Long-lived allocators/registries/queues that lock or perform I/O should accept `io` at init and expose it if needed. +- Prefer a shared application helper for common wall-clock timestamps. +- Keep dynamic-library behavior behind a platform abstraction, especially on Windows where `std.DynLib` support changed. + +## Import Loop Guidance + +Do not over-index on dependency loops when importing source files directly. + +- `@import("mything.zig")` is a direct file import and should be treated as safe. +- Only consider import-loop risk when importing a module name such as `@import("module")`. +- A top-level file plus same-named helper directory is normal: + +```zig +// module.zig +const moduleComponent = @import("module/moduleComponent.zig"); + +// module/moduleComponent.zig +const module = @import("../module.zig"); +``` + +## Quick Fixes + +| Symptom | Zig 0.16 Fix | +|---------|--------------| +| missing `std.fs.File` / `std.fs.Dir` | Use `std.Io.File` / `std.Io.Dir` and pass `io` | +| file `close()` takes an argument | Use `file.close(io)` or `dir.close(io)` | +| old stdout writer fails | `std.Io.File.stdout().writer(io, &buf)` | +| old `std.process.Child.run` fails | Use `std.process.run(gpa, io, .{ ... })` | +| old `Child.init` fails | Use `std.process.spawn(io, .{ ... })` | +| old `std.crypto.random` fails | Use `io.random` or `std.Random.IoSource` | +| old `std.Thread.Mutex.lock()` fails in I/O code | Use `std.Io.Mutex` and `lock(io)` / `lockUncancelable(io)` | +| `std.Thread.Pool` missing | Use an application scheduler or carefully designed `std.Io.Group` / `io.async` | +| `@Type` missing | Use specific type-constructor builtins | +| `@cImport` warning | Move translation to `b.addTranslateC(...)` | +| `std.mem.indexOf` not found | Use `std.mem.find` / `findScalar` / `cut` helpers | +| `{D}` duration format fails | Format `std.Io.Duration` with `{f}` | +| `std.fmt.Formatter` missing | Use `std.fmt.Alt` | +| `std.fmt.format` missing | Use `std.Io.Writer.print` | +| `std.fs.path.relative` signature changed | Pass cwd path and optional environment map | +| `stat.atime` type changed | Handle optional access time | + +## References + +Load these selectively: + +- **[Zig 0.16 Release Notes Migration Reference](references/zig-0.16-release-notes.md)** - First stop for version-specific upgrade work, organized by official release-note section. +- **[std.Io](references/std-io.md)** - 0.16 reader/writer, file I/O, `Io` ownership, sync and task concepts. +- **[std.fs / std.Io.Dir](references/std-fs.md)** - File system migration and common 0.16 patterns. +- **[std.process](references/std-process.md)** - Juicy Main, args/env, child process APIs, current directory. +- **[std.Thread / std.Io sync](references/std-thread.md)** - Thread spawning plus migration to `std.Io` synchronization primitives. +- **[std.time / std.Io time](references/std-time.md)** - Timestamp migration and duration handling. +- **[std.Build](references/std-build.md)** - Build graph, modules, package deps, 0.16 build-system changes. +- **[Built-in Functions](references/builtins.md)** - Builtin syntax and 0.16 `@Type` replacement. +- **[Comptime Reference](references/comptime.md)** - Type reflection, type construction, lazy type resolution, compile-time patterns. +- **[C Interop](references/c-interop.md)** - C ABI, exports, translated headers, and binding checks. +- **[Zig Patterns](references/patterns.md)** - Idiomatic patterns. Treat 0.16 release-note reference as authoritative when examples conflict. +- **[Code Review](references/code-review.md)** - Review checklist. Always apply the 0.16 release-note checks first. + +General references remain available for specific modules: `std.mem`, `std.fmt`, `std.json`, `std.zon`, `std.crypto`, `std.http`, `std.net`, `std.compress`, `std.tar`, `std.zip`, `std.testing`, `std.debug`, `std.log`, `std.atomic`, `std.simd`, and other files under `references/`. + +If any reference still contains a historical 0.14/0.15 example, the Zig 0.16 release-notes reference and the top-level rules in this file win. diff --git a/references/builtins.md b/references/builtins.md new file mode 100644 index 0000000..5689bc9 --- /dev/null +++ b/references/builtins.md @@ -0,0 +1,870 @@ +# Zig Built-in Functions Reference + +Built-in functions are compiler intrinsics prefixed with `@`. Parameters marked `comptime` must be compile-time known. + +## Table of Contents +- [Type Conversions](#type-conversions) +- [Integer/Float Operations](#integerfloat-operations) +- [Overflow Arithmetic](#overflow-arithmetic) +- [Bit Manipulation](#bit-manipulation) +- [Memory Operations](#memory-operations) +- [Atomics](#atomics) +- [Type Introspection](#type-introspection) +- [Comptime Utilities](#comptime-utilities) +- [SIMD/Vector](#simdvector) +- [C Interop](#c-interop) +- [Debug/Control Flow](#debugcontrol-flow) + +## Type Conversions + +### @as +```zig +@as(comptime T: type, expr) T +``` +Safe type coercion. Preferred over explicit casts when conversion is unambiguous. +```zig +const x = @as(u32, 5); // comptime_int → u32 +``` + +### @intCast +```zig +@intCast(value: anytype) anytype +``` +Convert between integer types. Runtime safety check if value doesn't fit. +```zig +const big: u64 = 100; +const small: u8 = @intCast(big); // OK if value fits +``` + +### @floatCast +```zig +@floatCast(value: anytype) anytype +``` +Convert between float types. Return type inferred. +```zig +const d: f64 = 3.14; +const f: f32 = @floatCast(d); +``` + +### @intFromFloat +```zig +@intFromFloat(value: anytype) anytype +``` +Float → integer. Truncates fractional part. Return type inferred. +```zig +const i: i32 = @intFromFloat(3.7); // i = 3 +``` + +### @floatFromInt +```zig +@floatFromInt(value: anytype) anytype +``` +Integer → float. Return type inferred. +```zig +const f: f32 = @floatFromInt(42); +``` + +### @intFromPtr +```zig +@intFromPtr(ptr: anytype) usize +``` +Pointer → `usize`. For pointer arithmetic or FFI. +```zig +const addr: usize = @intFromPtr(&x); +``` + +### @ptrFromInt +```zig +@ptrFromInt(addr: usize) anytype +``` +`usize` → pointer. Return type inferred. **Undefined behavior if invalid.** +```zig +const ptr: *u32 = @ptrFromInt(0x1000); +``` + +### @ptrCast +```zig +@ptrCast(ptr: anytype) anytype +``` +Pointer type cast. Return type inferred. +```zig +const bytes: [*]u8 = @ptrCast(some_ptr); +``` + +### @alignCast +```zig +@alignCast(ptr: anytype) anytype +``` +Change pointer alignment. Safety check at runtime. +```zig +const aligned: *align(16) u8 = @alignCast(ptr); +``` + +### @constCast +```zig +@constCast(ptr: anytype) anytype +``` +Remove `const` qualifier from pointer. Return type inferred. +```zig +const mutable_ptr: *u32 = @constCast(const_ptr); +``` + +### @volatileCast +```zig +@volatileCast(ptr: anytype) anytype +``` +Remove `volatile` qualifier from pointer. + +### @bitCast +```zig +@bitCast(value: anytype) anytype +``` +Reinterpret bits as different type. Sizes must match. Return type inferred. +```zig +const bits: u32 = @bitCast(@as(f32, 1.0)); +const f: f32 = @bitCast(@as(u32, 0x3f800000)); +``` + +### @truncate +```zig +@truncate(value: anytype) anytype +``` +Truncate integer to smaller type. Discards high bits. Return type inferred. +```zig +const small: u8 = @truncate(@as(u32, 0x12345678)); // 0x78 +``` + +### @intFromBool +```zig +@intFromBool(value: bool) u1 +``` +`false` → 0, `true` → 1. +```zig +const x: u8 = @intFromBool(true); // 1 +``` + +### @intFromEnum +```zig +@intFromEnum(value: anytype) anytype +``` +Enum → backing integer type. +```zig +const State = enum(u8) { idle = 0, running = 1 }; +const n: u8 = @intFromEnum(State.running); // 1 +``` + +### @enumFromInt +```zig +@enumFromInt(int: anytype) anytype +``` +Integer → enum. Return type inferred. +```zig +const state: State = @enumFromInt(1); // State.running +``` + +### @errorFromInt +```zig +@errorFromInt(int: anytype) anytype +``` +Integer → error. Return type inferred. + +### @intFromError +```zig +@intFromError(err: anytype) std.meta.Int(.unsigned, @bitSizeOf(anyerror)) +``` +Error → integer. + +### @errorCast +```zig +@errorCast(err: anytype) anytype +``` +Cast between error set types. + +### @addrSpaceCast +```zig +@addrSpaceCast(ptr: anytype) anytype +``` +Convert pointer between address spaces (GPU/embedded). + +## Integer/Float Operations + +### @abs +```zig +@abs(value: anytype) anytype +``` +Absolute value. Works on integers, floats, vectors. +```zig +const x = @abs(@as(i32, -5)); // 5 +``` + +### @min / @max +```zig +@min(a: T, b: T) T +@max(a: T, b: T) T +``` +Return minimum/maximum of two values. +```zig +const m = @max(3, 7); // 7 +``` + +### @divExact +```zig +@divExact(numerator: T, denominator: T) T +``` +Exact division. Asserts no remainder. +```zig +const x = @divExact(10, 2); // 5 +``` + +### @divFloor +```zig +@divFloor(numerator: T, denominator: T) T +``` +Floor division (rounds toward negative infinity). +```zig +const x = @divFloor(-7, 3); // -3 +``` + +### @divTrunc +```zig +@divTrunc(numerator: T, denominator: T) T +``` +Truncating division (rounds toward zero). +```zig +const x = @divTrunc(-7, 3); // -2 +``` + +### @mod +```zig +@mod(numerator: T, denominator: T) T +``` +Floor modulus. Result has same sign as denominator. +```zig +const x = @mod(-5, 3); // 1 +``` + +### @rem +```zig +@rem(numerator: T, denominator: T) T +``` +Remainder. Result has same sign as numerator. +```zig +const x = @rem(-5, 3); // -2 +``` + +### Math Functions (floats/vectors) +```zig +@sqrt(x) // Square root +@sin(x) // Sine +@cos(x) // Cosine +@tan(x) // Tangent +@exp(x) // e^x +@exp2(x) // 2^x +@log(x) // Natural log +@log2(x) // Log base 2 +@log10(x) // Log base 10 +@floor(x) // Round down +@ceil(x) // Round up +@round(x) // Round to nearest +@trunc(x) // Truncate toward zero +@mulAdd(T, a, b, c) // Fused (a*b)+c +``` + +## Overflow Arithmetic + +Returns tuple: `{ result, overflow_bit }` where overflow_bit is `u1`. + +### @addWithOverflow +```zig +@addWithOverflow(a: T, b: T) struct { T, u1 } +``` +```zig +const result, const overflow = @addWithOverflow(@as(u8, 250), 10); +if (overflow != 0) { /* handle overflow */ } +``` + +### @subWithOverflow +```zig +@subWithOverflow(a: T, b: T) struct { T, u1 } +``` + +### @mulWithOverflow +```zig +@mulWithOverflow(a: T, b: T) struct { T, u1 } +``` + +### @shlWithOverflow +```zig +@shlWithOverflow(a: T, b: Log2Int) struct { T, u1 } +``` + +## Bit Manipulation + +### @clz +```zig +@clz(value: anytype) anytype +``` +Count leading zeros. +```zig +const z = @clz(@as(u8, 0b00001111)); // 4 +``` + +### @ctz +```zig +@ctz(value: anytype) anytype +``` +Count trailing zeros. +```zig +const z = @ctz(@as(u8, 0b11110000)); // 4 +``` + +### @popCount +```zig +@popCount(value: anytype) anytype +``` +Count set bits (population count). +```zig +const c = @popCount(@as(u8, 0b10101010)); // 4 +``` + +### @byteSwap +```zig +@byteSwap(value: anytype) @TypeOf(value) +``` +Reverse byte order (endianness conversion). +```zig +const swapped = @byteSwap(@as(u32, 0x12345678)); // 0x78563412 +``` + +### @bitReverse +```zig +@bitReverse(value: anytype) @TypeOf(value) +``` +Reverse all bits. +```zig +const rev = @bitReverse(@as(u8, 0b11000001)); // 0b10000011 +``` + +### @shlExact / @shrExact +```zig +@shlExact(value: T, shift: Log2Int) T +@shrExact(value: T, shift: Log2Int) T +``` +Shift with assertion that no bits are lost. + +## Memory Operations + +### @memcpy +```zig +@memcpy(dest: []T, src: []const T) void +``` +Copy memory. Slices must not overlap. +```zig +@memcpy(dest[0..n], src[0..n]); +``` + +### @memset +```zig +@memset(dest: []T, value: T) void +``` +Fill memory with value. +```zig +@memset(buffer[0..n], 0); +``` + +### @memmove +```zig +@memmove(dest: []T, src: []const T) void +``` +Copy memory. Slices may overlap. + +### @sizeOf +```zig +@sizeOf(comptime T: type) comptime_int +``` +Size of type in bytes (includes padding). +```zig +const size = @sizeOf(u32); // 4 +``` + +### @bitSizeOf +```zig +@bitSizeOf(comptime T: type) comptime_int +``` +Size of type in bits. +```zig +const bits = @bitSizeOf(u24); // 24 +``` + +### @alignOf +```zig +@alignOf(comptime T: type) comptime_int +``` +Alignment requirement of type. +```zig +const align = @alignOf(u64); // typically 8 +``` + +### @offsetOf +```zig +@offsetOf(comptime T: type, comptime field: []const u8) comptime_int +``` +Byte offset of struct field. +```zig +const Point = struct { x: i32, y: i32 }; +const off = @offsetOf(Point, "y"); // 4 +``` + +### @bitOffsetOf +```zig +@bitOffsetOf(comptime T: type, comptime field: []const u8) comptime_int +``` +Bit offset of field (useful for packed structs). + +## Atomics + +### @atomicLoad +```zig +@atomicLoad(comptime T: type, ptr: *const T, comptime ordering: AtomicOrder) T +``` +Atomic read. +```zig +const val = @atomicLoad(u32, &counter, .acquire); +``` + +### @atomicStore +```zig +@atomicStore(comptime T: type, ptr: *T, value: T, comptime ordering: AtomicOrder) void +``` +Atomic write. +```zig +@atomicStore(u32, &counter, 42, .release); +``` + +### @atomicRmw +```zig +@atomicRmw(comptime T: type, ptr: *T, comptime op: AtomicRmwOp, operand: T, comptime ordering: AtomicOrder) T +``` +Atomic read-modify-write. Returns previous value. +```zig +const old = @atomicRmw(u32, &counter, .Add, 1, .seq_cst); +``` +Operations: `.Add`, `.Sub`, `.And`, `.Or`, `.Xor`, `.Nand`, `.Min`, `.Max`, `.Xchg` + +### @cmpxchgStrong / @cmpxchgWeak +```zig +@cmpxchgStrong(comptime T: type, ptr: *T, expected: T, new: T, success_order: AtomicOrder, fail_order: AtomicOrder) ?T +@cmpxchgWeak(comptime T: type, ptr: *T, expected: T, new: T, success_order: AtomicOrder, fail_order: AtomicOrder) ?T +``` +Compare-and-swap. Returns `null` on success, old value on failure. +```zig +while (@cmpxchgWeak(u32, &counter, expected, new, .seq_cst, .seq_cst)) |actual| { + expected = actual; +} +``` + +## Type Introspection + +### @TypeOf +```zig +@TypeOf(expr) type +``` +Get type of expression at comptime. +```zig +const T = @TypeOf(some_value); +``` + +### @typeInfo +```zig +@typeInfo(comptime T: type) std.builtin.Type +``` +Get detailed type information. +```zig +const info = @typeInfo(MyStruct); +if (info == .@"struct") { + for (info.@"struct".fields) |field| { + // field.name, field.type, etc. + } +} +``` + +### @Type - removed in Zig 0.16 + +`@Type` was removed in Zig 0.16. Use the specific type-construction builtin that matches the type you are creating: + +Common replacements: + +- integer type: `@Int(.signed, 32)` +- tuple type: `@Tuple(&.{ u32, []const u8 })` +- pointer type: `@Pointer(...)` +- function type: `@Fn(...)` +- struct type: `@Struct(...)` +- union type: `@Union(...)` +- enum type: `@Enum(...)` +- opaque type: write `opaque {}` directly + +Keep `@typeInfo` for reflection; use the new builtins only when constructing types. + +### @typeName +```zig +@typeName(comptime T: type) [:0]const u8 +``` +Get string name of type. +```zig +const name = @typeName(u32); // "u32" +``` + +### @hasDecl +```zig +@hasDecl(comptime T: type, comptime name: []const u8) bool +``` +Check if type has declaration (const, fn, etc.). +```zig +if (@hasDecl(T, "init")) { T.init(); } +``` + +### @hasField +```zig +@hasField(comptime T: type, comptime name: []const u8) bool +``` +Check if struct/union has field. + +### @field +```zig +@field(value: anytype, comptime name: []const u8) anytype +``` +Access field by comptime string name. +```zig +const x = @field(point, "x"); +``` + +### @FieldType +```zig +@FieldType(comptime T: type, comptime name: []const u8) type +``` +Get type of a struct field. + +### @fieldParentPtr +```zig +@fieldParentPtr(field_ptr: anytype, comptime field_name: []const u8) anytype +``` +Get pointer to containing struct from field pointer (for intrusive data structures). +```zig +const Node = struct { data: u32, hook: Hook }; +fn getNode(hook: *Hook) *Node { + return @fieldParentPtr(hook, "hook"); +} +``` + +### @tagName +```zig +@tagName(value: anytype) [:0]const u8 +``` +Get string name of enum/union tag. +```zig +const Color = enum { red, green, blue }; +const name = @tagName(Color.red); // "red" +``` + +### @errorName +```zig +@errorName(err: anyerror) [:0]const u8 +``` +Get string name of error. +```zig +const name = @errorName(error.OutOfMemory); // "OutOfMemory" +``` + +## Comptime Utilities + +### @import +```zig +@import(comptime path: []const u8) type +``` +Import module. Special: `"std"`, `"builtin"`. +```zig +const std = @import("std"); +const builtin = @import("builtin"); +const other = @import("other.zig"); +``` + +### @embedFile +```zig +@embedFile(comptime path: []const u8) *const [N:0]u8 +``` +Embed file contents as compile-time string. +```zig +const data = @embedFile("data.bin"); +``` + +### @compileError +```zig +@compileError(comptime msg: []const u8) noreturn +``` +Emit compile error with message. +```zig +if (condition) @compileError("Invalid configuration"); +``` + +### @compileLog +```zig +@compileLog(args: ...) void +``` +Print values at compile time for debugging. +```zig +@compileLog("x =", x, "T =", T); +``` + +### @This +```zig +@This() type +``` +Get enclosing struct/union/enum type. +```zig +const Self = @This(); +fn method(self: *Self) void { ... } +``` + +### @src +```zig +@src() std.builtin.SourceLocation +``` +Get current source location (file, line, column, fn name). + +### @inComptime +```zig +@inComptime() bool +``` +Check if currently executing at comptime. +```zig +if (@inComptime()) { + // comptime path +} else { + // runtime path +} +``` + +### @setEvalBranchQuota +```zig +@setEvalBranchQuota(quota: u32) void +``` +Increase comptime evaluation limit (default 1000). +```zig +@setEvalBranchQuota(100_000); +``` + +## SIMD/Vector + +### @Vector +```zig +@Vector(len: comptime_int, T: type) type +``` +Create SIMD vector type. +```zig +const Vec4f = @Vector(4, f32); +const v: Vec4f = .{ 1.0, 2.0, 3.0, 4.0 }; +``` + +### @splat +```zig +@splat(value: anytype) anytype +``` +Create vector with all elements equal to value. Return type inferred. +```zig +const ones: @Vector(4, f32) = @splat(1.0); +``` + +### @reduce +```zig +@reduce(comptime op: std.builtin.ReduceOp, value: anytype) ElementType +``` +Reduce vector to scalar. +```zig +const sum = @reduce(.Add, vec); // sum all elements +const max = @reduce(.Max, vec); // find maximum +``` +Operations: `.Add`, `.Mul`, `.And`, `.Or`, `.Xor`, `.Min`, `.Max` + +### @shuffle +```zig +@shuffle(T: type, a: @Vector(N, T), b: @Vector(N, T), mask: @Vector(M, i32)) @Vector(M, T) +``` +Rearrange vector elements using mask. +```zig +const a: @Vector(4, i32) = .{ 1, 2, 3, 4 }; +const b: @Vector(4, i32) = .{ 5, 6, 7, 8 }; +const result = @shuffle(i32, a, b, .{ 0, 4, 1, 5 }); // {1, 5, 2, 6} +// Positive indices select from a, indices >= len select from b +``` + +### @select +```zig +@select(T: type, pred: @Vector(N, bool), a: @Vector(N, T), b: @Vector(N, T)) @Vector(N, T) +``` +Element-wise select: `pred[i] ? a[i] : b[i]`. + +## C Interop + +### @cImport - deprecated migration path +```zig +@cImport(expr) type +``` +Import C header files. In Zig 0.16 this is deprecated as the long-term API; prefer translating headers in `build.zig` with `b.addTranslateC(...)` and importing `translate_c.createModule()`. +```zig +const c = @cImport({ + @cDefine("_GNU_SOURCE", {}); + @cInclude("stdio.h"); +}); +``` + +### @cInclude +```zig +@cInclude(comptime path: []const u8) void +``` +Include C header (inside `@cImport`). + +### @cDefine +```zig +@cDefine(comptime name: []const u8, value) void +``` +Define C macro (inside `@cImport`). + +### @cUndef +```zig +@cUndef(comptime name: []const u8) void +``` +Undefine C macro. + +### @extern +```zig +@extern(comptime T: type, options: ExternOptions) T +``` +Declare external symbol. + +### @export +```zig +@export(target: anytype, options: ExportOptions) void +``` +Export symbol. **Takes pointer in 0.14.0+**. +```zig +@export(&my_fn, .{ .name = "exported_name" }); +``` + +### C Varargs +```zig +@cVaStart() std.builtin.VaList // Start vararg processing +@cVaArg(*VaList, T) T // Get next vararg +@cVaCopy(*VaList) VaList // Copy vararg state +@cVaEnd(*VaList) void // End vararg processing +``` + +## Debug/Control Flow + +### @branchHint +```zig +@branchHint(hint: std.builtin.BranchHint) void +``` +Hint branch likelihood. Must be first statement in branch. +```zig +if (unlikely_condition) { + @branchHint(.cold); + // rarely executed +} +``` +Hints: `.none`, `.likely`, `.unlikely`, `.cold` + +### @breakpoint +```zig +@breakpoint() void +``` +Insert debugger breakpoint. + +### @trap +```zig +@trap() noreturn +``` +Crash immediately (illegal instruction). + +### @panic +```zig +@panic(msg: []const u8) noreturn +``` +Trigger panic with message. + +### @setRuntimeSafety +```zig +@setRuntimeSafety(enabled: bool) void +``` +Enable/disable safety checks in current scope. +```zig +@setRuntimeSafety(false); +// Unsafe operations here +``` + +### @setFloatMode +```zig +@setFloatMode(mode: std.builtin.FloatMode) void +``` +Set floating-point optimization mode. +```zig +@setFloatMode(.optimized); // Allow reordering, etc. +``` + +### @returnAddress +```zig +@returnAddress() usize +``` +Get return address of current function. + +### @frameAddress +```zig +@frameAddress() usize +``` +Get frame pointer of current function. + +### @errorReturnTrace +```zig +@errorReturnTrace() ?*std.builtin.StackTrace +``` +Get error return trace (if available). + +### @call +```zig +@call(modifier: std.builtin.CallModifier, fn: anytype, args: anytype) anytype +``` +Call function with modifier. +```zig +const result = @call(.always_inline, my_fn, .{ arg1, arg2 }); +``` +Modifiers: `.auto`, `.never_inline`, `.always_inline`, `.always_tail`, `.never_tail`, `.compile_time` + +### @prefetch +```zig +@prefetch(ptr: anytype, options: PrefetchOptions) void +``` +Prefetch memory into cache. +```zig +@prefetch(ptr, .{ .rw = .read, .locality = 3 }); +``` + +## WebAssembly + +### @wasmMemorySize +```zig +@wasmMemorySize(index: u32) u32 +``` +Get WebAssembly memory size in pages. + +### @wasmMemoryGrow +```zig +@wasmMemoryGrow(index: u32, delta: u32) u32 +``` +Grow WebAssembly memory by delta pages. + +## GPU/Workgroup + +```zig +@workGroupId(dim: u32) u32 // Get workgroup ID +@workGroupSize(dim: u32) u32 // Get workgroup size +@workItemId(dim: u32) u32 // Get work item ID within group +``` diff --git a/references/c-interop.md b/references/c-interop.md new file mode 100644 index 0000000..875c93d --- /dev/null +++ b/references/c-interop.md @@ -0,0 +1,875 @@ +# C Interoperability Reference (Zig 0.16.0) + +Zig can export C-compatible APIs for use from any language that supports the C ABI: Swift, Objective-C, Python, Ruby, Rust, etc. This enables architectures like Ghostty (93% Zig business logic + 4% platform-native GUI). + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 translate-c is Aro/translate-c based rather than libclang based. `@cImport` is deprecated as the long-term API; prefer `b.addTranslateC(...)` in build scripts and import `translate_c.createModule()`. + +For ABI-sensitive bindings, keep comptime assertions for `@sizeOf`, `@alignOf`, field offsets, enum/flag values, and calling conventions. Treat generated binding differences as high-risk until checked. + +## Table of Contents +- [Quick Start](#quick-start) +- [Exporting Functions](#exporting-functions) +- [C-Compatible Types](#c-compatible-types) +- [Building C Libraries](#building-c-libraries) +- [Creating Header Files](#creating-header-files) +- [macOS Integration](#macos-integration) +- [Swift Integration](#swift-integration) +- [Common Patterns](#common-patterns) + +## Quick Start + +Minimal C-compatible library: + +**src/lib.zig:** +```zig +const std = @import("std"); + +// Global state (opaque to C consumers) +var context: ?*Context = null; + +const Context = struct { + allocator: std.mem.Allocator, + value: i32, +}; + +/// Initialize the library. Returns 0 on success, -1 on failure. +export fn mylib_init() c_int { + const gpa = std.heap.c_allocator; + context = gpa.create(Context) catch return -1; + context.?.* = .{ .allocator = gpa, .value = 0 }; + return 0; +} + +/// Clean up resources. +export fn mylib_deinit() void { + if (context) |ctx| { + ctx.allocator.destroy(ctx); + context = null; + } +} + +/// Get the current value. +export fn mylib_get_value() c_int { + return if (context) |ctx| ctx.value else 0; +} + +/// Set the value. +export fn mylib_set_value(v: c_int) void { + if (context) |ctx| ctx.value = v; +} +``` + +**build.zig:** +```zig +const std = @import("std"); + +pub fn build(b: *std.Build) void { + const target = b.standardTargetOptions(.{}); + const optimize = b.standardOptimizeOption(.{}); + + const lib = b.addLibrary(.{ + .name = "mylib", + .linkage = .static, + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = target, + .optimize = optimize, + }), + }); + + // Link libc if using std.heap.c_allocator + lib.linkLibC(); + + b.installArtifact(lib); + + // Install header alongside library + b.installFile("include/mylib.h", "include/mylib.h"); +} +``` + +**include/mylib.h:** +```c +#ifndef MYLIB_H +#define MYLIB_H + +#ifdef __cplusplus +extern "C" { +#endif + +int mylib_init(void); +void mylib_deinit(void); +int mylib_get_value(void); +void mylib_set_value(int v); + +#ifdef __cplusplus +} +#endif + +#endif /* MYLIB_H */ +``` + +## Exporting Functions + +### `export` Keyword + +The `export` keyword creates a function with C ABI linkage: + +```zig +// Creates symbol "add" with C calling convention +export fn add(a: c_int, b: c_int) c_int { + return a + b; +} +``` + +Equivalent to: +```zig +fn add(a: c_int, b: c_int) callconv(.c) c_int { + return a + b; +} +comptime { + @export(&add, .{ .name = "add" }); +} +``` + +### Custom Symbol Names + +Use `@export` for custom symbol names: + +```zig +fn zigAdd(a: c_int, b: c_int) callconv(.c) c_int { + return a + b; +} + +comptime { + @export(&zigAdd, .{ .name = "mylib_add" }); // Symbol: mylib_add +} +``` + +### Calling Convention + +For internal C-callable functions (not exported): + +```zig +// C calling convention, but not exported as symbol +fn internalCallback(data: ?*anyopaque) callconv(.c) void { + // Called by C code via function pointer +} +``` + +### Restrictions on Exported Functions + +Exported function signatures are limited to C-compatible constructs: + +**Allowed:** +- C integer types: `c_int`, `c_uint`, `c_long`, `c_ulong`, `c_char`, etc. +- Fixed-width integers matching C: `i8`, `i16`, `i32`, `i64`, `u8`, `u16`, `u32`, `u64` +- Floating point: `f32` (`float`), `f64` (`double`) +- Pointers: `*T`, `[*]T`, `[*c]T`, `?*T` +- `bool` (maps to C `_Bool`) +- `void` +- `usize`, `isize` (map to `size_t`, `ptrdiff_t`) + +**Not allowed in signatures:** +- Comptime parameters +- Generic types (`anytype`) +- Zig error unions (`!T`) +- Zig optionals (except optional pointers `?*T`) +- Slices (`[]T`) - use pointer + length instead +- Non-extern structs/unions/enums +- Arbitrary bit-width integers (`u3`, `i47`) + +**Inside the function body**, all Zig features work: + +```zig +export fn process(data: [*]const u8, len: usize) c_int { + // Inside: full Zig features + const slice = data[0..len]; + + for (slice) |byte| { + if (byte == 0) return -1; + } + + return @intCast(slice.len); +} +``` + +## C-Compatible Types + +### Integer Type Mapping + +| Zig Type | C Type | Notes | +|----------|--------|-------| +| `c_char` | `char` | Signed or unsigned (platform-dependent) | +| `c_short` | `short` | | +| `c_int` | `int` | | +| `c_long` | `long` | 32-bit on Windows, 64-bit elsewhere | +| `c_longlong` | `long long` | | +| `c_uchar` | `unsigned char` | | +| `c_ushort` | `unsigned short` | | +| `c_uint` | `unsigned int` | | +| `c_ulong` | `unsigned long` | | +| `c_ulonglong` | `unsigned long long` | | +| `usize` | `size_t` | | +| `isize` | `ptrdiff_t` | | +| `i8`/`u8` | `int8_t`/`uint8_t` | | +| `i16`/`u16` | `int16_t`/`uint16_t` | | +| `i32`/`u32` | `int32_t`/`uint32_t` | | +| `i64`/`u64` | `int64_t`/`uint64_t` | | + +### Pointer Type Mapping + +| Zig Type | C Equivalent | Notes | +|----------|--------------|-------| +| `*T` | `T*` | Non-null pointer | +| `?*T` | `T*` | Nullable pointer | +| `[*]T` | `T*` | Many-item pointer | +| `[*c]T` | `T*` | C pointer (nullable, arithmetic allowed) | +| `*const T` | `const T*` | Const pointer | + +### Extern Structs + +For structs passed across FFI boundary: + +```zig +// Extern struct: C-compatible layout +pub const Point = extern struct { + x: f64, + y: f64, +}; + +// Can be passed by value or pointer +export fn distance(a: Point, b: Point) f64 { + const dx = a.x - b.x; + const dy = a.y - b.y; + return @sqrt(dx * dx + dy * dy); +} +``` + +### Extern Unions + +```zig +pub const Value = extern union { + i: c_int, + f: f32, + p: ?*anyopaque, +}; +``` + +### Extern Enums + +```zig +// Specify backing type for C compatibility +pub const Status = enum(c_int) { + ok = 0, + err_invalid = -1, + err_nomem = -2, +}; + +export fn get_status() Status { + return .ok; +} +``` + +## Building C Libraries + +### Static Library + +```zig +const lib = b.addLibrary(.{ + .name = "mylib", + .linkage = .static, // Creates libmylib.a + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = target, + .optimize = optimize, + }), +}); + +lib.linkLibC(); // If using c_allocator or libc functions +b.installArtifact(lib); +``` + +### Dynamic/Shared Library + +```zig +const lib = b.addLibrary(.{ + .name = "mylib", + .linkage = .dynamic, // Creates libmylib.so / libmylib.dylib / mylib.dll + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = target, + .optimize = optimize, + }), + .version = .{ .major = 1, .minor = 0, .patch = 0 }, +}); + +lib.linkLibC(); +b.installArtifact(lib); +``` + +### Cross-Compilation + +Build for specific targets: + +```zig +// Build for Apple Silicon Mac +const mac_arm = b.resolveTargetQuery(.{ + .cpu_arch = .aarch64, + .os_tag = .macos, +}); + +const lib = b.addLibrary(.{ + .name = "mylib", + .linkage = .static, + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = mac_arm, + .optimize = .ReleaseFast, + }), +}); +``` + +### Multi-Target Build + +```zig +const targets = [_]std.Target.Query{ + .{ .cpu_arch = .x86_64, .os_tag = .macos }, + .{ .cpu_arch = .aarch64, .os_tag = .macos }, + .{ .cpu_arch = .x86_64, .os_tag = .linux, .abi = .gnu }, + .{ .cpu_arch = .aarch64, .os_tag = .linux, .abi = .gnu }, +}; + +for (targets) |t| { + const resolved = b.resolveTargetQuery(t); + const lib = b.addLibrary(.{ + .name = b.fmt("mylib-{s}-{s}", .{ + @tagName(t.cpu_arch.?), + @tagName(t.os_tag.?), + }), + .linkage = .static, + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = resolved, + .optimize = .ReleaseFast, + }), + }); + b.installArtifact(lib); +} +``` + +## Creating Header Files + +Zig does not auto-generate C headers. Write them manually to match exported symbols. + +### Header Template + +```c +#ifndef MYLIB_H +#define MYLIB_H + +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/* Opaque handle type */ +typedef struct mylib_context mylib_context_t; + +/* Lifecycle */ +mylib_context_t* mylib_create(void); +void mylib_destroy(mylib_context_t* ctx); + +/* Operations */ +int mylib_process(mylib_context_t* ctx, const uint8_t* data, size_t len); +const char* mylib_get_error(mylib_context_t* ctx); + +/* Callback type */ +typedef void (*mylib_callback_t)(void* user_data, int result); +void mylib_set_callback(mylib_context_t* ctx, mylib_callback_t cb, void* user_data); + +#ifdef __cplusplus +} +#endif + +#endif /* MYLIB_H */ +``` + +### Matching Zig Implementation + +```zig +const std = @import("std"); + +pub const Context = struct { + allocator: std.mem.Allocator, + error_msg: ?[]const u8 = null, + callback: ?Callback = null, + + const Callback = struct { + func: *const fn (?*anyopaque, c_int) callconv(.c) void, + user_data: ?*anyopaque, + }; +}; + +export fn mylib_create() ?*Context { + const allocator = std.heap.c_allocator; + return allocator.create(Context) catch null; +} + +export fn mylib_destroy(ctx: ?*Context) void { + if (ctx) |c| { + c.allocator.destroy(c); + } +} + +export fn mylib_process(ctx: ?*Context, data: [*]const u8, len: usize) c_int { + const c = ctx orelse return -1; + const slice = data[0..len]; + + // Process data... + _ = slice; + + if (c.callback) |cb| { + cb.func(cb.user_data, 0); + } + + return 0; +} + +export fn mylib_get_error(ctx: ?*Context) [*:0]const u8 { + const c = ctx orelse return "null context"; + return if (c.error_msg) |msg| + msg.ptr + else + "no error"; +} + +export fn mylib_set_callback( + ctx: ?*Context, + cb: ?*const fn (?*anyopaque, c_int) callconv(.c) void, + user_data: ?*anyopaque, +) void { + if (ctx) |c| { + c.callback = if (cb) |f| .{ .func = f, .user_data = user_data } else null; + } +} +``` + +## macOS Integration + +### Universal Binaries (Fat Binaries) + +Build for both architectures and combine with `lipo`: + +**build.zig:** +```zig +pub fn build(b: *std.Build) void { + const optimize = b.standardOptimizeOption(.{}); + + // Build for both architectures + const arm64 = b.resolveTargetQuery(.{ .cpu_arch = .aarch64, .os_tag = .macos }); + const x86_64 = b.resolveTargetQuery(.{ .cpu_arch = .x86_64, .os_tag = .macos }); + + const lib_arm64 = b.addLibrary(.{ + .name = "mylib", + .linkage = .static, + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = arm64, + .optimize = optimize, + }), + }); + + const lib_x86_64 = b.addLibrary(.{ + .name = "mylib", + .linkage = .static, + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = x86_64, + .optimize = optimize, + }), + }); + + // Use lipo to create universal binary + const lipo = b.addSystemCommand(&.{ + "lipo", "-create", "-output", + }); + const universal_lib = lipo.addOutputFileArg("libmylib.a"); + lipo.addFileArg(lib_arm64.getEmittedBin()); + lipo.addFileArg(lib_x86_64.getEmittedBin()); + + // Install universal binary + const install = b.addInstallFile(universal_lib, "lib/libmylib.a"); + + const universal_step = b.step("universal", "Build universal binary"); + universal_step.dependOn(&install.step); +} +``` + +**Manual lipo usage:** +```bash +# Build each architecture +zig build -Dtarget=aarch64-macos -Doptimize=ReleaseFast +mv zig-out/lib/libmylib.a libmylib-arm64.a + +zig build -Dtarget=x86_64-macos -Doptimize=ReleaseFast +mv zig-out/lib/libmylib.a libmylib-x86_64.a + +# Combine into universal binary +lipo -create -output libmylib.a libmylib-arm64.a libmylib-x86_64.a + +# Verify architectures +lipo -info libmylib.a +``` + +### XCFramework Creation + +XCFrameworks are the modern way to distribute libraries for Apple platforms: + +```bash +# 1. Build universal library (see above) + +# 2. Create directory structure +mkdir -p MyLib.xcframework/macos-arm64_x86_64/Headers + +# 3. Copy library and headers +cp libmylib.a MyLib.xcframework/macos-arm64_x86_64/ +cp include/mylib.h MyLib.xcframework/macos-arm64_x86_64/Headers/ + +# 4. Create module map +cat > MyLib.xcframework/macos-arm64_x86_64/Headers/module.modulemap << 'EOF' +module MyLib { + umbrella header "mylib.h" + export * +} +EOF + +# 5. Create Info.plist +cat > MyLib.xcframework/Info.plist << 'EOF' + + + + + AvailableLibraries + + + HeadersPath + Headers + LibraryIdentifier + macos-arm64_x86_64 + LibraryPath + libmylib.a + SupportedArchitectures + + arm64 + x86_64 + + SupportedPlatform + macos + + + CFBundlePackageType + XFWK + XCFrameworkFormatVersion + 1.0 + + +EOF +``` + +**Using xcodebuild (simpler):** +```bash +xcodebuild -create-xcframework \ + -library libmylib.a \ + -headers include/ \ + -output MyLib.xcframework +``` + +## Swift Integration + +### Module Map + +Create `module.modulemap` alongside your header: + +```c +module MyLib { + umbrella header "mylib.h" + export * +} +``` + +### Using from Swift + +```swift +import MyLib + +// Use C functions directly +let result = mylib_init() +if result == 0 { + mylib_set_value(42) + print("Value: \(mylib_get_value())") + mylib_deinit() +} +``` + +### Swift-Friendly Wrapper + +```swift +import MyLib + +class MyLibWrapper { + private var initialized = false + + init?() { + guard mylib_init() == 0 else { return nil } + initialized = true + } + + deinit { + if initialized { + mylib_deinit() + } + } + + var value: Int32 { + get { mylib_get_value() } + set { mylib_set_value(newValue) } + } +} +``` + +### Xcode Project Integration + +1. Drag `MyLib.xcframework` into Xcode project +2. Ensure "Embed & Sign" or "Do Not Embed" (for static libs) +3. Import module: `import MyLib` + +For static libraries without XCFramework: +1. Add library to "Link Binary With Libraries" +2. Add header path to "Header Search Paths" +3. Create bridging header if not using module map + +### Improving Swift Interop (Advanced) + +For better Swift projection, use API notes (`.apinotes` files): + +**MyLib.apinotes:** +```yaml +Name: MyLib +Functions: + - Name: mylib_create + SwiftName: "MyLibContext.create()" + NullabilityOfRet: N # Non-null (returns Optional in Swift) + - Name: mylib_destroy + SwiftName: "MyLibContext.destroy(self:)" + - Name: mylib_get_error + NullabilityOfRet: N + ResultType: "const char * _Nonnull" +``` + +See [Swift.org: Improving the Usability of C APIs](https://www.swift.org/documentation/cxx-interop/) for more. + +## Common Patterns + +### Opaque Pointers + +Hide implementation details from C consumers: + +```zig +const std = @import("std"); + +const InternalState = struct { + allocator: std.mem.Allocator, + data: std.ArrayList(u8), + // Complex internal state... +}; + +// C sees: typedef struct handle handle_t; +// (opaque, can't access fields) + +export fn handle_create() ?*InternalState { + const allocator = std.heap.c_allocator; + const state = allocator.create(InternalState) catch return null; + state.* = .{ + .allocator = allocator, + .data = std.ArrayList(u8).init(allocator), + }; + return state; +} + +export fn handle_destroy(h: ?*InternalState) void { + if (h) |state| { + state.data.deinit(); + state.allocator.destroy(state); + } +} +``` + +### Error Handling Across FFI + +Zig errors can't cross FFI boundary. Use return codes or out parameters: + +```zig +pub const ErrorCode = enum(c_int) { + ok = 0, + invalid_argument = -1, + out_of_memory = -2, + io_error = -3, + unknown = -99, +}; + +export fn process_data( + data: [*]const u8, + len: usize, + out_result: *c_int, +) ErrorCode { + const slice = data[0..len]; + + // Internal Zig code can use errors + const result = processInternal(slice) catch |err| { + return switch (err) { + error.OutOfMemory => .out_of_memory, + error.InvalidData => .invalid_argument, + else => .unknown, + }; + }; + + out_result.* = result; + return .ok; +} + +fn processInternal(data: []const u8) !c_int { + // Full Zig error handling here + if (data.len == 0) return error.InvalidData; + return @intCast(data.len); +} +``` + +### Callbacks + +C callbacks with user data: + +```zig +const CallbackFn = *const fn ( + user_data: ?*anyopaque, + event_type: c_int, + event_data: ?*const anyopaque, +) callconv(.c) void; + +var stored_callback: ?CallbackFn = null; +var stored_user_data: ?*anyopaque = null; + +export fn register_callback(cb: ?CallbackFn, user_data: ?*anyopaque) void { + stored_callback = cb; + stored_user_data = user_data; +} + +export fn trigger_event(event_type: c_int) void { + if (stored_callback) |cb| { + cb(stored_user_data, event_type, null); + } +} +``` + +### String Handling + +Zig slices vs C strings: + +```zig +const std = @import("std"); + +// Accept C string, return length +export fn string_length(s: [*:0]const u8) usize { + return std.mem.len(s); +} + +// Accept pointer + length (more efficient) +export fn process_string(s: [*]const u8, len: usize) c_int { + const slice = s[0..len]; + // Process slice... + _ = slice; + return 0; +} + +// Return C string (must be static or allocated) +const greeting: [:0]const u8 = "Hello from Zig!"; + +export fn get_greeting() [*:0]const u8 { + return greeting.ptr; +} + +// Allocate string for caller to free +export fn alloc_string(len: usize) ?[*:0]u8 { + const allocator = std.heap.c_allocator; + const buf = allocator.allocSentinel(u8, len, 0) catch return null; + return buf.ptr; +} + +export fn free_string(s: ?[*:0]u8) void { + if (s) |ptr| { + const allocator = std.heap.c_allocator; + // Need to know length to free - typically tracked separately + // or use c_allocator which can query allocation size + _ = allocator; + _ = ptr; + } +} +``` + +### Thread Safety + +For thread-safe libraries, use atomics or mutexes: + +```zig +const std = @import("std"); + +var global_mutex: std.Thread.Mutex = .{}; +var shared_value: c_int = 0; + +export fn thread_safe_increment() c_int { + global_mutex.lock(); + defer global_mutex.unlock(); + + shared_value += 1; + return shared_value; +} + +// Or use atomics for simple cases +var atomic_counter: std.atomic.Value(c_int) = .init(0); + +export fn atomic_increment() c_int { + return atomic_counter.fetchAdd(1, .seq_cst) + 1; +} +``` + +### Versioning + +Export version info for runtime checking: + +```zig +pub const version_major: c_int = 1; +pub const version_minor: c_int = 2; +pub const version_patch: c_int = 3; + +comptime { + @export(&version_major, .{ .name = "mylib_version_major" }); + @export(&version_minor, .{ .name = "mylib_version_minor" }); + @export(&version_patch, .{ .name = "mylib_version_patch" }); +} + +export fn mylib_version_string() [*:0]const u8 { + return "1.2.3"; +} +``` + +**Header:** +```c +extern const int mylib_version_major; +extern const int mylib_version_minor; +extern const int mylib_version_patch; +const char* mylib_version_string(void); +``` diff --git a/references/code-review.md b/references/code-review.md new file mode 100644 index 0000000..ed887c5 --- /dev/null +++ b/references/code-review.md @@ -0,0 +1,1452 @@ +# Zig Code Review Reference (Zig 0.16.0) + +Systematic code review checklist organized by detection confidence level. Work through sections in order: ALWAYS FLAG -> FLAG WITH CONTEXT -> SUGGEST. For version-specific upgrade issues, load [Zig 0.16 Release Notes Migration Reference](zig-0.16-release-notes.md) first. + +## Quick Reference Card + +### Scan Order + +1. **ALWAYS FLAG** - Mechanical checks, objectively wrong +2. **FLAG WITH CONTEXT** - Requires understanding function semantics +3. **SUGGEST** - Style suggestions for review feedback + +### Most Common Issues + +| Detect | Issue | Risk | Fix | +|--------|-------|------|-----| +| file/process/net/time/random API without `io` | Old 0.15 API or bad 0.16 design | Compile error or wrong runtime ownership | Add/pass/store `std.Io` | +| `std.fs.cwd()` / `std.fs.File` / `std.fs.Dir` | File-system API moved | Compile error | Use `std.Io.Dir` / `std.Io.File` | +| `std.process.Child.run` / `Child.init` | Process API changed | Compile error | Use `std.process.run(gpa, io, ...)` / `spawn(io, ...)` | +| `std.Thread.Mutex` in I/O path | Blocking sync bypasses Io backend | Cancelation/perf/runtime bugs | Use `std.Io.Mutex` and pass `io` | +| `@Type(` | Removed builtin | Compile error | Use `@Int`, `@Struct`, `@Union`, `@Enum`, etc. | +| `@cImport` for new bindings | Deprecated path | Future breakage / translate-c drift | Use `b.addTranslateC(...)` | +| `writer(&buf)` without `.flush()` | Missing flush | Data loss | Add `try w.flush()` | +| `.{}` on container type | Wrong init | Compile error | Use `.empty` or `.init` | +| `root_source_file` in build.zig | Old build API | Compile error | Use `root_module = b.createModule(...)` | +| `"{}"` with custom format method | Old format spec | Compile error | Use `"{f}"` (see [1.4](#14-api-signature-changes)) | +| `getOrPut` then `ensure*` | Exception safety | Orphan entries | Reserve first, mutate with `*AssumeCapacity` | + +### Guaranteed Bug Patterns + +| Detect | Risk | Verify | +|--------|------|--------| +| `@intCast(val)` without bounds check | Runtime panic | No `std.math.cast` or prior validation | +| `.?` unwrap | Runtime panic | Not guarded by `if` or `orelse` | +| `catch unreachable` on alloc/create | Runtime panic | Allocation can fail | +| `return &local_var` | Dangling pointer | Returns address of stack variable | +| `&packed_struct.field` | Undefined behavior | Pointer to packed struct field | +| pointer field in `packed struct`/`packed union` | Forbidden in 0.16 | Compile error / invalid layout | Store integer representation only if ABI requires it | +| `DebugAllocator` without `defer.*deinit()` | No leak detection | Missing cleanup | +| build.zig without `standardTargetOptions` | No cross-compile | Missing standard options | + +### Context-Dependent Checks + +| Detect | When to Flag | Section | +|--------|--------------|---------| +| `anyerror` return type | Library public API | [2.7](#27-error-handling-selection) | +| Loop allocations with non-freeing allocator | Arena without reset | [2.8](#28-loop-resource-management) | +| `[*]T` pointer type | Pure Zig code (not FFI) | [2.10](#210-pointer-type-selection) | +| Regular `for` on `@typeInfo` fields | Need comptime iteration | [2.11](#211-comptime-propagation) | +| `ArenaAllocator` without `.reset()` | Long-running service | [2.9](#29-allocator-misuse-patterns) | +| Multi-step alloc without errdefer | Partial construction | [2.12](#212-missing-errdefer-for-partial-construction) | +| Alloc without `defer free` + error returns | Resource leak | [2.13](#213-missing-defer-for-allocation-cleanup) | + +### Style Quick Checks + +| Detect | Suggestion | Section | +|--------|------------|---------| +| Imports not grouped | std → third-party → local | [3.8](#38-import-organization) | +| Global allocator variable | Accept as parameter | [3.9](#39-allocator-design) | +| `allocPrint` for bounded strings | Use `bufPrint` with stack buffer | [3.10](#310-stack-vs-heap-allocation) | +| Runtime constant lookup | Comptime lookup table | [3.11](#311-comptime-optimization) | +| `expectEqual` with strings/slices | Use `expectEqualStrings`/`expectEqualSlices` | [3.14](#314-testing-best-practices) | + +--- + +## Review Workflow + +1. **Run `zig fmt`** — catches whitespace, trailing commas, basic style +2. **Scan for ALWAYS FLAG patterns** — removed features, old APIs, guaranteed bugs +3. **Check 0.16 I/O ownership** — `std.Io` parameter flow, Juicy Main, tests with `std.testing.io` +4. **Check safety** — defer/errdefer usage, reserve-first pattern, memory poisoning +5. **Check context-dependent issues** — allocator naming, error handling, loop resources +6. **Review style** — naming conventions, struct init, imports, documentation +7. **Verify build.zig** — `standardTargetOptions`, `standardOptimizeOption`, `root_module`, `addTranslateC` + +--- + +## Table of Contents + +- [1. ALWAYS FLAG (100% Confidence)](#1-always-flag-100-confidence) +- [2. FLAG WITH CONTEXT (High Confidence)](#2-flag-with-context-high-confidence) +- [3. SUGGEST (Advisory)](#3-suggest-advisory) + +--- + +## 1. ALWAYS FLAG (100% Confidence) + +Objective errors that cause compilation failures or guaranteed runtime bugs. + +### 1.1 Removed Language Features + +| Detect | Replacement | Risk | +|--------|-------------|------| +| `usingnamespace` | Explicit re-exports | Compile error | +| `async`/`await` keywords | Removed entirely | Compile error | +| `@fence()` | Stronger atomic orderings or RMW operations | Compile error | +| `@setCold(true/false)` | `@branchHint(.cold)` | Compile error | +| `@setAlignStack()` | `callconv(.withStackAlign(...))` | Compile error | +| `@Type(...)` | Specific 0.16 type builtin such as `@Int`/`@Struct` | Compile error | +| `std.Thread.Pool` | Application scheduler or `std.Io.Group`/`io.async` where semantically correct | Compile error | +| `std.Thread.Mutex.Recursive` | Redesign locking or use explicit state | Compile error | +| `std.BoundedArray` | `ArrayList.initBuffer()` | Compile error | + +**Wrong:** +```zig +pub usingnamespace @import("other.zig"); +``` + +**Right:** +```zig +const other = @import("other.zig"); +pub const foo = other.foo; +``` + +**Wrong:** +```zig +fn coldPath() void { + @setCold(true); +} +``` + +**Right:** +```zig +fn coldPath() void { + @branchHint(.cold); // Must be first statement +} +``` + +**Wrong:** +```zig +var stack = try std.BoundedArray(i32, 8).fromSlice(initial); +``` + +**Right:** +```zig +var buffer: [8]i32 = undefined; +var stack = std.ArrayList(i32).initBuffer(&buffer); +try stack.appendSliceBounded(initial); +``` + +### 1.2 Changed Syntax (0.14+) + +| Detect | New | Risk | +|--------|-----|------| +| `@export(foo, opts)` | `@export(&foo, opts)` | Compile error | +| `.Int`, `.Struct` in @typeInfo | `.int`, `.@"struct"` | Compile error | +| `.One`, `.Slice`, `.Many` | `.one`, `.slice`, `.many` | Compile error | +| `sentinel = &val` | `sentinel_ptr = &val` | Compile error | +| Inline asm clobbers `"rcx"` | `.{ .rcx = true }` | Compile error | +| `callconv(.C)` | `callconv(.c)` | Compile error | +| `callconv(.Stdcall)` | `callconv(.x86_stdcall)` | Compile error | + +**Wrong:** +```zig +@export(foo, .{ .name = "bar" }); +``` + +**Right:** +```zig +@export(&foo, .{ .name = "bar" }); +``` + +**Wrong:** +```zig +switch (@typeInfo(T)) { + .Int => {}, + .Struct => {}, + .Pointer => |p| if (p.size == .One) {}, +} +``` + +**Right:** +```zig +switch (@typeInfo(T)) { + .int => {}, + .@"struct" => {}, + .pointer => |p| if (p.size == .one) {}, +} +``` + +**Wrong:** +```zig +asm volatile ("syscall" + : [ret] "={rax}" (-> usize), + : [number] "{rax}" (number), + : "rcx", "r11" +); +``` + +**Right:** +```zig +asm volatile ("syscall" + : [ret] "={rax}" (-> usize), + : [number] "{rax}" (number), + : .{ .rcx = true, .r11 = true } +); +``` + +**Wrong:** +```zig +export fn foo() callconv(.C) void {} +``` + +**Right:** +```zig +export fn foo() callconv(.c) void {} +``` + +### 1.3 Removed/Renamed APIs + +| Detect | New | Risk | +|--------|-----|------| +| `root_source_file` | `root_module = b.createModule(...)` | Compile error | +| `exe.addModule(...)` | `exe.root_module.addImport(...)` | Compile error | +| `GeneralPurposeAllocator` | `DebugAllocator` | Alias works | +| `std.mem.page_size` | `std.heap.pageSize()` | Compile error | +| `BufferedWriter` | Buffer provided to `.writer(&buf)` | Compile error | +| `CountingWriter` | `std.Io.Writer.Discarding` | Compile error | +| `GenericWriter/Reader` | `std.Io.Writer/Reader` | Deprecated | +| `std.ArrayList` (managed) | `std.array_list.Managed` | Eventually removed | +| `std.ArrayListUnmanaged` | `std.ArrayList` | Unmanaged is now the default | +| `std.fifo.LinearFifo` | `std.Io.Reader`/`Writer` patterns | Removed | +| `std.RingBuffer` | `std.Io.Reader`/`Writer` patterns | Removed | +| `std.ChildProcess` | `std.process.Child` | Compile error | +| `*std.build.Builder` | `*std.Build` | Compile error | +| `.{ .path = "..." }` | `b.path("...")` | Compile error | + +**Wrong:** +```zig +b.addExecutable(.{ + .name = "app", + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, +}); +``` + +**Right:** +```zig +b.addExecutable(.{ + .name = "app", + .root_module = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, + }), +}); +``` + +**Wrong:** +```zig +exe.addModule("helper", helper_mod); +``` + +**Right:** +```zig +exe.root_module.addImport("helper", helper_mod); +``` + +### 1.4 API Signature Changes + +*Note: For `"{}"` → `"{f}"`, this is a migration check. For new code forgetting `.flush()`, see also [2.2](#22-missing-flush-after-io-write).* + +| Detect | New | Risk | +|--------|-----|------| +| `stdout.print(fmt, args)` without flush | Must call `stdout.flush()` | Data loss | +| `format(self, fmt, opts, writer)` | `format(self, *std.Io.Writer)` | Compile error | +| `"{}"` for format methods | `"{f}"` | Compile error | +| `std.process.Child.run` / `Child.init` | `std.process.run(gpa, io, ...)` / `std.process.spawn(io, ...)` | Compile error | + +**Wrong:** +```zig +const stdout = std.io.getStdOut().writer(); +try stdout.print("Hello\n", .{}); +``` + +**Right:** +```zig +var buf: [4096]u8 = undefined; +var stdout_writer = std.Io.File.stdout().writer(io, &buf); +const stdout = &stdout_writer.interface; +try stdout.print("Hello\n", .{}); +try stdout.flush(); // Required +``` + +**Wrong:** +```zig +pub fn format( + self: @This(), + comptime fmt: []const u8, + opts: std.fmt.FormatOptions, + writer: anytype, +) !void { ... } +``` + +**Right:** +```zig +pub fn format(self: @This(), writer: *std.Io.Writer) std.Io.Writer.Error!void { ... } +``` + +**Wrong:** +```zig +std.debug.print("{}", .{myFormattableType}); +``` + +**Right:** +```zig +std.debug.print("{f}", .{myFormattableType}); +``` + +### 1.5 Container Initialization + +| Detect | Right | Risk | +|--------|-------|------| +| `var list: ArrayList(T) = .{}` | `.empty` | Deprecated | +| `var gpa: DebugAllocator(.{}) = .{}` | `.init` | Deprecated | +| `var map: HashMapUnmanaged(...) = .{}` | `.empty` | Deprecated | +| `list.append(42)` (old managed API) | `try list.append(allocator, 42)` | Compile error | + +**Wrong:** +```zig +var list: std.ArrayList(u32) = .{}; +var gpa: std.heap.DebugAllocator(.{}) = .{}; +``` + +**Right:** +```zig +var list: std.ArrayList(u32) = .empty; +var map: std.AutoHashMapUnmanaged(u32, u32) = .empty; +var gpa: std.heap.DebugAllocator(.{}) = .init; +var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator); +``` + +**ArrayList is now unmanaged by default** — allocator is passed per method call, not stored: + +**Wrong (old managed API):** +```zig +var list = std.ArrayList(u32).init(allocator); +list.append(42); +``` + +**Right (unmanaged, allocator per call):** +```zig +var list: std.ArrayList(u32) = .empty; +try list.append(allocator, 42); +try list.appendSlice(allocator, &.{ 1, 2, 3 }); +defer list.deinit(allocator); +``` + +### 1.6 Guaranteed Runtime Bugs + +#### 1.6.1 Unchecked @intCast + +| Detect | `@intCast` without prior bounds check or `std.math.cast` | +|--------|----------------------------------------------------------| +| Risk | Runtime panic on overflow | + +**Wrong:** +```zig +fn convert(big: u64) u8 { + return @intCast(big); // Panics if big > 255 +} +``` + +**Right:** +```zig +fn convert(big: u64) ?u8 { + return std.math.cast(u8, big); +} +``` + +**Verify:** Search for `@intCast` calls, check for prior validation. + +#### 1.6.2 Unguarded Optional Unwrap + +| Detect | `.?` not preceded by `if` check or `orelse` | +|--------|---------------------------------------------| +| Risk | Runtime panic if null | + +**Wrong:** +```zig +fn getName(user: ?*User) []const u8 { + return user.?.name; +} +``` + +**Right:** +```zig +fn getName(user: ?*User) []const u8 { + return if (user) |u| u.name else "anonymous"; +} +``` + +**Verify:** Search for `.?` usage, confirm null case is handled. + +#### 1.6.3 Catch Unreachable on Allocation + +| Detect | `catch unreachable` after `alloc`/`create` calls | +|--------|--------------------------------------------------| +| Risk | Runtime panic on OOM | + +**Wrong:** +```zig +fn createBuffer(allocator: Allocator) *Buffer { + return allocator.create(Buffer) catch unreachable; +} +``` + +**Right:** +```zig +fn createBuffer(allocator: Allocator) !*Buffer { + return try allocator.create(Buffer); +} +``` + +**Verify:** Search for `catch unreachable`, check if operation can fail. + +#### 1.6.4 Returning Stack Pointer + +| Detect | `return &` of local variable | +|--------|------------------------------| +| Risk | Dangling pointer | + +**Wrong:** +```zig +fn getBuffer() *[256]u8 { + var buf: [256]u8 = undefined; + return &buf; +} +``` + +**Right:** +```zig +fn fillBuffer(buf: *[256]u8) void { ... } + +// OR return by value if small +fn getBuffer() [256]u8 { + var buf: [256]u8 = undefined; + return buf; +} +``` + +**Verify:** Search for `return &`, check if variable is local. + +#### 1.6.5 Self-Referential Struct Copy + +| Detect | Struct with self-pointer field returned by value | +|--------|--------------------------------------------------| +| Risk | Dangling internal pointer | + +**Wrong:** +```zig +const Node = struct { + self_ptr: *Node, + + fn init() Node { + var node: Node = undefined; + node.self_ptr = &node; + return node; // self_ptr now dangling + } +}; +``` + +**Right:** +```zig +fn init(allocator: Allocator) !*Node { + const node = try allocator.create(Node); + node.self_ptr = node; + return node; +} +``` + +**Verify:** Check structs with pointer fields pointing to self. + +#### 1.6.6 Division by Runtime Variable + +| Detect | `/` or `%` with non-constant divisor | +|--------|--------------------------------------| +| Risk | Runtime panic if zero | + +**Wrong:** +```zig +fn average(sum: u32, count: u32) u32 { + return sum / count; +} +``` + +**Right:** +```zig +fn average(sum: u32, count: u32) ?u32 { + if (count == 0) return null; + return sum / count; +} +``` + +**Verify:** Search for division, check if divisor is validated. + +#### 1.6.7 Inactive Union Field Access + +| Detect | Direct union field access without tag check | +|--------|---------------------------------------------| +| Risk | Undefined behavior | + +**Wrong:** +```zig +fn getInt(v: Value) i64 { + return v.int; // UB if v is not .int +} +``` + +**Right:** +```zig +fn getInt(v: Value) ?i64 { + return switch (v) { + .int => |i| i, + else => null, + }; +} +``` + +**Verify:** Check union field access, ensure tag is verified first. + +#### 1.6.8 Unchecked @enumFromInt + +| Detect | `@enumFromInt` without range validation | +|--------|----------------------------------------| +| Risk | Runtime panic if invalid | + +**Wrong:** +```zig +fn parseColor(byte: u8) Color { + return @enumFromInt(byte); +} +``` + +**Right:** +```zig +fn parseColor(byte: u8) ?Color { + return std.meta.intToEnum(Color, byte) catch null; +} +``` + +**Verify:** Search for `@enumFromInt`, check for validation. + +### 1.7 Memory Safety Violations + +#### 1.7.1 Missing DebugAllocator Deinit + +| Detect | `DebugAllocator` without `defer.*deinit()` | +|--------|-------------------------------------------| +| Risk | No leak detection in debug builds | + +**Wrong:** +```zig +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + const allocator = gpa.allocator(); + // Missing gpa.deinit() +} +``` + +**Right:** +```zig +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + const allocator = gpa.allocator(); +} +``` + +**Verify:** Search for `DebugAllocator`, check for `defer.*deinit()`. + +#### 1.7.2 @ptrCast Size Mismatch + +| Detect | `@ptrCast` between types of different sizes | +|--------|---------------------------------------------| +| Risk | Memory corruption | + +**Wrong:** +```zig +fn dangerous(ptr: *u32) *u64 { + return @ptrCast(ptr); // Reading u64 from u32 space +} +``` + +**Right:** +```zig +fn reinterpret(ptr: *u32) *[4]u8 { + return @ptrCast(ptr); // Same size +} +``` + +**Verify:** Check `@ptrCast` source and target sizes match. + +#### 1.7.3 Packed Struct Field Pointer + +| Detect | `&packed_struct.field` | +|--------|------------------------| +| Risk | Unaligned access UB | + +**Wrong:** +```zig +const Packet = packed struct { flags: u4, len: u12, data: u16 }; + +fn getLen(pkt: *Packet) *u12 { + return &pkt.len; // Unaligned pointer +} +``` + +**Right:** +```zig +fn getLen(pkt: *const Packet) u12 { + return pkt.len; // Copy value +} +``` + +**Verify:** Search for `&` on packed struct field access. + +#### 1.7.4 Disabled Runtime Safety + +| Detect | `@setRuntimeSafety(false)` | +|--------|---------------------------| +| Risk | All safety checks removed | + +**Wrong:** +```zig +fn fastPath(data: []u8) void { + @setRuntimeSafety(false); +} +``` + +**Right:** +```zig +fn fastPath(data: []u8) void { + // Use ReleaseFast build mode for controlled optimization +} +``` + +**Verify:** Search for `@setRuntimeSafety(false)`. + +### 1.8 Build System Anti-patterns + +| Detect | Problem | Risk | +|--------|---------|------| +| Missing `b.standardTargetOptions()` | No cross-compilation | Build inflexibility | +| Missing `b.standardOptimizeOption()` | No release builds | Build inflexibility | + +**Wrong:** +```zig +pub fn build(b: *std.Build) void { + const exe = b.addExecutable(.{ + .name = "app", + .root_module = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + // Missing target and optimize + }), + }); +} +``` + +**Right:** +```zig +pub fn build(b: *std.Build) void { + const target = b.standardTargetOptions(.{}); + const optimize = b.standardOptimizeOption(.{}); + + const exe = b.addExecutable(.{ + .name = "app", + .root_module = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, + }), + }); +} +``` + +**Verify:** Check build.zig for `standardTargetOptions` and `standardOptimizeOption`. + +--- + +## 2. FLAG WITH CONTEXT (High Confidence) + +Genuine issues when preconditions are satisfied. + +### 2.1 Exception Safety Bugs + +| Detect | `getOrPut` followed by `ensureCapacity` in same function | +|--------|----------------------------------------------------------| +| Risk | Orphan entries on allocation failure | + +**Preconditions:** +- [ ] Function inserts into multiple growable containers +- [ ] Mutation precedes reservation + +**If met:** Reorder to reserve-first pattern. + +**Wrong:** +```zig +const gop = try state.table.getOrPut(gpa, key); +try state.bytes.ensureUnusedCapacity(gpa, len); // Failure leaves orphan +``` + +**Right:** +```zig +try state.table.ensureUnusedCapacityContext(gpa, 1, ctx); +try state.bytes.ensureUnusedCapacity(gpa, len); + +errdefer comptime unreachable; // Assert no errors after this + +const gop = state.table.getOrPutAssumeCapacity(key); +state.bytes.appendSliceAssumeCapacity(data); +``` + +**Verify:** Search for `getOrPut` followed by `ensure` in same scope. + +### 2.2 Missing flush() After I/O Write + +*Note: For migrating old `stdout.print()` calls, see also [1.4](#14-api-signature-changes). This section covers new code that forgets `.flush()`.* + +| Detect | `writer(&buf)` or `.writer(&` without subsequent `.flush()` | +|--------|-------------------------------------------------------------| +| Risk | Data loss (buffered data not written) | + +**Preconditions:** +- [ ] Code uses a buffered `std.Io.Writer` +- [ ] Writer scope ends without flush + +**If met:** Add `try writer.interface.flush()` before scope exit. + +**Wrong:** +```zig +var buf: [4096]u8 = undefined; +var writer = file.writer(io, &buf); +try writer.interface.print("data", .{}); +// Missing flush +``` + +**Right:** +```zig +var buf: [4096]u8 = undefined; +var writer = file.writer(io, &buf); +try writer.interface.print("data", .{}); +try writer.interface.flush(); +``` + +**Verify:** Check for `writer(&` without corresponding `.flush()`. + +### 2.3 Allocator Pointer Comparison + +| Detect | `alloc1.ptr == alloc2.ptr` or similar | +|--------|---------------------------------------| +| Risk | Undefined for stateless allocators | + +**Preconditions:** +- [ ] Code compares `Allocator` or `Random` interface `ptr` fields + +**If met:** Remove comparison; `ptr` is undefined for `page_allocator`, `c_allocator`. + +### 2.4 Generic Allocator Naming + +| Detect | Allocator parameter named just `allocator` | +|--------|-------------------------------------------| +| Risk | Hidden memory ownership contract | + +**Preconditions:** +- [ ] Function has allocator parameter +- [ ] Ownership semantics unclear from name + +**If met:** Rename by memory contract: + +| Name | Contract | Return Data? | Who Frees? | +|------|----------|--------------|------------| +| `gpa` | Long-lived | Yes | Caller | +| `arena` | Request-scoped | Yes | Arena owner | +| `scratch` | Function-private | Never | Function | + +**Wrong:** +```zig +fn process(allocator: Allocator) ![]u8 { + const temp = try allocator.alloc(u8, 100); + const result = try allocator.dupe(u8, temp); + allocator.free(temp); + return result; +} +``` + +**Right:** +```zig +fn handleRequest( + arena: Allocator, // Response lifetime + gpa: Allocator, // Long-lived cache + scratch: Allocator, // Function temporaries +) !Response { ... } +``` + +### 2.5 Use-After-Free Potential + +| Detect | `deinit` method without `self.* = undefined;` | +|--------|-----------------------------------------------| +| Risk | Use-after-free not caught in debug | + +**Preconditions:** +- [ ] Type has `deinit` method that frees memory +- [ ] No memory poisoning after deallocation + +**If met:** Add `self.* = undefined;` at end of deinit. + +**Wrong:** +```zig +pub fn deinit(self: *Self, gpa: Allocator) void { + gpa.free(self.allocatedSlice()); +} +``` + +**Right:** +```zig +pub fn deinit(self: *Self, gpa: Allocator) void { + gpa.free(self.allocatedSlice()); + self.* = undefined; +} +``` + +**Verify:** Check `deinit` methods for memory poisoning. + +### 2.6 Type-Unsafe Index Usage + +| Detect | Multiple `u32`/`usize` indices for different arrays | +|--------|-----------------------------------------------------| +| Risk | Index type confusion | + +**Preconditions:** +- [ ] Code manages parallel arrays +- [ ] Same integer type used for different index spaces + +**If met:** Use distinct enum types for indices. + +**Wrong:** +```zig +fn getSection(index: u32) *Section { ... } +fn getSymbol(index: u32) *Symbol { ... } +``` + +**Right:** +```zig +const SectionIndex = enum(u32) { _ }; +const SymbolIndex = enum(u32) { _ }; + +fn getSection(index: SectionIndex) *Section { ... } +fn getSymbol(index: SymbolIndex) *Symbol { ... } +``` + +### 2.7 Error Handling Selection + +| Detect | `anyerror` return in public API, or blind `try` propagation | +|--------|-------------------------------------------------------------| +| Risk | Callers cannot handle errors specifically | + +**Preconditions:** +- [ ] Library public function returns `anyerror` +- [ ] OR: Function uses `try` where specific handling needed + +**If met:** Define specific error set; use `catch` with `switch` for meaningful errors. + +**Wrong:** +```zig +pub fn parse(input: []const u8) anyerror!Ast { ... } +``` + +**Right:** +```zig +pub const ParseError = error{ UnexpectedToken, InvalidSyntax, OutOfMemory }; +pub fn parse(input: []const u8) ParseError!Ast { ... } +``` + +### 2.8 Loop Resource Management + +| Detect | Allocations inside loop body | +|--------|------------------------------| +| Risk | Memory accumulation or premature free | + +**Preconditions:** +- [ ] Loop body allocates memory +- [ ] Allocator does not actually free (e.g., `ArenaAllocator`) +- [ ] OR: Results accumulate across iterations where `defer` would free too early + +**If met:** Use arena with per-iteration reset, or manage lifetime explicitly. + +Zig's `defer` is **block-scoped** — it runs at the end of each loop iteration, not at function exit. This means `defer free` inside a loop works correctly for per-iteration cleanup. The real risks are: + +**Risk 1: Non-freeing allocator makes `defer free` a no-op:** + +**Wrong:** +```zig +// ArenaAllocator.free() is a no-op — memory accumulates +for (items) |item| { + const data = try arena.alloc(u8, item.size); + defer arena.free(data); // Does nothing! + try process(data); +} +``` + +**Right:** +```zig +// Use a scratch arena with per-iteration reset +var scratch = std.heap.ArenaAllocator.init(backing_allocator); +defer scratch.deinit(); + +for (items) |item| { + defer _ = scratch.reset(.retain_capacity); + const data = try scratch.allocator().alloc(u8, item.size); + try process(data); +} +``` + +**Risk 2: Accumulating results across iterations where `defer` would free too early:** + +**Wrong:** +```zig +var results: std.ArrayList([]u8) = .empty; +for (items) |item| { + const data = try allocator.dupe(u8, item.name); + defer allocator.free(data); // Frees before we're done with it! + try results.append(allocator, data); +} +``` + +**Right:** +```zig +var results: std.ArrayList([]u8) = .empty; +for (items) |item| { + const data = try allocator.dupe(u8, item.name); + errdefer allocator.free(data); // Only free on error + try results.append(allocator, data); +} +``` + +### 2.9 Allocator Misuse Patterns + +#### FixedBufferAllocator Non-LIFO Free + +| Detect | Free in non-LIFO order with FixedBufferAllocator | +|--------|--------------------------------------------------| +| Risk | Unexpected allocation failures | + +**Preconditions:** +- [ ] Using `FixedBufferAllocator` +- [ ] Freeing allocations out of order + +**If met:** Use LIFO order or switch to ArenaAllocator. + +#### ArenaAllocator Without Reset + +| Detect | `ArenaAllocator` in loop without `.reset()` | +|--------|---------------------------------------------| +| Risk | Unbounded memory growth | + +**Preconditions:** +- [ ] Long-running service/loop +- [ ] Arena never reset + +**If met:** Add `defer _ = arena.reset(.retain_capacity);` in loop. + +**Wrong:** +```zig +while (true) { + const request = try readRequest(arena.allocator()); + try processRequest(request); + // Missing reset +} +``` + +**Right:** +```zig +while (true) { + defer _ = arena.reset(.retain_capacity); + const request = try readRequest(arena.allocator()); + try processRequest(request); +} +``` + +### 2.10 Pointer Type Selection + +| Detect | `[*]T` in pure Zig code (not C FFI) | +|--------|-------------------------------------| +| Risk | No bounds checking | + +**Preconditions:** +- [ ] Code uses `[*]T` pointer +- [ ] Not at C FFI boundary + +**If met:** Use slice `[]T` instead. + +**Wrong:** +```zig +fn process(data: [*]u8, len: usize) void { + var i: usize = 0; + while (i < len) : (i += 1) { + data[i] = transform(data[i]); + } +} +``` + +**Right:** +```zig +fn process(data: []u8) void { + for (data) |*byte| { + byte.* = transform(byte.*); + } +} +``` + +### 2.11 Comptime Propagation + +| Detect | Regular `for` iterating `@typeInfo` fields | +|--------|-------------------------------------------| +| Risk | Comptime values lost at runtime boundary | + +**Preconditions:** +- [ ] Iterating comptime-known collection +- [ ] Using field names or other comptime data + +**If met:** Use `inline for` or mark function `inline`. + +**Wrong:** +```zig +fn printFields(comptime T: type) void { + for (@typeInfo(T).@"struct".fields) |field| { + std.debug.print("{s}\n", .{field.name}); // Error + } +} +``` + +**Right:** +```zig +fn printFields(comptime T: type) void { + inline for (@typeInfo(T).@"struct".fields) |field| { + std.debug.print("{s}\n", .{field.name}); + } +} +``` + +### 2.12 Missing errdefer for Partial Construction + +| Detect | Multi-step allocation without errdefer cleanup | +|--------|------------------------------------------------| +| Risk | Resource leak on failure | + +**Preconditions:** +- [ ] Function performs multiple fallible allocations +- [ ] Earlier allocations not cleaned up if later ones fail + +**If met:** Add `errdefer` after each allocation that must be cleaned up on failure. + +**Wrong:** +```zig +pub fn init(gpa: Allocator) !Self { + const data = try gpa.alloc(u8, 100); + const more = try gpa.alloc(u8, 200); // data leaks if this fails + return .{ .data = data, .more = more }; +} +``` + +**Right:** +```zig +pub fn init(gpa: Allocator) !Self { + const data = try gpa.alloc(u8, 100); + errdefer gpa.free(data); + const more = try gpa.alloc(u8, 200); + return .{ .data = data, .more = more }; +} +``` + +**Verify:** Search for functions with multiple `try allocator.alloc`/`create` calls. Check that each allocation (except the last) has a corresponding `errdefer`. + +### 2.13 Missing defer for Allocation Cleanup + +| Detect | Allocation without corresponding `defer free` in function with error/early-return paths | +|--------|----------------------------------------------------------------------------------------| +| Risk | Memory leak on non-error paths | + +**Preconditions:** +- [ ] Function allocates memory for internal use (not returned to caller) +- [ ] Function has `try`, `return`, or other early-exit paths after allocation +- [ ] No `defer` to free the allocation + +**If met:** Add `defer allocator.free(...)` immediately after allocation. + +This is distinct from `errdefer` ([2.12](#212-missing-errdefer-for-partial-construction)) — `errdefer` only fires on error. This covers unconditional cleanup of internally-used allocations. + +**Wrong:** +```zig +fn process(gpa: Allocator, input: []const u8) !Result { + const temp = try gpa.alloc(u8, input.len); + const parsed = try parse(input); // temp leaks if this fails + // ... use temp ... + gpa.free(temp); + return parsed; +} +``` + +**Right:** +```zig +fn process(gpa: Allocator, input: []const u8) !Result { + const temp = try gpa.alloc(u8, input.len); + defer gpa.free(temp); + const parsed = try parse(input); // temp freed by defer + // ... use temp ... + return parsed; +} +``` + +**Verify:** Search for `alloc(` not followed by `defer.*free` in functions with `try` or early returns. + +--- + +## 3. SUGGEST (Advisory) + +Style and idiom suggestions. + +### 3.1 Unnecessary Self Alias + +| Detect | `const Self = @This();` used only once | +|--------|----------------------------------------| +| Risk | None (style) | + +**Wrong:** +```zig +pub const Iterator = struct { + const Self = @This(); // Used once + pub fn next(self: *Self) ?Entry { ... } +}; +``` + +**Right:** +```zig +pub const Iterator = struct { + pub fn next(self: *@This()) ?Entry { ... } +}; +``` + +### 3.2 Unnecessary Named Return Variable + +| Detect | Variable assigned then immediately returned | +|--------|---------------------------------------------| +| Risk | None (style) | + +**Wrong:** +```zig +const result = Entry{ .key = k, .value = v }; +return result; +``` + +**Right:** +```zig +return .{ .key = k, .value = v }; +``` + +### 3.3 Old Struct Init Syntax + +| Detect | `TypeName{}` instead of `.{}` with type annotation | +|--------|---------------------------------------------------| +| Risk | None (style) | + +**Wrong:** +```zig +var mutex = Mutex{}; +``` + +**Right:** +```zig +var mutex: Mutex = .{}; +``` + +### 3.4 Redundant Naming + +| Detect | Names like `JsonValue`, `DataManager`, `miscUtils` | +|--------|---------------------------------------------------| +| Risk | None (style) | + +**Wrong:** +```zig +const JsonValue = struct { ... }; +const JsonParser = struct { ... }; +``` + +**Right:** +```zig +const json = struct { + const Value = struct { ... }; + const Parser = struct { ... }; +}; +``` + +### 3.5 Large Struct Passed by Value + +| Detect | Struct >16 bytes passed as `self: T` (read-only) | +|--------|--------------------------------------------------| +| Risk | Unnecessary copy | + +**Wrong:** +```zig +pub fn format(uri: Uri, writer: *Writer) Writer.Error!void { ... } +``` + +**Right:** +```zig +pub fn format(uri: *const Uri, writer: *Writer) Writer.Error!void { ... } +``` + +Also applies to payload captures: +```zig +// Wrong: if (m.target) |target| { ... } +// Right: if (m.target) |*target| { ... } +``` + +### 3.6 errdefer Error Capture for Debugging + +| Detect | Complex init/setup function without `errdefer` diagnostics | +|--------|-----------------------------------------------------------| +| Risk | Hard to diagnose failures | + +In complex initialization functions, use `errdefer |err|` to capture and log the error for debugging context: + +**Basic:** +```zig +pub fn init(gpa: Allocator, io: std.Io, path: []const u8) !Self { + const file = std.Io.Dir.cwd().openFile(io, path, .{}) catch |err| return err; + // No context if later steps fail + ... +} +``` + +**Better:** +```zig +pub fn init(gpa: Allocator, io: std.Io, path: []const u8) !Self { + errdefer |err| { + std.log.err("init failed for '{s}': {}", .{ path, err }); + } + const file = try std.Io.Dir.cwd().openFile(io, path, .{}); + errdefer file.close(io); + ... +} +``` + +### 3.7 Style Guide Violations (not caught by zig fmt) + +| Element | Convention | Example | +|---------|-----------|---------| +| Types | `TitleCase` | `XmlParser`, `HashMap` | +| Namespace structs | `snake_case` | `std.json`, `std.mem` | +| Functions | `camelCase` | `readU32Be`, `parseJson` | +| Type-returning functions | `TitleCase` | `ArrayList`, `HashMap` | +| Variables/constants | `snake_case` | `const_name`, `file_path` | + +**Acronyms:** Treat as regular words (`XmlParser` not `XMLParser`). + +**File naming:** +- Type file: `TitleCase.zig` (e.g., `ArrayList.zig`) +- Namespace file: `snake_case.zig` (e.g., `mem.zig`) + +### 3.8 Import Organization + +| Detect | Imports not grouped or inconsistently ordered | +|--------|-----------------------------------------------| +| Risk | None (style) | + +**Right:** +```zig +const std = @import("std"); +const builtin = @import("builtin"); + +const json = @import("json"); // Third-party + +const MyModule = @import("my_module.zig"); // Local +``` + +### 3.9 Allocator Design + +| Detect | Global allocator variable | +|--------|--------------------------| +| Risk | Untestable code | + +**Wrong:** +```zig +var global_allocator: Allocator = undefined; +pub fn process() ![]u8 { + return global_allocator.alloc(u8, 100); +} +``` + +**Right:** +```zig +pub fn process(allocator: Allocator) ![]u8 { + return allocator.alloc(u8, 100); +} +``` + +### 3.10 Stack vs Heap Allocation + +| Detect | Heap allocation for comptime-known bounded size | +|--------|------------------------------------------------| +| Risk | Unnecessary allocation | + +**Wrong:** +```zig +fn formatVersion(allocator: Allocator, major: u32, minor: u32) ![]u8 { + return std.fmt.allocPrint(allocator, "{d}.{d}", .{ major, minor }); +} +``` + +**Right (caller provides buffer):** +```zig +fn formatVersion(buf: []u8, major: u32, minor: u32) []u8 { + return std.fmt.bufPrint(buf, "{d}.{d}", .{ major, minor }) catch unreachable; +} + +// Call site — buffer outlives the returned slice +var buf: [32]u8 = undefined; +const version = formatVersion(&buf, 1, 2); +``` + +### 3.11 Comptime Optimization + +| Detect | Runtime loop for constant lookup | +|--------|----------------------------------| +| Risk | Suboptimal performance | + +**Wrong:** +```zig +fn isVowel(c: u8) bool { + for ("aeiouAEIOU") |v| { + if (c == v) return true; + } + return false; +} +``` + +**Right:** +```zig +fn isVowel(c: u8) bool { + const table = comptime blk: { + var t: [256]bool = .{false} ** 256; + for ("aeiouAEIOU") |v| t[v] = true; + break :blk t; + }; + return table[c]; +} +``` + +### 3.12 SIMD Opportunities + +| Detect | Scalar loop on arrays where SIMD helps | +|--------|----------------------------------------| +| Risk | Suboptimal performance | + +**Scalar:** +```zig +fn addArrays(a: []const f32, b: []const f32, result: []f32) void { + for (a, b, result) |av, bv, *rv| { + rv.* = av + bv; + } +} +``` + +**SIMD:** +```zig +fn addArrays(a: []const f32, b: []const f32, result: []f32) void { + const Vec = @Vector(8, f32); + var i: usize = 0; + while (i + 8 <= a.len) : (i += 8) { + const va: Vec = a[i..][0..8].*; + const vb: Vec = b[i..][0..8].*; + result[i..][0..8].* = va + vb; + } + while (i < a.len) : (i += 1) { + result[i] = a[i] + b[i]; + } +} +``` + +Only suggest for hot loops with measurable impact. + +### 3.13 Struct Layout + +| Detect | `packed` for non-binary data, or poor extern field ordering | +|--------|-------------------------------------------------------------| +| Risk | Slower access or wasted padding | + +**Use:** +- Regular `struct` for normal use +- `packed struct` for binary protocols/hardware +- `extern struct` for C ABI + +**Extern field order:** Order by size descending to minimize padding. + +### 3.14 Testing Best Practices + +| Detect | Production allocator in tests, or wrong assertion type | +|--------|--------------------------------------------------------| +| Risk | Missed leaks, false passes | + +**Allocator:** +```zig +// Wrong: var gpa: std.heap.DebugAllocator(.{}) = .init; +// Right: use std.testing.allocator +``` + +**Assertions:** +```zig +// Strings: use expectEqualStrings (not expectEqual) +// Slices: use expectEqualSlices (not expectEqual) +``` + +### 3.15 Documentation + +| Detect | Public API without doc comments | +|--------|--------------------------------| +| Risk | Poor discoverability | + +Focus on: what it does, error conditions, ownership, usage example. + +### 3.16 Stateless Context Pattern + +| Detect | `self` parameter that's never used | +|--------|-----------------------------------| +| Risk | None (style) | + +**Wrong:** +```zig +pub fn eql(self: @This(), a: []const u8, b: []const u8) bool { + _ = self; + return std.mem.eql(u8, a, b); +} +``` + +**Right:** +```zig +pub fn eql(_: @This(), a: []const u8, b: []const u8) bool { + return std.mem.eql(u8, a, b); +} +``` + +--- + +## File References + +- [SKILL.md](../SKILL.md) - Breaking changes overview +- [patterns.md](patterns.md) - Best practices patterns +- [std-io.md](std-io.md) - New I/O API +- [std-allocators.md](std-allocators.md) - Allocator naming conventions +- [style-guide.md](style-guide.md) - Style conventions diff --git a/references/comptime.md b/references/comptime.md new file mode 100644 index 0000000..8256866 --- /dev/null +++ b/references/comptime.md @@ -0,0 +1,462 @@ +# Comptime Reference + +Zig's comptime system enables metaprogramming through partial evaluation and type reflection. This reference covers comptime fundamentals, type reflection, and common techniques. + +## Table of Contents + +- [Fundamentals](#fundamentals) +- [Type Reflection](#type-reflection) +- [Loop Variants](#loop-variants) +- [Branch Elimination](#branch-elimination) +- [Type Generation](#type-generation) +- [Limitations](#limitations) + +--- + +## Fundamentals + +### Comptime Parameters + +Values that must be known at compile time. Types are always comptime. + +```zig +fn max(comptime T: type, a: T, b: T) T { + return if (a > b) a else b; +} + +const result = max(i32, 5, 10); // T=i32 known at compile time +``` + +A `comptime` parameter means: +- At the callsite, the value must be known at compile time +- In the function definition, the value is comptime-known + +### Comptime Variables + +Variables whose loads/stores happen at compile time. + +```zig +comptime var i: usize = 0; +inline while (i < 3) : (i += 1) { + // i is comptime-known each iteration +} +``` + +### Comptime Blocks + +Force expression evaluation at compile time. + +```zig +const primes = comptime blk: { + var result: [10]u32 = undefined; + // ... compute primes ... + break :blk result; +}; + +comptime { + // All code here runs at compile time + if (@sizeOf(MyStruct) > 64) @compileError("too large"); +} +``` + +### Container-Level Comptime + +Top-level declarations are implicitly comptime. + +```zig +// These are computed at compile time automatically +const lookup_table = generateTable(); +const config = parseConfig(@embedFile("config.json")); +``` + +--- + +## Type Reflection + +### Builtins + +| Builtin | Purpose | +|---------|---------| +| `@typeInfo(T)` | Get type metadata as `std.builtin.Type` | +| `@Int` / `@Struct` / `@Union` / `@Enum` / `@Pointer` / `@Fn` / `@Tuple` | Zig 0.16 type-construction builtins replacing removed `@Type` | +| `@TypeOf(expr)` | Get type of expression | +| `@typeName(T)` | Get type name as `[:0]const u8` | +| `@hasDecl(T, name)` | Check if type has declaration | +| `@hasField(T, name)` | Check if type has field | +| `@field(value, name)` | Access field by comptime-known name | + +### Zig 0.16 Type Construction + +`@Type` is removed in Zig 0.16. Continue using `@typeInfo` for reflection, but construct new types with the specific builtin for the container or scalar type you need: + +```zig +const Id = @Int(.unsigned, 32); +const Pair = @Tuple(&.{ []const u8, Id }); +``` + +For generated structs/unions/enums, use `@Struct`, `@Union`, and `@Enum` with separate arrays of field names, field types/values, and field attributes. Prefer plain Zig syntax when the type can be written directly. + +Zig 0.16 also changed type resolution: + +- Field analysis is lazier, so namespace-like types can often exist without resolving all fields. +- Some dependency-loop diagnostics changed and should be evaluated in context rather than dismissed. +- Pointers to comptime-only types may exist at runtime, but runtime dereference remains invalid. +- Explicitly aligned pointer types are distinct from naturally aligned pointer types, though they often coerce. +- Zero-bit tuple fields are no longer implicitly marked `comptime` in type info. + +### Accessing Type Info for Keywords + +Use `@"keyword"` syntax because `union`, `struct`, `enum` are reserved: + +```zig +const union_info = @typeInfo(MyUnion).@"union"; +const struct_info = @typeInfo(MyStruct).@"struct"; +const enum_info = @typeInfo(MyEnum).@"enum"; +const fn_info = @typeInfo(@TypeOf(myFn)).@"fn"; +``` + +### Common Type Info Fields + +**Struct:** +```zig +const info = @typeInfo(MyStruct).@"struct"; +// info.fields: []const StructField +// info.decls: []const Declaration +// info.is_tuple: bool +``` + +**Union:** +```zig +const info = @typeInfo(MyUnion).@"union"; +// info.tag_type: ?type (null if untagged) +// info.fields: []const UnionField +// info.layout: .auto, .@"extern", .@"packed" +``` + +**Enum:** +```zig +const info = @typeInfo(MyEnum).@"enum"; +// info.tag_type: type (backing integer) +// info.fields: []const EnumField +// info.is_exhaustive: bool +``` + +### Creating Union Values with Comptime Tag + +Use `@unionInit` when the tag is comptime-known: + +```zig +const Action = union(enum) { + move: struct { x: i32, y: i32 }, + jump, + attack: u32, +}; + +// Create union with comptime-known field name +const action = @unionInit(Action, "move", .{ .x = 10, .y = 20 }); +``` + +--- + +## Loop Variants + +### comptime for + +Full compile-time evaluation. Can use `break` to return values. Cannot reference runtime values. + +```zig +// Return value from comptime loop +fn hasField(comptime T: type, comptime name: []const u8) bool { + const fields = @typeInfo(T).@"struct".fields; + return comptime for (fields) |f| { + if (std.mem.eql(u8, f.name, name)) break true; + } else false; +} + +// Computation in comptime block +fn sumComptime(comptime values: []const i32) i32 { + comptime { + var sum: i32 = 0; + for (values) |v| sum += v; + return sum; + } +} +``` + +**Verified in stdlib:** `std/Build.zig:1953` + +### inline for + +Loop unrolling with code generation. Body is duplicated per iteration. Can reference runtime values. Cannot use `break` to return values. + +```zig +fn printFields(value: anytype) void { + const T = @TypeOf(value); + const fields = @typeInfo(T).@"struct".fields; + + // Each iteration generates separate code + inline for (fields) |field| { + const field_value = @field(value, field.name); + std.debug.print("{s} = {any}\n", .{ field.name, field_value }); + } +} + +// Runtime comparison via unrolling +fn eqlAny(comptime T: type, a: T, b: T) bool { + const fields = @typeInfo(T).@"struct".fields; + inline for (fields) |field| { + if (@field(a, field.name) != @field(b, field.name)) { + return false; // Runtime return + } + } + return true; +} +``` + +**Verified in stdlib:** `std/meta.zig:27`, `compiler_rt/fmax.zig:63` + +### Decision Table + +| Need | Use | Reason | +|------|-----|--------| +| Return value from loop | `comptime for` | Only comptime allows `break` with value | +| Access runtime values in body | `inline for` | Comptime can't see runtime | +| Type-level computation only | `comptime for` | Clearer intent, no code gen | +| Generate code per iteration | `inline for` | Each iteration = separate code | +| Normal runtime iteration | regular `for` | No unrolling needed | + +--- + +## Branch Elimination + +Comptime-known conditions eliminate dead branches entirely—no runtime cost. + +### Basic Elimination + +```zig +fn process(comptime T: type, value: T) T { + if (T == bool) { + return !value; // Only exists for bool + } else { + return value + 1; // Only exists for integers + } +} +``` + +### Platform-Specific Code + +```zig +const builtin = @import("builtin"); +const native_endian = builtin.cpu.arch.endian(); + +pub fn readIntBig(comptime T: type, bytes: []const u8) T { + const value: T = @bitCast(bytes[0..@sizeOf(T)].*); + if (comptime native_endian == .big) { + return value; + } else { + return @byteSwap(value); + } +} +``` + +### Propagating Across Functions + +Use `inline fn` to propagate comptime conditions to call sites: + +```zig +// WITHOUT inline: branch exists at runtime +fn maybeLog(comptime enabled: bool, msg: []const u8) void { + if (enabled) std.debug.print("{s}\n", .{msg}); +} + +// WITH inline: branch eliminated at each call site +inline fn maybeLogInline(comptime enabled: bool, msg: []const u8) void { + if (comptime enabled) std.debug.print("{s}\n", .{msg}); +} + +pub fn example() void { + maybeLogInline(false, "debug"); // Entire call eliminated + maybeLogInline(true, "important"); // Only this generates code +} +``` + +**Verified in stdlib:** `std/math.zig:708`, `std/log.zig:122` + +--- + +## Type Generation + +### Returning Types from Functions + +```zig +fn Pair(comptime A: type, comptime B: type) type { + return struct { + first: A, + second: B, + + const Self = @This(); + + pub fn swap(self: Self) Pair(B, A) { + return .{ .first = self.second, .second = self.first }; + } + }; +} + +const IntStr = Pair(i32, []const u8); +var p: IntStr = .{ .first = 42, .second = "hello" }; +``` + +### Generating Union Subsets + +Generate a subset union type from a larger union: + +**Zig 0.16 note:** the following pattern is historical because it uses removed `@Type`. In new code, keep the reflection idea but build the resulting type with `@Union` and related 0.16 type-construction builtins. + +```zig +pub fn Subset(comptime T: type, comptime fields: []const std.meta.FieldEnum(T)) type { + const source_info = @typeInfo(T).@"union"; + var new_fields: [fields.len]std.builtin.Type.UnionField = undefined; + + for (fields, 0..) |field_enum, i| { + const field_name = @tagName(field_enum); + for (source_info.fields) |source_field| { + if (std.mem.eql(u8, source_field.name, field_name)) { + new_fields[i] = source_field; + break; + } + } + } + + return @Type(.{ + .@"union" = .{ + .layout = source_info.layout, + .tag_type = std.meta.FieldEnum(@Type(.{ + .@"union" = .{ + .layout = source_info.layout, + .tag_type = null, + .fields = &new_fields, + .decls = &.{}, + }, + })), + .fields = &new_fields, + .decls = &.{}, + }, + }); +} +``` + +### Converting Between Union Types + +Use `inline else` to capture tag at comptime: + +```zig +/// Convert subset to full union type. +pub fn toFull(comptime Full: type, subset: anytype) Full { + return switch (subset) { + inline else => |payload, tag| @unionInit(Full, @tagName(tag), payload), + }; +} + +/// Try to narrow full union to subset type. +pub fn toSubset(comptime Subset: type, full: anytype) ?Subset { + const subset_fields = @typeInfo(Subset).@"union".fields; + return switch (full) { + inline else => |payload, tag| { + inline for (subset_fields) |sf| { + if (std.mem.eql(u8, sf.name, @tagName(tag))) { + return @unionInit(Subset, sf.name, payload); + } + } + return null; + }, + }; +} +``` + +**Verified in stdlib:** `std/meta.zig`, `std/json/static.zig:299-302` + +--- + +## Limitations + +Zig's comptime is deliberately constrained for cross-compilation safety and code clarity. + +### No Host Architecture Detection + +```zig +// This reflects TARGET, not host +const is_64bit = comptime @sizeOf(usize) == 8; + +// Use build.zig for host detection +// build.zig runs as a program and can query host +``` + +### No String-to-Code Evaluation + +```zig +// NOT POSSIBLE +const code = "x + y"; +const result = @eval(code); // No such builtin + +// Alternative: Parse strings to data structures at comptime +const query = comptime sql.parse("SELECT * FROM users"); +``` + +### No Runtime Type Information + +```zig +// This works - type known at comptime +fn getTypeName(value: anytype) []const u8 { + return @typeName(@TypeOf(value)); +} + +// NOT POSSIBLE - can't turn runtime string into type +fn typeFromName(name: []const u8) type { ... } +``` + +### No I/O at Comptime + +```zig +// NOT POSSIBLE +const config = comptime std.fs.cwd().readFile("config.json"); + +// Alternatives: +const config = @embedFile("config.json"); // Static embedding +// Or use build.zig which runs as a normal program +``` + +### No Dynamic Method Injection + +```zig +// This works - methods defined in type +fn Pair(comptime A: type, comptime B: type) type { + return struct { + first: A, + second: B, + pub fn swap(self: @This()) Pair(B, A) { ... } + }; +} + +// NOT POSSIBLE - can't add methods to existing types +fn addMethod(comptime T: type, comptime name: []const u8, impl: anytype) type { ... } +``` + +### Summary Table + +| Want to do | Comptime? | Alternative | +|------------|-----------|-------------| +| Type reflection | Yes | `@typeInfo`, `@TypeOf` | +| Generate types | Yes | Return struct from function | +| Add methods to types | No | Define in type definition | +| Read files | No | `@embedFile` or build.zig | +| Syscalls | No | build.zig runs as program | +| Parse strings to code | No | Parse to data structures | +| Host detection | No | Build system queries | + +### Design Rationale + +These constraints ensure: +1. **Cross-compilation works** - comptime sees target, not host +2. **Code is readable** - no hidden code generation +3. **Builds are reproducible** - no I/O side effects +4. **All API is visible** - no dynamic method injection diff --git a/references/language.md b/references/language.md new file mode 100644 index 0000000..8e7da21 --- /dev/null +++ b/references/language.md @@ -0,0 +1,745 @@ +# Zig Language Basics Reference (Zig 0.16.0) + +Core language features, control flow, and type system fundamentals. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Key 0.16 language changes: + +- `@Type` removed; use specific type-construction builtins. +- `@cImport` is deprecated as the long-term C translation API; prefer build-system translation. +- Runtime vector indexes are forbidden. +- Vectors and arrays no longer support in-memory coercion. +- Returning the address of an expired local is diagnosed. +- Packed structs/unions cannot contain pointers. +- Packed unions and extern packed/enum layouts require more explicit backing types. +- Type resolution is lazier in some cases and stricter in others; investigate new dependency-loop diagnostics. + +## Table of Contents +- [Types](#types) +- [Control Flow](#control-flow) +- [Error Handling](#error-handling) +- [Optionals](#optionals) +- [Structs](#structs) +- [Enums](#enums) +- [Unions](#unions) +- [Pointers and Slices](#pointers-and-slices) +- [Comptime](#comptime) +- [Functions](#functions) + +## Types + +### Primitive Types +```zig +// Integers (signed and unsigned, any bit width 1-65535) +i8, i16, i32, i64, i128, isize // signed +u8, u16, u32, u64, u128, usize // unsigned +i7, u24, i53 // arbitrary widths + +// Floats +f16, f32, f64, f80, f128 + +// Other +bool // true or false +void // zero-size type +noreturn // function never returns +type // type of types (comptime only) +anyopaque // type-erased pointer target +comptime_int // arbitrary precision integer (comptime only) +comptime_float // arbitrary precision float (comptime only) + +// C interop types +c_char, c_short, c_int, c_long, c_longlong +c_ushort, c_uint, c_ulong, c_ulonglong +c_longdouble +``` + +### Type Coercion +```zig +// Implicit coercions (safe, automatic) +const a: u16 = 42; // comptime_int → u16 +const b: i32 = a; // u16 → i32 (widening) +const c: f64 = 3.14; // comptime_float → f64 +const d: []const u8 = "hello"; // *const [5:0]u8 → []const u8 +const e: ?i32 = 5; // i32 → ?i32 + +// @as for explicit safe coercion +const x = @as(u32, 100); + +// Casts for unsafe/reinterpret conversions +const y: u8 = @intCast(big_value); // may panic if value doesn't fit +const z: u32 = @bitCast(float_val); // reinterpret bits +``` + +### Arrays +```zig +// Fixed-size arrays +const arr: [5]u8 = .{ 1, 2, 3, 4, 5 }; +const arr2 = [_]u8{ 1, 2, 3 }; // infer length +const zeros = [_]u8{0} ** 100; // repeat pattern + +// Sentinel-terminated arrays +const str: [5:0]u8 = "hello".*; // null-terminated +const arr: [3:255]u8 = .{ 1, 2, 3 }; // 255-terminated + +// Access +const elem = arr[2]; +const len = arr.len; + +// Iteration +for (arr) |elem| { ... } +for (arr, 0..) |elem, i| { ... } // with index +for (&arr) |*elem| { elem.* = 0; } // mutable +``` + +### Tuples +```zig +const tuple = .{ 42, "hello", true }; +const first = tuple[0]; // 42 +const len = tuple.len; // 3 + +// Destructuring +const a, const b, const c = tuple; +``` + +## Control Flow + +### if +```zig +// Basic +if (condition) { + // ... +} else if (other) { + // ... +} else { + // ... +} + +// Expression form +const value = if (condition) x else y; + +// With optionals +if (optional_value) |unwrapped| { + // unwrapped is non-null +} else { + // was null +} + +// With error unions +if (error_union) |value| { + // success +} else |err| { + // handle err +} +``` + +### switch +```zig +const result = switch (value) { + 1 => "one", + 2, 3 => "two or three", + 4...10 => "four to ten", + else => "other", +}; + +// Capture +switch (tagged_union) { + .variant => |payload| { ... }, + .other => |*ptr| { ptr.* = new_value; }, // mutable capture +} + +// Comptime switch on types +switch (@typeInfo(T)) { + .int => |info| { ... }, + .float => { ... }, + else => @compileError("unsupported type"), +} +``` + +### Labeled switch (0.14.0+) - State Machines +```zig +state: switch (initial_state) { + .idle => { + continue :state .running; // transition + }, + .running => { + if (done) break :state result; // exit with value + continue :state .running; // loop + }, + .error => return error.Failed, +} +``` + +### Non-exhaustive enum switch (0.15.x) +```zig +switch (non_exhaustive_enum) { + .known_a => {}, + .known_b => {}, + else => {}, // other named tags + _ => {}, // unnamed integer values +} +``` + +### while +```zig +// Basic +while (condition) { ... } + +// With else (runs if condition was never true or on break) +while (condition) { ... } else { ... } + +// With continue expression +var i: usize = 0; +while (i < 10) : (i += 1) { ... } + +// With optional +while (iterator.next()) |item| { ... } + +// With error union +while (reader.readByte()) |byte| { + ... +} else |err| { + if (err != error.EndOfStream) return err; +} + +// Infinite loop +while (true) { ... } +``` + +### for +```zig +// Iterate slice/array +for (items) |item| { ... } + +// With index +for (items, 0..) |item, i| { ... } + +// Multiple sequences (must have same length) +for (a, b, c) |x, y, z| { ... } + +// Mutable iteration +for (&items) |*item| { item.* = new_value; } + +// Range (comptime only for runtime, but works in comptime blocks) +inline for (0..10) |i| { ... } +``` + +### Labels and Control +```zig +// Labeled blocks +const result = blk: { + if (condition) break :blk value; + break :blk other_value; +}; + +// Labeled loops +outer: for (rows) |row| { + for (row) |cell| { + if (cell == target) break :outer; + } +} + +// continue with label +outer: for (items) |item| { + for (sub_items) |sub| { + if (skip) continue :outer; + } +} +``` + +### defer / errdefer +```zig +// Always runs when scope exits +fn example() void { + const resource = acquire(); + defer release(resource); // runs on return + // use resource... +} + +// Only runs on error return +fn example() !void { + const ptr = try allocate(); + errdefer free(ptr); // runs only if function returns error + try doSomething(ptr); + return ptr; // errdefer does NOT run +} + +// errdefer with capture +fn example() !void { + errdefer |err| { + log.err("Failed with: {}", .{err}); + }; + try riskyOperation(); +} +``` + +## Error Handling + +### Error Sets +```zig +// Define error set +const FileError = error{ + NotFound, + AccessDenied, + OutOfMemory, +}; + +// Inferred error set (use sparingly) +fn foo() !void { ... } // error set inferred from body + +// Merge error sets +const AllErrors = FileError || NetworkError; + +// anyerror - global error set (avoid when possible) +fn bar() anyerror!void { ... } +``` + +### Error Unions +```zig +// Error union type: ErrorSet!PayloadType +fn parse(s: []const u8) ParseError!u32 { ... } +fn read() ![]u8 { ... } // inferred error set + +// Return errors +return error.InvalidInput; + +// Return success +return value; +``` + +### Handling Errors +```zig +// try - propagate error, unwrap on success +const value = try mayFail(); + +// catch - provide default on error +const value = mayFail() catch 0; +const value = mayFail() catch |err| { + log.err("failed: {}", .{err}); + return default; +}; + +// catch unreachable - assert no error (crashes if error) +const value = mayFail() catch unreachable; + +// if with error union +if (mayFail()) |value| { + // success +} else |err| { + // handle error +} + +// switch on specific errors +mayFail() catch |err| switch (err) { + error.NotFound => return null, + error.AccessDenied => return error.PermissionDenied, + else => return err, +}; +``` + +## Optionals + +### Optional Types +```zig +// Optional type: ?T +var maybe: ?i32 = null; +maybe = 42; + +// Check for null +if (maybe != null) { ... } +if (maybe == null) { ... } +``` + +### Unwrapping +```zig +// orelse - default value +const value = maybe orelse 0; +const value = maybe orelse return error.Missing; +const value = maybe orelse unreachable; // assert non-null + +// .? - assert and unwrap (crashes on null) +const value = maybe.?; + +// if unwrap +if (maybe) |value| { + // value is non-null +} else { + // was null +} + +// while unwrap +while (iterator.next()) |item| { ... } +``` + +### Optional Pointers +```zig +// ?*T has null representation as 0 (same size as *T) +var ptr: ?*Node = null; +ptr = &node; + +if (ptr) |p| { + p.*.data = 42; +} +``` + +## Structs + +### Basic Structs +```zig +const Point = struct { + x: f32, + y: f32, + + // Methods + pub fn distance(self: Point, other: Point) f32 { + const dx = self.x - other.x; + const dy = self.y - other.y; + return @sqrt(dx * dx + dy * dy); + } + + // Static method + pub fn origin() Point { + return .{ .x = 0, .y = 0 }; + } +}; + +// Usage +const p = Point{ .x = 1.0, .y = 2.0 }; +const p2: Point = .{ .x = 3.0, .y = 4.0 }; // type inferred +const dist = p.distance(p2); +``` + +### Default Values +```zig +const Config = struct { + name: []const u8, + port: u16 = 8080, // default value + debug: bool = false, +}; + +const cfg: Config = .{ .name = "server" }; // uses defaults +``` + +### @This() for Self-Reference +```zig +const Node = struct { + const Self = @This(); + next: ?*Self = null, + data: i32, + + pub fn append(self: *Self, node: *Self) void { + self.next = node; + } +}; +``` + +### Packed Structs +```zig +const Flags = packed struct { + enabled: bool, // 1 bit + mode: u2, // 2 bits + _reserved: u5, // 5 bits +}; // Total: 1 byte + +const flags: Flags = @bitCast(@as(u8, 0b10100001)); +``` + +### Extern Structs (C ABI) +```zig +const CStruct = extern struct { + x: c_int, + y: c_int, +}; +``` + +## Enums + +### Basic Enums +```zig +const Color = enum { + red, + green, + blue, +}; + +const c: Color = .red; + +// Switch (must be exhaustive) +switch (c) { + .red => {}, + .green => {}, + .blue => {}, +} +``` + +### Enums with Values +```zig +const HttpStatus = enum(u16) { + ok = 200, + not_found = 404, + internal_error = 500, + _, // non-exhaustive marker +}; + +const code: u16 = @intFromEnum(HttpStatus.ok); // 200 +const status: HttpStatus = @enumFromInt(404); // .not_found +``` + +### Enum Methods +```zig +const Direction = enum { + north, + south, + east, + west, + + pub fn opposite(self: Direction) Direction { + return switch (self) { + .north => .south, + .south => .north, + .east => .west, + .west => .east, + }; + } +}; +``` + +## Unions + +### Tagged Unions +```zig +const Value = union(enum) { + int: i64, + float: f64, + string: []const u8, + none, // void payload + + pub fn isNumeric(self: Value) bool { + return switch (self) { + .int, .float => true, + else => false, + }; + } +}; + +const v: Value = .{ .int = 42 }; + +switch (v) { + .int => |n| std.debug.print("{}", .{n}), + .float => |f| std.debug.print("{}", .{f}), + .string => |s| std.debug.print("{s}", .{s}), + .none => {}, +} +``` + +### Bare Unions (no tag) +```zig +const Bare = union { + int: i32, + float: f32, +}; +// Must track active field manually - unsafe +``` + +### Extern Unions (C ABI) +```zig +const CUnion = extern union { + as_int: c_int, + as_float: f32, +}; +``` + +## Pointers and Slices + +### Pointer Types +```zig +*T // single-item pointer +*const T // pointer to const +[*]T // many-item pointer (unknown length) +[*:0]T // null-terminated many-item pointer +?*T // optional pointer + +// Alignment +*align(16) T // pointer with explicit alignment +``` + +### Slices +```zig +[]T // slice (pointer + length) +[]const T // slice to const data +[:0]T // null-terminated slice + +// Create slice from array +const arr = [_]u8{ 1, 2, 3, 4, 5 }; +const slice: []const u8 = &arr; +const sub: []const u8 = arr[1..4]; // {2, 3, 4} + +// Slice operations +const len = slice.len; +const ptr = slice.ptr; // [*]const u8 +const elem = slice[2]; +``` + +### Pointer Arithmetic +```zig +// Many-item pointers support arithmetic +const ptr: [*]u8 = buffer.ptr; +const next = ptr + 1; +const offset = ptr + n; + +// Single-item pointers do NOT support arithmetic +// Use slicing instead: +const slice = ptr[0..n]; +``` + +### Sentinel-Terminated +```zig +// Null-terminated string +const str: [:0]const u8 = "hello"; +const c_str: [*:0]const u8 = str.ptr; + +// Custom sentinel +const arr: [3:255]u8 = .{ 1, 2, 3 }; // followed by 255 +``` + +## Comptime + +### Comptime Variables +```zig +comptime var count: u32 = 0; + +// Comptime block +comptime { + count += 1; +} + +// Comptime parameter +fn repeat(comptime n: usize, value: u8) [n]u8 { + return [_]u8{value} ** n; +} +``` + +### Comptime Functions +```zig +fn factorial(comptime n: u32) u32 { + if (n == 0) return 1; + return n * factorial(n - 1); +} + +const result = factorial(5); // computed at compile time +``` + +### Type as First-Class Value +```zig +fn Container(comptime T: type) type { + return struct { + items: []T, + + pub fn get(self: @This(), i: usize) T { + return self.items[i]; + } + }; +} + +const IntContainer = Container(i32); +``` + +### @typeInfo +```zig +fn isInteger(comptime T: type) bool { + return @typeInfo(T) == .int; +} + +fn fieldNames(comptime T: type) []const []const u8 { + const info = @typeInfo(T); + if (info != .@"struct") @compileError("expected struct"); + + var names: [info.@"struct".fields.len][]const u8 = undefined; + for (info.@"struct".fields, 0..) |field, i| { + names[i] = field.name; + } + return &names; +} +``` + +### inline for/while +```zig +// Unroll loop at compile time +inline for (0..4) |i| { + array[i] = computeValue(i); +} + +// Generate code for each field +inline for (std.meta.fields(T)) |field| { + @field(value, field.name) = default; +} +``` + +## Functions + +### Function Basics +```zig +fn add(a: i32, b: i32) i32 { + return a + b; +} + +// With error +fn parse(s: []const u8) !i32 { ... } + +// Void return +fn log(msg: []const u8) void { ... } + +// Noreturn +fn abort() noreturn { + @panic("aborted"); +} +``` + +### Generic Functions +```zig +fn max(comptime T: type, a: T, b: T) T { + return if (a > b) a else b; +} + +// Using anytype +fn print(value: anytype) void { + const T = @TypeOf(value); + // ... +} +``` + +### Function Pointers +```zig +const BinaryOp = *const fn (i32, i32) i32; + +fn apply(op: BinaryOp, a: i32, b: i32) i32 { + return op(a, b); +} +``` + +### Calling Conventions +```zig +fn cFunc() callconv(.c) void { ... } +fn nakedFunc() callconv(.naked) noreturn { ... } +fn inlineFunc() callconv(.@"inline") i32 { ... } +``` + +### Export/Extern +```zig +// Export to C +export fn my_function() void { ... } + +// Import from C +extern "c" fn printf(fmt: [*:0]const u8, ...) c_int; + +// Link with library +extern "SDL2" fn SDL_Init(flags: u32) c_int; +``` + +### Inline Functions +```zig +inline fn fastAdd(a: i32, b: i32) i32 { + return a + b; +} +// Forces inlining - compile error if impossible +``` diff --git a/references/patterns.md b/references/patterns.md new file mode 100644 index 0000000..4557b0c --- /dev/null +++ b/references/patterns.md @@ -0,0 +1,1455 @@ +# Zig Patterns Reference (Zig 0.16.0) + +Comprehensive patterns for writing idiomatic Zig code. Zig 0.16.0 changes I/O ownership, process APIs, time, randomness, synchronization, and type construction. When older examples conflict with the 0.16 release notes, follow [Zig 0.16 Release Notes Migration Reference](zig-0.16-release-notes.md). + +## Table of Contents + +### Quick Patterns +- [Memory and Allocators](#memory-and-allocators) +- [File I/O (0.16.x)](#file-io-016x) +- [HTTP Client (0.16.x)](#http-client-016x) +- [JSON](#json) +- [Testing](#testing) +- [Build System Patterns](#build-system-patterns) + +### Idiomatic Code Patterns +- [I. Syntax Patterns](#i-syntax-patterns) + - [Closure Pattern](#closure-pattern) + - [Context Pattern](#context-pattern) + - [Pointer Size Type Selection](#pointer-size-type-selection) + - [Default Arguments via Options Struct](#default-arguments-via-options-struct) + - [Side Computation Block](#side-computation-block) + - [Destructuring Assignment](#destructuring-assignment) + - [Hashed Mappings Storage](#hashed-mappings-storage) + - [Module-wide Overridable Options](#module-wide-overridable-options) + - [Self-referential Type Alias](#self-referential-type-alias) + - [Variable Struct Initialization](#variable-struct-initialization) + - [Return Value Struct Initialization](#return-value-struct-initialization) +- [II. Polymorphism Patterns](#ii-polymorphism-patterns) + - [Duck Typing](#duck-typing) + - [Generic Type](#generic-type) + - [Generic Function](#generic-function) + - [Basic Type Formatting](#basic-type-formatting) + - [Custom Type Formatting](#custom-type-formatting) + - [Custom Type JSON](#custom-type-json) + - [Compile-time Implementation Switching](#compile-time-implementation-switching) + - [Dynamic Dispatch (Fat Pointer)](#dynamic-dispatch-fat-pointer) + - [Static Dispatch (Tagged Union)](#static-dispatch-tagged-union-with-inline-switch) +- [III. Safety Patterns](#iii-safety-patterns) + - [Diagnostics](#diagnostics) + - [Index-Based Data Structures](#index-based-data-structures) + - [Error Payloads](#error-payloads) + - [Compile-time Assertion](#compile-time-assertion) + - [Granular Error Handling](#granular-error-handling) + - [Deallocated Memory Poisoning](#deallocated-memory-poisoning) + - [Deferred Resource Deinitialization](#deferred-resource-deinitialization) + - [Error-deferred Resource Deinitialization](#error-deferred-resource-deinitialization) + - [Compile-time Unreachable Switch Prong](#compile-time-unreachable-switch-prong) + - [Compile-time Error Absence Guarantee](#compile-time-error-absence-guarantee) + - [Reserve-First Exception Safety](#reserve-first-exception-safety) +- [IV. Performance Patterns](#iv-performance-patterns) + - [Big Struct Constant Pointer Passing](#big-struct-constant-pointer-passing) + - [Big Struct Constant Pointer Capturing](#big-struct-constant-pointer-capturing) +- [V. Workarounds](#v-workarounds) + - [Inlined Loop with Runtime Logic](#inlined-loop-with-runtime-logic) + +--- + +## Quick Patterns + +### Memory and Allocators + +> **Naming convention:** Name allocators by their memory contract (`gpa`, `arena`, `scratch`) not generically as `allocator`. See [Allocator Naming Conventions](std-allocators.md#allocator-naming-conventions) for details. + +#### Allocator Setup +```zig +// Debug allocator (development - detects leaks, use-after-free) +// Note: GeneralPurposeAllocator is now an alias for DebugAllocator +var gpa: std.heap.DebugAllocator(.{}) = .init; +defer _ = gpa.deinit(); +const allocator = gpa.allocator(); + +// Arena (batch operations - free all at once) +var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator); +defer arena.deinit(); +const allocator = arena.allocator(); + +// Fixed buffer (no heap, stack allocation) +var buffer: [4096]u8 = undefined; +var fba = std.heap.FixedBufferAllocator.init(&buffer); +const allocator = fba.allocator(); +``` + +#### Allocation Patterns +```zig +// Single item +const ptr = try allocator.create(T); +defer allocator.destroy(ptr); + +// Slice +const slice = try allocator.alloc(u8, 100); +defer allocator.free(slice); + +// Duplicate +const copy = try allocator.dupe(u8, source); +defer allocator.free(copy); +``` + +### File I/O (0.16.x) + +#### Reading Files +```zig +const file = try std.Io.Dir.cwd().openFile(io, "data.txt", .{}); +defer file.close(io); + +var buf: [4096]u8 = undefined; +var reader = file.reader(io, &buf); +const r = &reader.interface; + +// Line by line (takeDelimiter returns null at EOF) +while (try r.takeDelimiter('\n')) |line| { + // process line (doesn't include '\n') +} +``` + +#### Writing Files +```zig +const file = try std.Io.Dir.cwd().createFile(io, "out.txt", .{}); +defer file.close(io); + +var buf: [4096]u8 = undefined; +var writer = file.writer(io, &buf); +const w = &writer.interface; + +try w.print("Hello {s}\n", .{"world"}); +try w.flush(); +``` + +#### Stdout/Stderr +```zig +var stdout_buf: [4096]u8 = undefined; +var stdout_writer = std.Io.File.stdout().writer(io, &stdout_buf); +const stdout = &stdout_writer.interface; + +try stdout.print("Output\n", .{}); +try stdout.flush(); +``` + +### HTTP Client (0.16.x) + +See [std-http.md](std-http.md) for full documentation including server, WebSocket, and compression. + +```zig +var client: std.http.Client = .{ + .allocator = allocator, + .io = io, +}; +defer client.deinit(); + +var req = try client.request(.GET, .{ + .scheme = "https", + .host = .{ .percent_encoded = "example.com" }, + .path = .{ .percent_encoded = "/" }, +}, .{ + .extra_headers = &.{ + .{ .name = "Authorization", .value = "Bearer token" }, + }, +}); +defer req.deinit(); + +try req.sendBodiless(); + +var redirect_buf: [8192]u8 = undefined; +const response = try req.receiveHead(&redirect_buf); +_ = response; +``` + +### JSON + +#### Parsing +```zig +const Config = struct { + name: []const u8, + count: u32, +}; + +const parsed = try std.json.parseFromSlice(Config, allocator, json_bytes, .{}); +defer parsed.deinit(); +const config = parsed.value; +``` + +#### Stringifying +```zig +const json = try std.json.stringifyAlloc(allocator, config, .{}); +defer allocator.free(json); +``` + +### Testing + +```zig +const std = @import("std"); +const testing = std.testing; + +test "example" { + try testing.expectEqual(4, 2 + 2); + try testing.expectEqualStrings("hello", "hello"); + try testing.expect(condition); +} + +test "with allocator" { + var list: std.ArrayListUnmanaged(u32) = .empty; + defer list.deinit(testing.allocator); + try list.append(testing.allocator, 42); +} +``` + +### Build System Patterns + +#### Basic Executable +```zig +pub fn build(b: *std.Build) void { + const target = b.standardTargetOptions(.{}); + const optimize = b.standardOptimizeOption(.{}); + + const exe = b.addExecutable(.{ + .name = "app", + .root_module = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, + }), + }); + b.installArtifact(exe); + + const run = b.addRunArtifact(exe); + run.step.dependOn(b.getInstallStep()); + b.step("run", "Run the app").dependOn(&run.step); +} +``` + +#### Adding Dependencies +```zig +const dep = b.dependency("pkg_name", .{ .target = target, .optimize = optimize }); +exe.root_module.addImport("pkg_name", dep.module("module_name")); +``` + +#### Library + Executable +```zig +const lib_mod = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = target, + .optimize = optimize, +}); + +const exe = b.addExecutable(.{ + .name = "app", + .root_module = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, + .imports = &.{.{ .name = "mylib", .module = lib_mod }}, + }), +}); +``` + +#### Tests +```zig +const tests = b.addTest(.{ + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = target, + .optimize = optimize, + }), +}); +const run_tests = b.addRunArtifact(tests); +b.step("test", "Run tests").dependOn(&run_tests.step); +``` + +--- + +## Idiomatic Code Patterns + +These patterns are maintained for Zig 0.16 and represent established idioms for writing clean, efficient Zig code. + +### I. Syntax Patterns + +#### Closure Pattern +Use a local struct with `@fieldParentPtr` to capture context when you need a function pointer with associated state. + +```zig +pub fn spawnWg(pool: *Pool, wait_group: *WaitGroup, comptime func: anytype, args: anytype) void { + wait_group.start(); + + const Args = @TypeOf(args); + const Closure = struct { + arguments: Args, + pool: *Pool, + runnable: Runnable = .{ .runFn = runFn }, + wait_group: *WaitGroup, + + fn runFn(runnable: *Runnable, _: ?usize) void { + const closure: *@This() = @alignCast(@fieldParentPtr("runnable", runnable)); + @call(.auto, func, closure.arguments); + closure.wait_group.finish(); + closure.pool.allocator.destroy(closure); + } + }; + + const closure = pool.allocator.create(Closure) catch return; + closure.* = .{ + .arguments = args, + .pool = pool, + .wait_group = wait_group, + }; + pool.run_queue.prepend(&closure.runnable.node); +} +``` + +**When to use:** Thread pools, callbacks, event handlers—anywhere you need a function pointer but also need captured state. + +#### Context Pattern +Parameterize a generic type with *behavior*, not just data types. A context struct bundles related operations that the generic type calls at runtime, allowing callers to customize how the type operates. + +**The principle:** When a generic type needs to perform operations that depend on how the caller wants to use it (comparison, hashing, ordering, formatting), accept a `Context` type parameter. The context provides methods the generic calls internally. + +```zig +// DEFINING a generic type that accepts a context: +pub fn SortedSet(comptime T: type, comptime Context: type) type { + return struct { + items: std.ArrayList(T), + ctx: Context, // Store the context instance + + pub fn contains(self: *@This(), value: T) bool { + for (self.items.items) |item| { + // Call the context's comparison method + if (self.ctx.eql(item, value)) return true; + } + return false; + } + + pub fn insert(self: *@This(), value: T) !void { + if (!self.contains(value)) { + // Use context's lessThan for sorted insertion + const pos = for (self.items.items, 0..) |item, i| { + if (self.ctx.lessThan(value, item)) break i; + } else self.items.items.len; + try self.items.insert(pos, value); + } + } + }; +} +``` + +**Stateless context** - when behavior doesn't need configuration, `self` is unused: +```zig +// Case-sensitive string comparison (no state needed) +const CaseSensitive = struct { + pub fn eql(_: @This(), a: []const u8, b: []const u8) bool { + return std.mem.eql(u8, a, b); + } + pub fn lessThan(_: @This(), a: []const u8, b: []const u8) bool { + return std.mem.lessThan(u8, a, b); + } +}; + +var set: SortedSet([]const u8, CaseSensitive) = .{ + .items = .empty, + .ctx = .{}, // No state to initialize +}; +``` + +**Stateful context** - when behavior needs configuration: +```zig +// Floating-point comparison with configurable tolerance +const ApproxEql = struct { + tolerance: f64, + + pub fn eql(self: @This(), a: f64, b: f64) bool { + return @abs(a - b) <= self.tolerance; + } + pub fn lessThan(self: @This(), a: f64, b: f64) bool { + return a < b - self.tolerance; + } +}; + +var precise: SortedSet(f64, ApproxEql) = .{ + .items = .empty, + .ctx = .{ .tolerance = 0.0001 }, +}; +var loose: SortedSet(f64, ApproxEql) = .{ + .items = .empty, + .ctx = .{ .tolerance = 1.0 }, +}; +``` + +**Standard library example** - HashMap requires `hash` and `eql`: +```zig +// Case-insensitive string keys +const CaseInsensitive = struct { + pub fn hash(_: @This(), key: []const u8) u64 { + var h = std.hash.Wyhash.init(0); + for (key) |c| h.update(&.{std.ascii.toLower(c)}); + return h.final(); + } + pub fn eql(_: @This(), a: []const u8, b: []const u8) bool { + return std.ascii.eqlIgnoreCase(a, b); + } +}; + +const Headers = std.HashMap([]const u8, []const u8, CaseInsensitive, 80); +``` + +**When to use:** +- Your generic type needs customizable comparison, hashing, ordering, or formatting +- Different use cases need different behavior for the same data type +- You want compile-time polymorphism without runtime vtable overhead + +**Design guidelines:** +- Document which methods the context must provide +- Use `_: @This()` for stateless contexts (optimizer eliminates the parameter) +- Store `ctx: Context` as a field when the generic needs to call methods later +- Consider providing a default context type for common cases + +#### Pointer Size Type Selection +Use `switch (@sizeOf(usize))` to select platform-appropriate types at compile time. + +```zig +pub const Auxv = switch (@sizeOf(usize)) { + 4 => Elf32_auxv_t, + 8 => Elf64_auxv_t, + else => @compileError("expected pointer size of 32 or 64"), +}; + +pub const Ehdr = switch (@sizeOf(usize)) { + 4 => Elf32_Ehdr, + 8 => Elf64_Ehdr, + else => @compileError("expected pointer size of 32 or 64"), +}; +``` + +**When to use:** FFI with C libraries, ELF parsing, or any code that needs different types for 32-bit vs 64-bit platforms. + +#### Default Arguments via Options Struct +Use a struct with default field values to simulate optional/default function arguments. + +```zig +pub const Options = struct { + /// The alignment of the memory pool items. Use `null` for natural alignment. + alignment: ?Alignment = null, + /// If `true`, the memory pool can allocate additional items after initial setup. + growable: bool = true, +}; + +pub fn MemoryPoolExtra(comptime Item: type, comptime pool_options: Options) type { + return struct { + // Implementation uses pool_options.alignment, pool_options.growable + }; +} + +// Usage: +const Pool1 = MemoryPoolExtra(u32, .{}); // All defaults +const Pool2 = MemoryPoolExtra(u32, .{ .growable = false }); // Override one +``` + +**When to use:** Functions with many optional parameters, builder patterns, configuration structs. + +#### Side Computation Block +Use labeled blocks to perform intermediate calculations with a clear scope boundary. + +```zig +// Calculate length needed for resulting joined path buffer. +const total_len = blk: { + var sum: usize = paths[first_path_index].len; + var prev_path = paths[first_path_index]; + var i: usize = first_path_index + 1; + while (i < paths.len) : (i += 1) { + const this_path = paths[i]; + if (this_path.len == 0) continue; + sum += this_path.len; + prev_path = this_path; + } + if (zero) sum += 1; + break :blk sum; +}; +``` + +**When to use:** Complex expressions that need temporary variables, when you want to limit variable scope. + +#### Destructuring Assignment +Unpack tuples, arrays, and vectors into individual variables. + +```zig +// From function returning tuple +fn divmod(n: u32, d: u32) struct { u32, u32 } { + return .{ n / d, n % d }; +} +const div, const mod = divmod(10, 3); + +// Array destructuring with swizzle +fn swizzleRgbaToBgra(rgba: [4]u8) [4]u8 { + const r, const g, const b, const a = rgba; + return .{ b, g, r, a }; +} + +// Ignore elements with _ +const first, _, const third, _ = some_array; + +// Works with comptime +comptime const x, const y = .{ 1, 2 }; +``` + +**When to use:** Multiple return values, array element extraction, SIMD vector unpacking. + +#### Hashed Mappings Storage +Use multiple `AutoArrayHashMapUnmanaged` fields when storing complex interned data. + +```zig +// From llvm/Builder.zig - demonstrating the pattern of parallel maps for interned data +string_map: std.AutoArrayHashMapUnmanaged(void, void), +string_indices: std.ArrayListUnmanaged(u32), +string_bytes: std.ArrayListUnmanaged(u8), + +types: std.AutoArrayHashMapUnmanaged(String, Type), +type_map: std.AutoArrayHashMapUnmanaged(void, void), +type_items: std.ArrayListUnmanaged(Type.Item), +type_extra: std.ArrayListUnmanaged(u32), + +attributes: std.AutoArrayHashMapUnmanaged(Attribute.Storage, void), +attributes_map: std.AutoArrayHashMapUnmanaged(void, void), +attributes_indices: std.ArrayListUnmanaged(u32), +``` + +**When to use:** Interning strings/symbols, IR builders, AST storage, deduplication with stable indices. + +#### Module-wide Overridable Options +Use `@import("root")` to allow users to customize library behavior. + +```zig +const root = @import("root"); + +/// Stdlib-wide options that can be overridden by the root file. +pub const options: Options = if (@hasDecl(root, "std_options")) root.std_options else .{}; + +pub const Options = struct { + enable_segfault_handler: bool = debug.default_enable_segfault_handler, + log_level: log.Level = log.default_level, + // ... +}; + +// In user's main.zig: +pub const std_options: std.Options = .{ + .log_level = .debug, +}; +``` + +**When to use:** Library configuration, logging levels, feature flags that users can customize. + +#### Self-referential Type Alias +Use `const Self = @This();` inside a struct for self-reference. + +```zig +// GOOD: Simple alias at top of struct +pub fn EnumSet(comptime E: type) type { + return struct { + const Self = @This(); + + pub const Indexer = EnumIndexer(E); + pub const Key = Indexer.Key; + + bits: BitSet, + + pub fn contains(self: Self, key: Key) bool { + return self.bits.isSet(Indexer.indexOf(key)); + } + }; +} + +// ANTI-PATTERN: Unnecessary Self usage when @This() would be clearer +pub const PaxIterator = struct { + size: usize, + reader: *std.Io.Reader, + + const Self = @This(); // Unnecessary - only used once + + // Better: just use @This() inline or the actual type name +}; +``` + +**When to use:** Generic types where `Self` is used multiple times. Avoid in non-generic structs where it adds no value. + +#### Variable Struct Initialization +Use `pub const init: T = .{...};` for default/initial values. + +```zig +// GOOD: Named constant with explicit type +pub const Recursive = struct { + mutex: Mutex, + thread_id: std.Thread.Id, + lock_count: usize, + + pub const init: Recursive = .{ + .mutex = .{}, + .thread_id = invalid_thread_id, + .lock_count = 0, + }; +}; + +// Usage: +var rec: Recursive = Recursive.init; + +// ANTI-PATTERN: Using T{} syntax (older style) +var mutex = Mutex{}; // Works but .{} is preferred +``` + +**When to use:** Types with meaningful default states, especially when zero-initialization isn't appropriate. + +#### Return Value Struct Initialization +Return `.{...}` directly instead of creating named locals. + +```zig +// GOOD: Direct return +pub fn initContext(allocator: Allocator, ctx: Context) Self { + return .{ + .unmanaged = .empty, + .allocator = allocator, + .ctx = ctx, + }; +} + +// ANTI-PATTERN: Unnecessary named local +pub fn getEntryAdapted(self: Self, key: anytype, ctx: anytype) ?Entry { + const index = self.getIndexAdapted(key, ctx) orelse return null; + const slice = self.entries.slice(); + return Entry{ // Could just be: return .{ + .key_ptr = &slice.items(.key)[index], + .value_ptr = &slice.items(.value)[index], + }; +} +``` + +**When to use:** Simple struct construction in return statements. Use named locals only when the struct is complex or needs multiple statements to build. + +### II. Polymorphism Patterns + +#### Duck Typing +Use `anytype` parameters with compile-time interface checking. + +```zig +pub fn sort( + comptime T: type, + items: []T, + context: anytype, // Duck typed: must have lessThan behavior + comptime lessThanFn: fn (@TypeOf(context), lhs: T, rhs: T) bool, +) void { + std.sort.block(T, items, context, lessThanFn); +} + +// Works with any type that can be passed to lessThanFn +sort(i32, &items, {}, struct { + fn lt(_: void, a: i32, b: i32) bool { return a < b; } +}.lt); +``` + +**When to use:** Callbacks, comparators, iterators—when the exact type doesn't matter, only its capabilities. + +#### Generic Type +Return a parameterized struct from a function. + +```zig +pub fn HashMap( + comptime K: type, + comptime V: type, + comptime Context: type, + comptime max_load_percentage: u64, +) type { + return struct { + unmanaged: Unmanaged, + allocator: Allocator, + ctx: Context, + + pub const Unmanaged = HashMapUnmanaged(K, V, Context, max_load_percentage); + pub const Entry = Unmanaged.Entry; + // ... + }; +} + +const MyMap = HashMap([]const u8, u32, std.hash_map.StringContext, 80); +``` + +**When to use:** Data structures, containers, any reusable component parameterized by types. + +#### Generic Function +Use `comptime T: type` for functions operating on any type. + +```zig +pub fn allEqual(comptime T: type, slice: []const T, scalar: T) bool { + for (slice) |item| { + if (item != scalar) return false; + } + return true; +} + +// Usage: +const all_zero = std.mem.allEqual(u8, buffer, 0); +const all_space = std.mem.allEqual(u8, text, ' '); +``` + +**When to use:** Utility functions, algorithms that work on any type with certain properties. + +#### Basic Type Formatting +Built-in format specifiers for `std.fmt.print`. + +```zig +// x/X: hexadecimal +try w.print("{x}", .{255}); // "ff" +try w.print("{X}", .{255}); // "FF" + +// s: strings and slices +try w.print("{s}", .{"hello"}); // "hello" + +// t: tag names for enums/unions/errors +try w.print("{t}", .{MyEnum.foo}); // "foo" + +// d: decimal, b: binary, o: octal +try w.print("{d} {b} {o}", .{10, 10, 10}); // "10 1010 12" + +// e: scientific notation +try w.print("{e}", .{1234.5}); // "1.2345e+03" + +// c: ASCII character, u: UTF-8 codepoint +try w.print("{c} {u}", .{65, 0x1F600}); // "A 😀" + +// f: custom formatter, including std.Io.Duration +try w.print("{f}", .{std.Io.Duration.fromNanoseconds(3_661_001_000_000)}); + +// B/Bi: bytes in SI/IEC units +try w.print("{B} {Bi}", .{1536, 1536}); // "1.536kB 1.5KiB" +``` + +#### Custom Type Formatting +Implement `format` method for custom types. + +```zig +const Version = struct { + major: u32, + minor: u32, + patch: u32, + pre: ?[]const u8 = null, + build: ?[]const u8 = null, + + pub fn format(self: Version, w: *std.Io.Writer) std.Io.Writer.Error!void { + try w.print("{d}.{d}.{d}", .{ self.major, self.minor, self.patch }); + if (self.pre) |pre| try w.print("-{s}", .{pre}); + if (self.build) |build| try w.print("+{s}", .{build}); + } +}; + +// Usage with {f} specifier +try stdout.print("{f}", .{version}); +``` + +**When to use:** Any type that needs custom string representation. + +#### Custom Type JSON +Implement `jsonParse`, `jsonParseFromValue`, and `jsonStringify` for JSON support. + +```zig +pub fn ArrayHashMap(comptime T: type) type { + return struct { + map: std.StringArrayHashMapUnmanaged(T) = .empty, + + pub fn jsonParse( + allocator: Allocator, + source: anytype, + options: ParseOptions, + ) !@This() { + var map: std.StringArrayHashMapUnmanaged(T) = .empty; + errdefer map.deinit(allocator); + + if (.object_begin != try source.next()) return error.UnexpectedToken; + while (true) { + const token = try source.nextAlloc(allocator, options.allocate.?); + switch (token) { + inline .string, .allocated_string => |k| { + const gop = try map.getOrPut(allocator, k); + gop.value_ptr.* = try innerParse(T, allocator, source, options); + }, + .object_end => break, + else => unreachable, + } + } + return .{ .map = map }; + } + + pub fn jsonStringify(self: @This(), w: anytype) !void { + // ... serialize to JSON + } + }; +} +``` + +**When to use:** Types with non-trivial JSON representation, maps with dynamic keys. + +#### Compile-time Implementation Switching +Select platform-specific implementations at compile time. + +```zig +const native_os = builtin.os.tag; +const use_libc = builtin.link_libc; + +/// A libc-compatible API layer. +pub const system = if (use_libc) + std.c +else switch (native_os) { + .linux => linux, + .plan9 => std.os.plan9, + else => struct { + pub const ucontext_t = void; + pub const pid_t = void; + pub const fd_t = void; + }, +}; + +pub const AF = system.AF; +pub const pid_t = system.pid_t; +``` + +**When to use:** Cross-platform code, OS-specific syscall wrappers, CPU architecture selection. + +#### Dynamic Dispatch (Fat Pointer) +Use `ptr: *anyopaque` with `vtable: *const VTable` for runtime polymorphism. + +```zig +const Allocator = struct { + ptr: *anyopaque, + vtable: *const VTable, + + pub const VTable = struct { + alloc: *const fn (*anyopaque, len: usize, alignment: Alignment, ret_addr: usize) ?[*]u8, + resize: *const fn (*anyopaque, []u8, Alignment, new_len: usize, ret_addr: usize) bool, + free: *const fn (*anyopaque, []u8, Alignment, ret_addr: usize) void, + }; + + pub fn alloc(self: Allocator, len: usize, alignment: Alignment) ?[*]u8 { + return self.vtable.alloc(self.ptr, len, alignment, @returnAddress()); + } +}; +``` + +**When to use:** Allocators, I/O interfaces, plugin systems—when you need runtime polymorphism and can't use comptime generics. + +#### Static Dispatch (Tagged Union with inline switch) +Use tagged unions with `inline else` for compile-time generated dispatch. + +```zig +const U = union(enum) { + a: u32, + b: f32, +}; + +fn getNum(u: U) u32 { + switch (u) { + // Generates separate code paths at compile time + inline else => |num, tag| { + if (tag == .b) { + return @intFromFloat(num); + } + return num; + }, + } +} + +// More common: uniform operations on all variants +const AnySlice = union(enum) { + a: []const u8, + b: []const u16, + c: []const u32, +}; + +fn len(any: AnySlice) usize { + return switch (any) { + inline else => |slice| slice.len, + }; +} +``` + +**When to use:** Variants with similar operations, type-safe enums, when you want compiler-generated dispatch instead of vtables. + +### III. Safety Patterns + +#### Diagnostics +Use an optional diagnostics struct to provide detailed error information. + +```zig +pub const ParseOptions = struct { + string: ?[]const u8 = null, + dynamic_linker: ?[]const u8 = null, + + /// If provided, the function will populate information about parsing failures. + diagnostics: ?*Diagnostics = null, + + pub const Diagnostics = struct { + arch: ?Target.Cpu.Arch = null, + os_name: ?[]const u8 = null, + os_tag: ?Target.Os.Tag = null, + abi: ?Target.Abi = null, + cpu_name: ?[]const u8 = null, + unknown_feature_name: ?[]const u8 = null, + }; +}; + +pub fn parse(args: ParseOptions) !Query { + var dummy_diags: ParseOptions.Diagnostics = undefined; + const diags = args.diagnostics orelse &dummy_diags; + + // On error, populate diags before returning + diags.arch = detected_arch; + return error.UnknownArchitecture; +} +``` + +**When to use:** Parser functions, validators, anywhere you want to report multiple issues or provide context with errors. + +#### Index-Based Data Structures + +Zig enums are strongly-typed integer constants. By default, the compiler chooses a minimal backing type, but you can specify one explicitly with `enum(u32)`. Adding a trailing `_` field makes the enum *non-exhaustive*: any value of the backing type becomes valid, not just named members. This means `enum(u32) { _ }` is effectively "u32, but a distinct type"—the compiler won't implicitly convert between different enum types even if they share the same backing integer. + +Use this to create distinct index types that the type system can distinguish. This pattern prevents bugs from accidentally mixing up indices into different arrays or confusing semantically different integers. + +```zig +/// Index into `sections` array. +const SectionIndex = enum(u32) { + _, +}; + +/// Index into `functions` array. +const FunctionIndex = enum(u32) { + _, +}; + +/// Index into `symbols` array. +const SymbolIndex = enum(u32) { + _, +}; +``` + +These types are incompatible with each other, even though they share the same backing integer: + +```zig +fn getSection(index: SectionIndex) *Section { ... } +fn getFunction(index: FunctionIndex) *Function { ... } + +// COMPILE ERROR: type mismatch +const section = getSection(func_index); // func_index is FunctionIndex, not SectionIndex + +// CORRECT: types match +const section = getSection(section_index); +``` + +**Converting to/from the underlying integer:** +```zig +const index: SectionIndex = @enumFromInt(42); +const raw: u32 = @intFromEnum(index); +``` + +**Optional variants** - use a sentinel value for null representation: +```zig +/// Index into `functions`, or null. +const OptionalFunctionIndex = enum(u32) { + none = std.math.maxInt(u32), + _, + + pub fn unwrap(self: OptionalFunctionIndex) ?FunctionIndex { + if (self == .none) return null; + return @enumFromInt(@intFromEnum(self)); + } +}; + +/// Non-optional index with conversion helper. +const FunctionIndex = enum(u32) { + _, + + pub fn toOptional(self: FunctionIndex) OptionalFunctionIndex { + const result: OptionalFunctionIndex = @enumFromInt(@intFromEnum(self)); + std.debug.assert(result != .none); + return result; + } +}; +``` + +**With sentinel states** - multiple special values: +```zig +const Parent = enum(u8) { + /// Unallocated storage. + unused = std.math.maxInt(u8) - 1, + /// Indicates root node. + none = std.math.maxInt(u8), + /// Index into `node_storage`. + _, + + fn unwrap(self: @This()) ?NodeIndex { + return switch (self) { + .unused, .none => null, + _ => @enumFromInt(@intFromEnum(self)), + }; + } +}; +``` + +**When to use:** +- Any index into an array/slice where you have multiple arrays +- Handles, IDs, or tokens that should not be interchangeable +- Any integer with semantic meaning that could be confused with other integers +- Linkers, compilers, parsers, and any code managing multiple parallel data structures + +**Benefits:** +- Compile-time detection of index mix-ups (otherwise painful runtime debugging) +- Self-documenting code - the type name explains what the integer represents +- Zero runtime cost - same representation as the underlying integer + +**Why Indices Over Pointers** + +Index-based data structures offer significant advantages over pointer-based ones: + +- **Memory efficiency**: 4 bytes (u32) vs 8 bytes (pointer) per reference—50% savings +- **Cache locality**: Contiguous array storage means better cache utilization +- **Fewer allocations**: Append to array vs individual `create()` calls +- **Instant bulk frees**: One `deinit()` vs recursive traversal +- **Natural serialization**: Indices are relocatable; can `@memcpy` entire arrays + +**Tree and Graph Modeling** + +Use the "collective noun first" idiom: define the container (`Tree`) before the index type (`Node`). The index is just an integer—actual data lives in the container. + +```zig +/// A tree structure using index-based nodes. +pub const Tree = struct { + /// Node storage - the actual data lives here. + nodes: std.MultiArrayList(Node.Data), + /// Root is always index 0. + root: Node = .root, + + pub const Node = enum(u32) { + root = 0, + _, + + /// Data stored for each node. + pub const Data = struct { + tag: Tag, + parent: OptionalNode, + children: Children, + // ... other fields + }; + + pub const Tag = enum { leaf, branch }; + + pub const Children = struct { + start: Node, + len: u32, + }; + }; + + pub const OptionalNode = enum(u32) { + none = std.math.maxInt(u32), + _, + + pub fn unwrap(self: OptionalNode) ?Node { + if (self == .none) return null; + return @enumFromInt(@intFromEnum(self)); + } + }; + + /// Access node data by index. + pub fn get(self: *const Tree, node: Node) Node.Data { + return self.nodes.get(@intFromEnum(node)); + } + + /// Get mutable pointer to node data. + pub fn getPtr(self: *Tree, node: Node) *Node.Data { + const slice = self.nodes.slice(); + return &slice.items(.tag)[@intFromEnum(node)]; + } +}; +``` + +This pattern mirrors `std.zig.Ast` from the Zig standard library: +- `Ast.Node.Index` is `enum(u32) { _, }` (lines 3020-3035 in Ast.zig) +- Node data accessed via `ast.nodes.get(index)` +- Optional indices use `maxInt` sentinel: `Ast.Node.OptionalIndex` + +**Index Ranges** + +For variable-length children, use the `Index.Range` pattern from `Zoir.zig`: + +```zig +pub const Node = enum(u32) { + _, + + /// A range of contiguous node indices. + pub const Range = struct { + start: Node, + len: u32, + + /// Get the node at offset `i` within this range. + pub fn at(r: Range, i: u32) Node { + std.debug.assert(i < r.len); + return @enumFromInt(@intFromEnum(r.start) + i); + } + + /// Iterate over all nodes in range. + pub fn slice(r: Range) []const Node { + // Note: requires nodes stored contiguously + return @ptrCast(@as([*]const u32, @ptrFromInt(@intFromEnum(r.start)))[0..r.len]); + } + }; +}; + +// Usage in tree traversal: +fn visitChildren(tree: *const Tree, node: Tree.Node) void { + const data = tree.get(node); + var i: u32 = 0; + while (i < data.children.len) : (i += 1) { + const child = data.children.at(i); + visit(tree, child); + } +} +``` + +**Freelist for Deletion** + +When individual node deletion is needed, maintain a freelist stack: + +```zig +pub const NodePool = struct { + nodes: std.ArrayListUnmanaged(Node.Data), + /// Head of freelist, or none if no free slots. + free_head: OptionalNode = .none, + + pub fn alloc(self: *NodePool) !Node { + if (self.free_head.unwrap()) |free| { + // Reuse freed slot + self.free_head = self.nodes.items[@intFromEnum(free)].next_free; + return free; + } + // Allocate new slot + const index: Node = @enumFromInt(self.nodes.items.len); + try self.nodes.append(undefined); + return index; + } + + pub fn free(self: *NodePool, node: Node) void { + // Push onto freelist + self.nodes.items[@intFromEnum(node)].next_free = self.free_head; + self.free_head = node.toOptional(); + } +}; +``` + +**Stdlib examples of these patterns:** +| Pattern | File | Description | +|---------|------|-------------| +| `Node.Index` | `Ast.zig:3020-3035` | Basic index type | +| `Node.OptionalIndex` | `Ast.zig:3038-3050` | Optional with maxInt sentinel | +| `Index.Range` | `Zoir.zig:151-159` | Contiguous index ranges | +| Multiple sentinels | `Progress.zig:157-171` | `unused`, `none` states | +| Accessor pattern | `Ast.zig:88-106` | `ast.nodes.get(index)` | + +#### Error Payloads +Use a tagged union to attach context to errors. + +```zig +pub const Diagnostics = struct { + errors: std.ArrayListUnmanaged(Error) = .empty, + entries: usize = 0, + + pub const Error = union(enum) { + unable_to_create_sym_link: struct { + code: anyerror, + file_name: []const u8, + link_name: []const u8, + }, + unable_to_create_file: struct { + code: anyerror, + file_name: []const u8, + }, + unsupported_file_type: struct { + file_name: []const u8, + file_type: Header.Kind, + }, + }; +}; + +// Usage: collect errors instead of failing immediately +fn extract(d: *Diagnostics, ...) !void { + file.create(...) catch |err| { + try d.errors.append(allocator, .{ + .unable_to_create_file = .{ .code = err, .file_name = name }, + }); + return; + }; +} +``` + +**When to use:** Batch processing with multiple possible failures, when you need more context than error codes provide. + +#### Compile-time Assertion +Use `comptime { assert(...); }` for compile-time invariant checking. + +```zig +const upper_bound_msg_len = 1 + node_storage_buffer_len * @sizeOf(Node.Storage) + + node_storage_buffer_len * @sizeOf(Node.OptionalIndex); +comptime assert(upper_bound_msg_len <= 4096); + +// Also works with @compileError for custom messages +comptime { + if (@sizeOf(MyStruct) > 64) { + @compileError("MyStruct too large for cache line"); + } +} +``` + +**When to use:** Size constraints, alignment requirements, invariants that must hold at compile time. + +#### Granular Error Handling +Use exhaustive switch on specific error values for precise handling. + +```zig +fn oom(err: anytype) noreturn { + switch (err) { + error.OutOfMemory => @panic("out of memory"), + } +} + +// Or handle different errors differently: +file.open() catch |err| switch (err) { + error.FileNotFound => return createDefault(), + error.AccessDenied => return error.PermissionDenied, + error.IsDir => return error.InvalidPath, + else => return err, +}; +``` + +**When to use:** When different errors need different handling, converting between error sets. + +#### Deallocated Memory Poisoning +Set `self.* = undefined;` after deallocation to catch use-after-free. + +```zig +pub fn deinit(self: *Self, gpa: Allocator) void { + gpa.free(self.allocatedSlice()); + self.* = undefined; // Poison the memory +} +``` + +**When to use:** In `deinit` functions to help catch use-after-free bugs in debug builds. Zig writes `0xaa` bytes to undefined memory in debug mode. + +#### Deferred Resource Deinitialization +Use `defer` for unconditional cleanup. + +```zig +fn doWork(self: *Self) void { + self.mutex.lock(); + defer self.mutex.unlock(); // Always runs, even on error + + // ... work with locked resource +} +``` + +**When to use:** Mutexes, file handles, any resource that must be released regardless of control flow. + +#### Error-deferred Resource Deinitialization +Use `errdefer` for cleanup only on error paths. + +```zig +pub fn put(self: *BufMap, key: []const u8, value: []const u8) !void { + const value_copy = try self.copy(value); + errdefer self.free(value_copy); // Only runs if we return an error + + const get_or_put = try self.hash_map.getOrPut(key); + // ... if this succeeds, value_copy ownership is transferred +} +``` + +**When to use:** Partial initialization, multi-step construction where early steps need rollback on later failures. + +#### Compile-time Unreachable Switch Prong +Use `else => comptime unreachable` for exhaustive compile-time switches. + +```zig +fn shl(a: anytype, shift_amt: anytype) @TypeOf(a) { + const casted_shift_amt = switch (@typeInfo(@TypeOf(shift_amt))) { + .int => @as(Log2Int(@TypeOf(a)), @intCast(shift_amt)), + .comptime_int => @as(Log2Int(@TypeOf(a)), shift_amt), + else => comptime unreachable, // Only int types allowed + }; + return a << casted_shift_amt; +} +``` + +**When to use:** Generic functions where only certain type categories are valid. + +#### Compile-time Error Absence Guarantee +Use `errdefer comptime unreachable;` to assert no errors can occur after a point. + +```zig +fn spawnChild(self: *Child) !void { + const pid_result = posix.fork(); + if (pid_result == 0) { + // Child process + posix.execvpeZ(...); + forkChildErrReport(err_pipe[1], err); + } + + // Parent process - after fork, we must not error + errdefer comptime unreachable; // Compile error if any code below can error + + posix.close(err_pipe[1]); + self.err_pipe = err_pipe[0]; + // ... all operations here must be infallible +} +``` + +**When to use:** After point-of-no-return operations like fork(), to ensure subsequent code is truly infallible. + +#### Reserve-First Exception Safety +When mutating data structures that can fail (e.g., growing arrays or hash maps), separate the fallible reservation phase from the infallible mutation phase. This ensures strong exception safety: if an error occurs, the object remains unchanged. + +**The pattern:** +1. **Reserve** - Call `ensureUnusedCapacity` for all containers that will grow. These calls can fail but don't mutate data. +2. **Mark boundary** - Use `errdefer comptime unreachable;` to assert no errors can occur after this point. +3. **Mutate** - Use `*AssumeCapacity` methods which cannot fail. + +```zig +// WRONG - exception safety bug +pub fn internString(state: *State, gpa: Allocator, bytes: []const u8) !String { + // BUG: getOrPut inserts a slot, then ensureUnusedCapacity can fail, + // leaving an uninitialized entry in the hash table + const gop = try state.string_table.getOrPut(gpa, bytes); + if (gop.found_existing) return gop.key_ptr.*; + + try state.string_bytes.ensureUnusedCapacity(gpa, bytes.len + 1); // Can fail! + // ... rest of function +} + +// CORRECT - reserve first, then mutate +pub fn internString(state: *State, gpa: Allocator, bytes: []const u8) !String { + // Phase 1: Reserve capacity (all fallible operations) + try state.string_table.ensureUnusedCapacityContext(gpa, 1, .{ + .bytes = state.string_bytes.items, + }); + try state.string_bytes.ensureUnusedCapacity(gpa, bytes.len + 1); + + errdefer comptime unreachable; // Phase 2: No errors after this point + + // Phase 3: Mutate using AssumeCapacity methods (infallible) + const gop = state.string_table.getOrPutAssumeCapacityAdapted(bytes, .{ + .bytes = state.string_bytes.items, + }); + if (gop.found_existing) return gop.key_ptr.*; + + const new_off: String = @enumFromInt(state.string_bytes.items.len); + state.string_bytes.appendSliceAssumeCapacity(bytes); + state.string_bytes.appendAssumeCapacity(0); + gop.key_ptr.* = new_off; + + return new_off; +} +``` + +**Real-world example from HashMap.grow:** +```zig +fn grow(self: *Self, allocator: Allocator, new_capacity: Size, ctx: Context) Allocator.Error!void { + var map: Self = .{}; + try map.allocate(allocator, new_cap); // Can fail + errdefer comptime unreachable; // No errors after this point + + map.initMetadatas(); // Infallible + map.available = @truncate((new_cap * max_load_percentage) / 100); + + // Copy all entries using putAssumeCapacityNoClobberContext (infallible) + if (self.size != 0) { + for (self.metadata.?[0..old_capacity], self.keys()[0..old_capacity], self.values()[0..old_capacity]) |m, k, v| { + if (!m.isUsed()) continue; + map.putAssumeCapacityNoClobberContext(k, v, ctx); // Infallible + } + } + // ... swap and cleanup +} +``` + +**Exception safety levels:** +- **Strong** (reserve-first): Object unchanged if error occurs +- **Basic**: Object left in valid but different state +- **None**: Object may be corrupted + +**When to use:** +- Any function that must insert into multiple containers +- Growing data structures where partial mutation would corrupt state +- String/symbol interning (hash table + byte array) +- Any operation where failure after partial mutation leaves invalid state + +**Key insight:** `ensureUnusedCapacity` is magic—it contains all the failure modes but changes nothing. Reservation failures are safe to retry; partial mutations are not. + +### IV. Performance Patterns + +#### Big Struct Constant Pointer Passing +Pass large structs by `*const` to avoid copies. + +```zig +// GOOD: Pass by const pointer for large structs +pub fn format(uri: *const Uri, writer: *Writer) Writer.Error!void { + return writeToStream(uri, writer, .all); +} + +pub fn writeToStream(uri: *const Uri, writer: *Writer, flags: Format.Flags) Writer.Error!void { + if (flags.scheme) { + try writer.print("{s}:", .{uri.scheme}); + } + // ... +} +``` + +**When to use:** Structs larger than ~2 pointers that are read-only. The calling convention may copy small structs in registers anyway. + +#### Big Struct Constant Pointer Capturing +Use `*const` in closures to avoid copying large captured values. + +```zig +if (m.resolved_target) |*target| { // *target is a pointer, not a copy + if (!target.query.isNative()) { + try zig_args.appendSlice(&.{ + "-target", try target.query.zigTriple(b.allocator), + "-mcpu", try target.query.serializeCpuAlloc(b.allocator), + }); + } +} +``` + +**When to use:** When capturing large structs in closures or iterating with payload capture on large items. + +### V. Workarounds + +#### Inlined Loop with Runtime Logic +When `inline for` doesn't work (e.g., modifying slice during iteration), use `comptime var` with `inline while`. + +```zig +fn deinterlace(interlaced: anytype, comptime vec_count: usize) [vec_count]@Vector(...) { + const vec_len = vectorLength(@TypeOf(interlaced)) / vec_count; + const Child = std.meta.Child(@TypeOf(interlaced)); + + var out: [vec_count]@Vector(vec_len, Child) = undefined; + + // inline for doesn't work here due to runtime slice mutation + comptime var i: usize = 0; + inline while (i < out.len) : (i += 1) { + const indices = comptime iota(i32, vec_len) * + @as(@Vector(vec_len, i32), @splat(@intCast(vec_count))) + + @as(@Vector(vec_len, i32), @splat(@intCast(i))); + out[i] = @shuffle(Child, interlaced, undefined, indices); + } + + return out; +} +``` + +**When to use:** When you need compile-time loop unrolling but `inline for` fails due to control flow or mutation requirements. diff --git a/references/std-allocators.md b/references/std-allocators.md new file mode 100644 index 0000000..0e9755c --- /dev/null +++ b/references/std-allocators.md @@ -0,0 +1,592 @@ +# std.heap - Allocators + +Zig has no default allocator. Functions that need heap memory accept an `Allocator` parameter. + +## Quick Reference + +| Allocator | Use Case | Thread-Safe | +|-----------|----------|-------------| +| `std.testing.allocator` | Unit tests (leak detection) | No | +| `std.heap.FixedBufferAllocator` | Stack-based, bounded size known | Optional | +| `std.heap.ArenaAllocator` | Batch free, CLI apps, request handlers | No | +| `std.heap.page_allocator` | Backing for other allocators | Yes | +| `std.heap.c_allocator` | Linking libc, interop | Yes | +| `std.heap.raw_c_allocator` | Libc arena backing (no alignment overhead) | Yes | +| `std.heap.DebugAllocator` | Debug builds, leak/corruption detection | Configurable | +| `std.heap.smp_allocator` | ReleaseFast production multithreaded | Yes | +| `std.heap.MemoryPool` | High-frequency same-type allocations | No | +| `std.heap.ThreadSafeAllocator` | Wrap non-thread-safe allocator | Yes | +| `std.heap.StackFallbackAllocator` | Stack buffer with heap fallback | Depends | +| `std.heap.wasm_allocator` | WebAssembly targets | Yes | + +## Allocator Naming Conventions + +Using a generic `allocator` name hides memory ownership contracts. Name allocators by their **memory contract** to make code self-documenting: + +| Name | Contract | Can Return Data? | +|------|----------|------------------| +| `gpa` | Caller **must** free with `defer gpa.free()` | Yes | +| `arena` | Bulk-deallocated at system boundary | Yes | +| `scratch` | Function-private temporary space | **Never** | + +### The Problem + +```zig +// BAD - "allocator" says nothing about ownership +fn process(allocator: Allocator) ![]u8 { + const temp = try allocator.alloc(u8, 100); // Who frees this? + const result = try allocator.dupe(u8, temp); // Who owns this? + allocator.free(temp); // Is this correct? + return result; // Can caller free with same allocator? +} +``` + +### The Solution + +Name allocators by their contract: + +```zig +// GOOD - names communicate ownership contracts +fn process( + gpa: Allocator, // General-purpose: caller must free returned data + scratch: Allocator, // Temporary: never return data allocated here +) ![]u8 { + // scratch is for intermediate computation only + const temp = try scratch.alloc(u8, 100); + defer scratch.free(temp); + + // gpa for data that outlives this function + return try gpa.dupe(u8, computeResult(temp)); +} +``` + +### Full Example with All Three + +```zig +fn handleRequest( + request: *Request, + arena: Allocator, // Response lifetime - bulk freed after response sent + gpa: Allocator, // Long-lived data - cache, shared state + scratch: Allocator, // This function only - intermediate computation +) !Response { + // Scratch: temporary parsing buffers (never escapes this function) + const parsed = try parseBody(request.body, scratch); + + // GPA: update shared cache (outlives request) + try updateCache(gpa, parsed.cache_key, parsed.value); + + // Arena: response data (freed when response completes) + const response_body = try formatResponse(arena, parsed); + + return Response{ .body = response_body }; +} +``` + +### Common Patterns + +**CLI applications** - arena for everything, freed at exit: +```zig +pub fn main() !void { + var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator); + defer arena.deinit(); + try run(arena.allocator()); // Name as "arena" - bulk freed at end +} +``` + +**Request handlers** - arena per request, gpa for shared state: +```zig +fn handleRequest(gpa: Allocator, request: Request) !Response { + var request_arena = std.heap.ArenaAllocator.init(gpa); + defer request_arena.deinit(); + const arena = request_arena.allocator(); + + // arena: request-scoped data + // gpa: data that outlives the request (caches, connections) +} +``` + +**Functions with temporary allocations** - scratch parameter: +```zig +/// Computes result using scratch for intermediate work. +/// Caller owns returned slice (allocated from gpa). +fn compute(gpa: Allocator, scratch: Allocator, input: []const u8) ![]u8 { + const temp = try scratch.alloc(u8, input.len * 2); + defer scratch.free(temp); + // ... use temp for intermediate computation ... + return try gpa.dupe(u8, result); +} +``` + +## Allocator Interface + +```zig +const Allocator = std.mem.Allocator; + +// Single items: create/destroy +const ptr: *T = try allocator.create(T); +defer allocator.destroy(ptr); + +// Slices: alloc/free +const slice: []T = try allocator.alloc(T, count); +defer allocator.free(slice); + +// Duplicate existing slice +const copy = try allocator.dupe(u8, source); +defer allocator.free(copy); + +// Resize (returns bool - true if resized in place) +if (allocator.resize(slice, new_len)) { + // slice is now new_len (pointer unchanged) +} + +// Reallocate (may move, returns new slice) +slice = try allocator.realloc(slice, new_len); +``` + +## Choosing an Allocator + +**Decision flow:** + +1. **Library code?** Accept `Allocator` parameter - let caller decide +2. **Unit test?** Use `std.testing.allocator` (has leak detection) +3. **Size known at comptime?** Use `FixedBufferAllocator` with stack buffer +4. **Stack with heap fallback?** Use `stackFallback(N, backing_allocator)` +5. **CLI app / one-shot?** Use `ArenaAllocator` wrapping `page_allocator` +6. **Request loop (web/game)?** Use `ArenaAllocator`, reset per iteration +7. **Many same-type objects?** Use `MemoryPool(T)` for fast create/destroy +8. **Debug build?** Use `DebugAllocator` for leak/corruption detection +9. **ReleaseFast production?** Use `std.heap.smp_allocator` +10. **Linking libc?** Use `c_allocator` or `raw_c_allocator` (as arena backing) + +## Common Allocators + +### Testing Allocator + +```zig +test "example" { + const allocator = std.testing.allocator; + const data = try allocator.alloc(u8, 100); + defer allocator.free(data); // Leak detected if missing! +} +``` + +### FixedBufferAllocator + +No heap allocations - allocates into a fixed buffer. Useful for kernels, embedded, or performance-critical code. Returns `OutOfMemory` when buffer exhausted: + +```zig +var buffer: [4096]u8 = undefined; +var fba = std.heap.FixedBufferAllocator.init(&buffer); +const allocator = fba.allocator(); + +const data = try allocator.alloc(u8, 100); +// Free/resize only works for most recent allocation +allocator.free(data); + +// Reset to reuse buffer +fba.reset(); +``` + +**Thread-safe variant** (allocate only - no resize/free): +```zig +const ts_allocator = fba.threadSafeAllocator(); +``` + +**Ownership checks:** +```zig +if (fba.ownsPtr(ptr)) { ... } // Check if pointer is within buffer +if (fba.ownsSlice(slice)) { ... } // Check if slice is within buffer +``` + +### ArenaAllocator + +Wraps a child allocator. Allocate many times, free all at once with `.deinit()`. Individual `free()` only works for most recent allocation: + +```zig +// CLI app pattern - allocate freely, free all at end +pub fn main() !void { + var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator); + defer arena.deinit(); + const allocator = arena.allocator(); + + const data = try allocator.alloc(u8, 1000); + const more = try allocator.alloc(u8, 2000); + // No need to free individual allocations +} + +// Request loop pattern - reset per iteration +var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator); +defer arena.deinit(); + +while (running) { + _ = arena.reset(.retain_capacity); // Keep memory, reset state + const allocator = arena.allocator(); + try handleRequest(allocator); +} +``` + +**Reset modes:** + +- `.free_all` - Release all memory to backing allocator +- `.retain_capacity` - Keep allocated pages for reuse (faster) +- `.{ .retain_with_limit = N }` - Retain up to N bytes + +**Query current usage:** +```zig +const bytes_used = arena.queryCapacity(); // Excludes internal overhead +``` + +**State optimization** - store just the state to save memory: +```zig +const State = std.heap.ArenaAllocator.State; +var state: State = .{}; + +// Promote to full allocator when needed +var arena = state.promote(std.heap.page_allocator); +defer arena.deinit(); +``` + +### DebugAllocator + +Detects leaks, double-free, use-after-free. Designed for safety over performance, but still faster than `page_allocator`. Safety checks and thread safety configurable: + +```zig +var gpa: std.heap.DebugAllocator(.{}) = .init; +defer { + const check = gpa.deinit(); + if (check == .leak) { + std.debug.print("Memory leak detected!\n", .{}); + } +} +const allocator = gpa.allocator(); +``` + +**Configuration options:** + +```zig +var gpa: std.heap.DebugAllocator(.{ + .stack_trace_frames = 10, // Capture more frames + .enable_memory_limit = true, // Track total bytes + .safety = true, // Enable safety checks + .thread_safe = true, // Multi-thread support + .never_unmap = true, // Debug use-after-free + .retain_metadata = true, // Better double-free detection +}) = .init; +``` + +### SmpAllocator + +Maximum performance for multithreaded ReleaseFast builds. Few safety features: + +```zig +const allocator = std.heap.smp_allocator; +const data = try allocator.alloc(u8, 1000); +allocator.free(data); +``` + +### C Allocator + +Alternative when `smp_allocator` is not available. Requires linking libc (`-lc`): + +```zig +const allocator = std.heap.c_allocator; +``` + +### Page Allocator + +Requests entire pages from OS via syscall. A 1-byte allocation reserves multiple kibibytes - inefficient for small allocations. Use as backing allocator for `ArenaAllocator` or `DebugAllocator`: + +```zig +const allocator = std.heap.page_allocator; +``` + +### MemoryPool + +Fast allocator for many objects of the same type. Outperforms general-purpose allocators when allocating/freeing objects in rapid succession: + +```zig +var pool = std.heap.MemoryPool(MyStruct).init(std.heap.page_allocator); +defer pool.deinit(); + +// Allocate objects (very fast) +const obj1 = try pool.create(); +const obj2 = try pool.create(); + +// Free returns to pool for reuse (not to backing allocator) +pool.destroy(obj1); + +// Reuses freed slot +const obj3 = try pool.create(); // likely same address as obj1 + +// Reset all - batch destroy without individual frees +_ = pool.reset(.retain_capacity); +``` + +**Options:** +```zig +// Pre-allocate slots +var pool = try std.heap.MemoryPool(T).initPreheated(allocator, 100); + +// Custom alignment +var pool = std.heap.MemoryPoolAligned(T, .@"64").init(allocator); + +// Non-growable (fixed capacity) +var pool = try std.heap.MemoryPoolExtra(T, .{ .growable = false }).initPreheated(allocator, 50); +``` + +### ThreadSafeAllocator + +Wraps any allocator with mutex for thread safety: + +```zig +var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator); +defer arena.deinit(); + +var ts = std.heap.ThreadSafeAllocator{ + .child_allocator = arena.allocator(), +}; +const allocator = ts.allocator(); // Safe to use from multiple threads +``` + +### StackFallbackAllocator + +Allocates from stack buffer first, falls back to another allocator when exhausted: + +```zig +var fallback = std.heap.stackFallback(4096, std.heap.page_allocator); +const allocator = fallback.get(); + +// First 4KB comes from stack (no heap allocation) +const small = try allocator.alloc(u8, 100); + +// Falls back to page_allocator if stack buffer exhausted +const large = try allocator.alloc(u8, 10000); +``` + +### raw_c_allocator + +Direct malloc/free without alignment overhead. Use as `ArenaAllocator` backing when linking libc: + +```zig +// More efficient than c_allocator when wrapping with ArenaAllocator +var arena = std.heap.ArenaAllocator.init(std.heap.raw_c_allocator); +defer arena.deinit(); +``` + +Requires linking libc. Does not support custom alignment - asserts alignment <= `@alignOf(std.c.max_align_t)`. + +### Wasm Allocator + +Optimized for WebAssembly. Uses `@wasmMemoryGrow`: + +```zig +const allocator = std.heap.wasm_allocator; // Only on wasm32/wasm64 +``` + +## Page Size Constants + +```zig +std.heap.page_size_min // Comptime minimum page size for target +std.heap.page_size_max // Comptime maximum page size for target +std.heap.pageSize() // Runtime page size (may be comptime if min == max) +``` + +## Passing Allocators + +**In libraries - accept allocator parameter:** + +```zig +pub fn MyContainer(comptime T: type) type { + return struct { + allocator: std.mem.Allocator, + data: []T, + + pub fn init(allocator: std.mem.Allocator) @This() { + return .{ .allocator = allocator, .data = &.{} }; + } + + pub fn deinit(self: *@This()) void { + if (self.data.len > 0) { + self.allocator.free(self.data); + } + } + + pub fn add(self: *@This(), item: T) !void { + // Use self.allocator for internal allocations + } + }; +} +``` + +**Functions returning allocated memory - document ownership:** + +```zig +/// Caller owns returned memory. +pub fn readFile(allocator: Allocator, path: []const u8) ![]u8 { + // ... + return try allocator.dupe(u8, content); +} + +// Caller must free: +const content = try readFile(allocator, "file.txt"); +defer allocator.free(content); +``` + +## Common Patterns + +### Wrapping Allocators (Sub-Allocators) + +```zig +// Arena on top of debug allocator +var gpa: std.heap.DebugAllocator(.{}) = .init; +defer _ = gpa.deinit(); + +var arena = std.heap.ArenaAllocator.init(gpa.allocator()); +defer arena.deinit(); + +const allocator = arena.allocator(); +``` + +### Temporary Allocations in Loops + +```zig +var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator); +defer arena.deinit(); + +for (items) |item| { + // Reset arena each iteration for automatic cleanup + _ = arena.reset(.retain_capacity); + const temp = try arena.allocator().alloc(u8, item.size); + // temp is automatically "freed" on next reset +} +``` + +### Sentinel-Terminated Allocations + +```zig +// Allocate with null terminator +const str = try allocator.allocSentinel(u8, len, 0); +defer allocator.free(str); + +// Duplicate with sentinel +const c_str = try allocator.dupeZ(u8, "hello"); // [:0]u8 +defer allocator.free(c_str); +``` + +## Error Handling + +Always handle `error.OutOfMemory`: + +```zig +// Option 1: Propagate +fn process(allocator: Allocator) !void { + const data = try allocator.alloc(u8, size); + defer allocator.free(data); +} + +// Option 2: Handle gracefully +fn process(allocator: Allocator) void { + const data = allocator.alloc(u8, size) catch { + log.err("Out of memory", .{}); + return; + }; + defer allocator.free(data); +} +``` + +## Zig 0.16 Notes + +Primary release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +- `std.heap.ArenaAllocator` is thread-safe and lock-free in Zig 0.16. +- `std.heap.ThreadSafe` allocator was removed. +- Allocators that perform blocking synchronization or file/entropy/time work should accept/store `std.Io` at initialization rather than constructing a local backend. +- Keep allocator naming by memory contract (`gpa`, `arena`, `scratch`) and expose stored `io` through a small accessor only when callsites need it. + +## Initialization (0.16.x) + +Use `.init` not `.{}`: + +```zig +// WRONG - deprecated +var gpa: std.heap.DebugAllocator(.{}) = .{}; + +// CORRECT +var gpa: std.heap.DebugAllocator(.{}) = .init; +``` + +## Debugging Memory Issues + +### Leak Detection + +```zig +test "check for leaks" { + // std.testing.allocator automatically reports leaks + var list: std.ArrayList(u32) = .empty; + try list.append(std.testing.allocator, 42); + // Missing: list.deinit(std.testing.allocator); + // Test will FAIL with leak report +} +``` + +### DebugAllocator in Main + +```zig +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer { + const check = gpa.deinit(); + if (check == .leak) @panic("memory leak"); + } + try run(gpa.allocator()); +} +``` + +## Implementing Custom Allocators + +Allocators implement `std.mem.Allocator.VTable`: + +```zig +const MyAllocator = struct { + // State fields here + + pub fn allocator(self: *MyAllocator) std.mem.Allocator { + return .{ + .ptr = self, + .vtable = &vtable, + }; + } + + const vtable: std.mem.Allocator.VTable = .{ + .alloc = alloc, + .resize = resize, + .remap = remap, + .free = free, + }; + + fn alloc(ctx: *anyopaque, len: usize, alignment: std.mem.Alignment, ra: usize) ?[*]u8 { + const self: *MyAllocator = @ptrCast(@alignCast(ctx)); + _ = ra; // return address for stack traces + // Return aligned pointer or null + } + + fn resize(ctx: *anyopaque, buf: []u8, alignment: std.mem.Alignment, new_len: usize, ra: usize) bool { + // Return true if resize succeeded in-place + } + + fn remap(ctx: *anyopaque, buf: []u8, alignment: std.mem.Alignment, new_len: usize, ra: usize) ?[*]u8 { + // Return new pointer (may move) or null if can't remap + } + + fn free(ctx: *anyopaque, buf: []u8, alignment: std.mem.Alignment, ra: usize) void { + // Free memory + } +}; +``` + +**Validation wrapper** - for testing allocators: +```zig +var my_alloc = MyAllocator.init(); +var validated = std.mem.validationWrap(my_alloc.allocator()); +const allocator = validated.allocator(); // Adds safety checks +``` diff --git a/references/std-array-hash-map.md b/references/std-array-hash-map.md new file mode 100644 index 0000000..aa356b3 --- /dev/null +++ b/references/std-array-hash-map.md @@ -0,0 +1,233 @@ +# Array Hash Map Migration (Zig 0.16.0) + +Primary release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 removed the managed array hash map aliases: + +- `std.ArrayHashMap` removed. +- `std.AutoArrayHashMap` removed. +- `std.StringArrayHashMap` removed. +- `std.AutoArrayHashMapUnmanaged` -> `std.array_hash_map.Auto` +- `std.StringArrayHashMapUnmanaged` -> `std.array_hash_map.String` +- `std.ArrayHashMapUnmanaged` -> `std.array_hash_map.Custom` + +Old examples below may use removed names; translate them to the `std.array_hash_map` names before using them in Zig 0.16 code. + +A hash map that preserves insertion order and stores keys/values in contiguous arrays. Combines hash table lookup with array-like iteration. + +## When to Use + +- Need deterministic iteration order (insertion order) +- Need array-style access to keys/values +- JSON object preservation +- When iteration performance matters more than removal performance + +## Variants + +| Type | Description | +|------|-------------| +| `AutoArrayHashMap(K, V)` | Auto-hashing for common key types | +| `ArrayHashMap(K, V, Ctx, store_hash)` | Custom hash/equal context | +| `StringArrayHashMap(V)` | String keys | +| `ArrayHashMapUnmanaged(...)` | No stored allocator | + +## Basic Usage + +```zig +const std = @import("std"); + +var map = std.AutoArrayHashMap(u32, []const u8).init(allocator); +defer map.deinit(); + +// Insert +try map.put(1, "one"); +try map.put(2, "two"); +try map.put(3, "three"); + +// Lookup +if (map.get(2)) |value| { + std.debug.print("2 = {s}\n", .{value}); +} + +// Check existence +if (map.contains(1)) { + // key exists +} +``` + +## Insertion Order Preserved + +```zig +try map.put(10, "ten"); +try map.put(5, "five"); +try map.put(15, "fifteen"); + +// Iteration is in insertion order: 10, 5, 15 +var it = map.iterator(); +while (it.next()) |entry| { + std.debug.print("{}: {s}\n", .{ entry.key_ptr.*, entry.value_ptr.* }); +} +``` + +## Array Access + +```zig +// Direct access to underlying arrays +const keys = map.keys(); // []K slice of all keys +const values = map.values(); // []V slice of all values + +// Access by index +for (keys, values) |k, v| { + std.debug.print("{}: {s}\n", .{ k, v }); +} +``` + +## Removal (Two Options) + +```zig +// O(1) removal - swaps with last element, changes order +_ = map.swapRemove(key); + +// O(n) removal - shifts elements, preserves order +_ = map.orderedRemove(key); + +// Fetch and remove +if (map.fetchSwapRemove(key)) |kv| { + std.debug.print("removed {}: {s}\n", .{ kv.key, kv.value }); +} +``` + +## Get or Put + +```zig +// Get existing or insert new +const result = try map.getOrPut(key); +if (!result.found_existing) { + result.value_ptr.* = "new_value"; +} + +// Get or put with default value +const result2 = try map.getOrPutValue(key, "default"); +``` + +## Index-Based Operations + +```zig +// Get index of key +if (map.getIndex(key)) |idx| { + // Remove by index + map.swapRemoveAt(idx); + // or + map.orderedRemoveAt(idx); +} +``` + +## Capacity Management + +```zig +try map.ensureTotalCapacity(100); +try map.ensureUnusedCapacity(10); + +const cap = map.capacity(); +const len = map.count(); + +map.clearRetainingCapacity(); +map.clearAndFree(); +``` + +## String Keys + +```zig +var map = std.StringArrayHashMap(i32).init(allocator); +defer map.deinit(); + +try map.put("apple", 1); +try map.put("banana", 2); + +// Keys are stored by reference, not copied +// Make sure string lifetime exceeds map usage +``` + +## Custom Context + +```zig +const CaseInsensitiveContext = struct { + pub fn hash(_: @This(), key: []const u8) u32 { + var h: u32 = 0; + for (key) |c| { + h = h *% 31 +% std.ascii.toLower(c); + } + return h; + } + pub fn eql(_: @This(), a: []const u8, b: []const u8, _: usize) bool { + return std.ascii.eqlIgnoreCase(a, b); + } +}; + +var map = std.ArrayHashMap( + []const u8, + i32, + CaseInsensitiveContext, + true, // store_hash for better performance +).initContext(allocator, .{}); +defer map.deinit(); + +try map.put("Hello", 1); +_ = map.get("HELLO"); // finds it! +``` + +## Complete Example: Word Counter + +```zig +const std = @import("std"); + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + + var counts = std.StringArrayHashMap(u32).init(gpa.allocator()); + defer counts.deinit(); + + const words = [_][]const u8{ "apple", "banana", "apple", "cherry", "banana", "apple" }; + + for (words) |word| { + const result = try counts.getOrPut(word); + if (result.found_existing) { + result.value_ptr.* += 1; + } else { + result.value_ptr.* = 1; + } + } + + // Print in insertion order + var it = counts.iterator(); + while (it.next()) |entry| { + std.debug.print("{s}: {}\n", .{ entry.key_ptr.*, entry.value_ptr.* }); + } + // Output (insertion order): + // apple: 3 + // banana: 2 + // cherry: 1 +} +``` + +## Comparison with HashMap + +| Feature | HashMap | ArrayHashMap | +|---------|---------|--------------| +| Lookup | O(1) | O(1) | +| Insert | O(1) amortized | O(1) amortized | +| swapRemove | O(1) | O(1) | +| orderedRemove | N/A | O(n) | +| Iteration order | Undefined | Insertion order | +| Key/value arrays | No | Yes | +| Memory layout | Scattered | Contiguous | + +## Notes + +- Iteration order equals insertion order +- `swapRemove` is O(1) but changes order +- `orderedRemove` preserves order but is O(n) +- Use `store_hash=true` when `eql` is expensive +- Keys/values are stored in `MultiArrayList` (cache-friendly) +- Pointer stability only guaranteed with pre-allocated capacity diff --git a/references/std-arraylist.md b/references/std-arraylist.md new file mode 100644 index 0000000..ae3c974 --- /dev/null +++ b/references/std-arraylist.md @@ -0,0 +1,165 @@ +# std.ArrayList (Zig 0.16.0) + +Dynamic array (vector) that grows as needed. + +**Note:** `std.ArrayListUnmanaged` is deprecated - use `std.ArrayList` (unmanaged-style API with allocator passed to methods). + +## Initialization + +```zig +// CRITICAL: Use .empty, not .{} +var list: std.ArrayList(u32) = .empty; +defer list.deinit(allocator); + +// With pre-allocated capacity +var list = try std.ArrayList(u32).initCapacity(allocator, 100); + +// From existing slice (takes ownership) +var list = std.ArrayList(u32).fromOwnedSlice(existing_slice); + +// Fixed buffer (no allocator needed for operations) +var buffer: [8]i32 = undefined; +var stack = std.ArrayList(i32).initBuffer(&buffer); +``` + +## Basic Operations + +```zig +// Append +try list.append(allocator, 42); +try list.appendSlice(allocator, &[_]u32{1, 2, 3}); + +// Append without allocation (asserts capacity exists) +list.appendAssumeCapacity(42); +list.appendSliceAssumeCapacity(&[_]u32{1, 2, 3}); + +// Access items +const items = list.items; // []T slice +const first = list.items[0]; +const last = list.getLast(); // returns ?T +const popped = list.pop(); // returns ?T, removes last + +// Insert at index +try list.insert(allocator, 2, value); +try list.insertSlice(allocator, 2, slice); + +// Remove +const removed = list.orderedRemove(index); // O(n), preserves order +const removed = list.swapRemove(index); // O(1), doesn't preserve order +``` + +## Capacity Management + +```zig +// Ensure space for N more items +try list.ensureUnusedCapacity(allocator, 10); + +// Ensure total capacity is at least N +try list.ensureTotalCapacity(allocator, 100); + +// Shrink to fit +list.shrinkAndFree(allocator, list.items.len); + +// Clear +list.clearRetainingCapacity(); // keeps memory +list.clearAndFree(allocator); // frees memory +``` + +## Ownership Transfer + +```zig +// Get owned slice (empties list, caller owns memory) +const owned = try list.toOwnedSlice(allocator); +defer allocator.free(owned); + +// Get null-terminated slice +const z_str = try list.toOwnedSliceSentinel(allocator, 0); +``` + +## Iteration + +```zig +for (list.items) |item| { + // read-only +} + +for (list.items) |*item| { + item.* += 1; // modify in place +} + +for (list.items, 0..) |item, i| { + // with index +} +``` + +## Common Patterns + +```zig +// Collect from iterator +var list: std.ArrayList(u8) = .empty; +for (some_iterator) |item| { + try list.append(allocator, item); +} + +// Build string +var buf: std.ArrayList(u8) = .empty; +try buf.appendSlice(allocator, "Hello "); +try buf.appendSlice(allocator, name); +const result = try buf.toOwnedSlice(allocator); + +// Remove while iterating (iterate backwards) +var i: usize = list.items.len; +while (i > 0) { + i -= 1; + if (shouldRemove(list.items[i])) { + _ = list.swapRemove(i); + } +} +``` + +## Reserve-First Pattern (Exception Safety) + +When inserting into multiple containers or when partial mutation would corrupt state, use **reserve-first**: separate fallible reservation from infallible mutation. + +```zig +// BAD - partial failure leaves invalid state +fn addItem(list: *std.ArrayList(u32), map: *std.AutoHashMap(u32, usize), gpa: Allocator, value: u32) !void { + try list.append(gpa, value); // Can fail + try map.put(gpa, value, list.items.len); // If this fails, list has orphan entry! +} + +// GOOD - reserve first, then mutate +fn addItem(list: *std.ArrayList(u32), map: *std.AutoHashMap(u32, usize), gpa: Allocator, value: u32) !void { + // Phase 1: Reserve (fallible, but no mutation) + try list.ensureUnusedCapacity(gpa, 1); + try map.ensureUnusedCapacity(gpa, 1); + + errdefer comptime unreachable; // Phase 2: No errors after this point + + // Phase 3: Mutate (infallible) + list.appendAssumeCapacity(value); + map.getOrPutAssumeCapacity(value).value_ptr.* = list.items.len; +} +``` + +**Key methods:** +- `ensureUnusedCapacity(gpa, n)` - Reserve space for n more items (can fail, doesn't mutate) +- `appendAssumeCapacity(item)` - Append without allocation (cannot fail, asserts capacity) +- `appendSliceAssumeCapacity(items)` - Append slice without allocation + +See **[Reserve-First Exception Safety](patterns.md#reserve-first-exception-safety)** for detailed explanation and real-world examples. + +## BoundedArray Replacement + +`std.BoundedArray` was REMOVED in 0.15.x. Use `initBuffer` instead: + +```zig +// OLD (removed) +var arr = std.BoundedArray(u8, 64){}; + +// NEW +var buffer: [64]u8 = undefined; +var arr = std.ArrayList(u8).initBuffer(&buffer); +// Note: Operations will panic if capacity exceeded +try arr.appendBounded(value); // returns error.OutOfMemory if full +``` diff --git a/references/std-ascii.md b/references/std-ascii.md new file mode 100644 index 0000000..dabcdda --- /dev/null +++ b/references/std-ascii.md @@ -0,0 +1,144 @@ +# std.ascii + +7-bit ASCII character classification and manipulation. For Unicode handling, use `std.unicode`. + +## Character Classification + +```zig +const std = @import("std"); +const ascii = std.ascii; + +// Character type checks (all return bool) +ascii.isAlphanumeric('a') // A-Z, a-z, 0-9 +ascii.isAlphabetic('a') // A-Z, a-z +ascii.isDigit('5') // 0-9 +ascii.isHex('F') // A-F, a-f, 0-9 +ascii.isUpper('A') // A-Z +ascii.isLower('a') // a-z +ascii.isWhitespace(' ') // space, \t, \n, \r, \v, \f +ascii.isPrint('!') // printable (not control) +ascii.isControl('\n') // control characters (0x00-0x1F, 0x7F) +ascii.isAscii(c) // c < 128 +``` + +## Case Conversion + +```zig +// Single character +ascii.toUpper('a') // 'A' +ascii.toLower('A') // 'a' + +// Strings - to buffer +var buf: [100]u8 = undefined; +const lower = ascii.lowerString(&buf, "HeLLo"); // "hello" +const upper = ascii.upperString(&buf, "HeLLo"); // "HELLO" + +// Strings - allocating +const lower = try ascii.allocLowerString(allocator, "HeLLo"); +defer allocator.free(lower); // "hello" + +const upper = try ascii.allocUpperString(allocator, "HeLLo"); +defer allocator.free(upper); // "HELLO" +``` + +## Case-Insensitive Comparison + +```zig +// Equality +ascii.eqlIgnoreCase("Hello", "HELLO") // true + +// Prefix/suffix +ascii.startsWithIgnoreCase("Hello World", "hello") // true +ascii.endsWithIgnoreCase("Hello World", "WORLD") // true + +// Search +ascii.indexOfIgnoreCase("Hello World", "world") // ?usize = 6 + +// Lexicographical order +ascii.orderIgnoreCase("abc", "ABC") // .eq +ascii.lessThanIgnoreCase("abc", "abd") // true +``` + +## Constants + +```zig +// Character sets (as strings) +ascii.lowercase // "abcdefghijklmnopqrstuvwxyz" +ascii.uppercase // "ABCDEFGHIJKLMNOPQRSTUVWXYZ" +ascii.letters // lowercase ++ uppercase + +// Whitespace array (for use with std.mem.trim) +ascii.whitespace // [_]u8{ ' ', '\t', '\n', '\r', '\v', '\f' } +``` + +## Control Codes + +```zig +const cc = std.ascii.control_code; + +// Common control codes +cc.nul // 0x00 Null +cc.bel // 0x07 Bell +cc.bs // 0x08 Backspace +cc.ht // 0x09 Horizontal Tab (\t) +cc.lf // 0x0A Line Feed (\n) +cc.vt // 0x0B Vertical Tab +cc.ff // 0x0C Form Feed +cc.cr // 0x0D Carriage Return (\r) +cc.esc // 0x1B Escape +cc.del // 0x7F Delete + +// Flow control +cc.xon // 0x11 XON (alias for dc1) +cc.xoff // 0x13 XOFF (alias for dc3) +``` + +## Hex Escape Formatting + +Format bytes with non-printable characters escaped: + +```zig +const data = "hello\xffworld"; + +// Format with hex escapes for non-printable bytes +try stdout.print("{f}\n", .{ascii.hexEscape(data, .lower)}); +// Output: hello\xffworld + +try stdout.print("{f}\n", .{ascii.hexEscape(data, .upper)}); +// Output: hello\xFFworld +``` + +## Common Patterns + +### Trim whitespace +```zig +const trimmed = std.mem.trim(u8, " hello ", &ascii.whitespace); +// "hello" +``` + +### Validate ASCII string +```zig +fn isAsciiString(s: []const u8) bool { + for (s) |c| { + if (!ascii.isAscii(c)) return false; + } + return true; +} +``` + +### Case-insensitive map lookup +```zig +// Use ascii.lowerString to normalize keys +var buf: [64]u8 = undefined; +const normalized = ascii.lowerString(&buf, user_input); +if (map.get(normalized)) |value| { + // found +} +``` + +## Notes + +- All functions handle bytes > 127 gracefully (return `false` for classification) +- Functions use `u8` not `u7` for convenience +- For Unicode text, use `std.unicode` instead +- `lowerString`/`upperString` assert output buffer is large enough diff --git a/references/std-atomic.md b/references/std-atomic.md new file mode 100644 index 0000000..60afa6e --- /dev/null +++ b/references/std-atomic.md @@ -0,0 +1,416 @@ +# std.atomic - Atomic Operations Reference (Zig 0.16.0) + +Lock-free atomic operations for concurrent programming. Wraps Zig's atomic builtins with a type-safe interface. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Lock-free atomics do not need `std.Io`. Blocking synchronization should use `std.Io.Mutex`, `std.Io.Condition`, `std.Io.Semaphore`, `std.Io.RwLock`, `std.Io.Event`, or `std.Io.Group` when used in I/O-aware code. + +## Table of Contents +- [Module Structure](#module-structure) +- [Atomic Value Wrapper](#atomic-value-wrapper) +- [Atomic Operations](#atomic-operations) +- [Atomic Ordering](#atomic-ordering) +- [Spin Loop Hint](#spin-loop-hint) +- [Cache Line Size](#cache-line-size) +- [Common Patterns](#common-patterns) + +## Module Structure + +```zig +std.atomic.Value(T) // Atomic wrapper for T (integers, enums, floats, bools, pointers) +std.atomic.spinLoopHint() // CPU hint for spin-wait loops +std.atomic.cache_line // CPU cache line size (comptime constant) +``` + +## Atomic Value Wrapper + +`std.atomic.Value(T)` wraps a value to enable atomic operations. Supported types: integers, enums, floats, bools, optional pointers. + +### Creation + +```zig +const std = @import("std"); + +// Initialize with value +var counter = std.atomic.Value(u64).init(0); + +// Initialize in struct +const State = struct { + count: std.atomic.Value(usize) = std.atomic.Value(usize).init(0), + flag: std.atomic.Value(bool) = std.atomic.Value(bool).init(false), +}; + +// Direct access (careful - not atomic!) +var x = std.atomic.Value(u32).init(10); +x.raw = 20; // Non-atomic write - use only when no concurrent access +``` + +### Basic Operations + +```zig +var x = std.atomic.Value(u32).init(5); + +// Load (atomic read) +const val = x.load(.acquire); + +// Store (atomic write) +x.store(10, .release); + +// Swap (exchange, returns old value) +const old = x.swap(20, .seq_cst); // old = 10, x = 20 +``` + +## Atomic Operations + +### Fetch-and-Modify Operations + +All return the **previous** value before modification: + +```zig +var x = std.atomic.Value(i32).init(10); + +// Arithmetic +_ = x.fetchAdd(5, .seq_cst); // x = 15, returns 10 +_ = x.fetchSub(3, .seq_cst); // x = 12, returns 15 +_ = x.fetchMin(8, .seq_cst); // x = 8, returns 12 +_ = x.fetchMax(20, .seq_cst); // x = 20, returns 8 + +// Bitwise +var bits = std.atomic.Value(u8).init(0b1100); +_ = bits.fetchAnd(0b1010, .seq_cst); // 0b1000 +_ = bits.fetchOr(0b0011, .seq_cst); // 0b1011 +_ = bits.fetchXor(0b1111, .seq_cst); // 0b0100 +_ = bits.fetchNand(0b1100, .seq_cst); // ~(0b0100 & 0b1100) = ~0b0100 + +// Generic RMW (any AtomicRmwOp) +_ = x.rmw(.Add, 1, .seq_cst); +``` + +### Compare-and-Swap (CAS) + +```zig +var x = std.atomic.Value(u32).init(100); + +// Strong CAS - guaranteed to succeed if values match +const result = x.cmpxchgStrong(100, 200, .seq_cst, .seq_cst); +// result = null (success, x is now 200) +// result = 100 (failure, current value if mismatch) + +// Weak CAS - may spuriously fail, use in loops +var current: u32 = x.load(.acquire); +while (x.cmpxchgWeak(current, current + 1, .acq_rel, .acquire)) |actual| { + current = actual; // Retry with actual value +} +``` + +**When to use which:** +- `cmpxchgStrong`: Single attempt, no retry loop +- `cmpxchgWeak`: In retry loops (more efficient on some architectures) + +### Bit Operations + +Individual bit manipulation, returning previous bit state: + +```zig +var flags = std.atomic.Value(u32).init(0); + +// Set bit (returns previous bit value: 0 or 1) +const was_set = flags.bitSet(3, .seq_cst); // Set bit 3 + +// Reset bit +const was_reset = flags.bitReset(3, .seq_cst); // Clear bit 3 + +// Toggle bit +const prev = flags.bitToggle(5, .seq_cst); // Flip bit 5 +``` + +## Atomic Ordering + +Memory orderings control synchronization guarantees. From `std.builtin.AtomicOrder`: + +| Order | Guarantees | Use Case | +|-------|------------|----------| +| `.unordered` | No ordering, loads/stores only (no RMW) | Preventing torn reads/writes only (e.g., data inside SeqLock) | +| `.monotonic` | Coherent on same variable, reorderable with other atomics | Simple counters, progress indicators | +| `.acquire` | Subsequent reads/writes won't move before this | Loading shared data after flag check | +| `.release` | Prior reads/writes won't move after this | Publishing data before setting flag | +| `.acq_rel` | Both acquire and release | Read-modify-write on shared data | +| `.seq_cst` | Total order among all seq_cst operations | When multiple atomics must be globally ordered | + +### Ordering Guidelines + +```zig +// Producer-consumer pattern +var data: Data = undefined; +var ready = std.atomic.Value(bool).init(false); + +// Producer thread +fn produce() void { + data = computeData(); // Write data first + ready.store(true, .release); // Then publish (release ensures order) +} + +// Consumer thread +fn consume() void { + while (!ready.load(.acquire)) { // Acquire synchronizes with release + std.atomic.spinLoopHint(); + } + useData(data); // Safe to read after acquire +} +``` + +**Rules of thumb:** +- `.monotonic` for counters that don't guard other data (can still reorder with other atomics) +- `.release` when publishing/storing data that others will read +- `.acquire` when consuming/loading data others published +- `.acq_rel` for RMW operations that both read and write shared state +- `.seq_cst` when multiple atomics must have a single global order visible to all threads (only orders with other seq_cst ops) + +## Spin Loop Hint + +`spinLoopHint()` tells the CPU it's in a spin-wait loop, improving power efficiency and SMT performance: + +```zig +fn spinWait(flag: *std.atomic.Value(bool)) void { + while (!flag.load(.acquire)) { + std.atomic.spinLoopHint(); // Reduce power, yield to sibling threads + } +} +``` + +Architecture-specific behavior: +- **x86/x86_64**: `pause` instruction +- **AArch64**: `isb` instruction +- **ARM**: `yield` instruction (v6k+) +- **RISC-V**: `pause` (Zihintpause extension) +- **Others**: No-op + +## Cache Line Size + +`cache_line` is the CPU cache line size, used to prevent false sharing: + +```zig +const cache_line = std.atomic.cache_line; // 64, 128, etc. + +// Pad struct to avoid false sharing between threads +const PaddedCounter = struct { + value: std.atomic.Value(u64) align(cache_line) = std.atomic.Value(u64).init(0), + _padding: [cache_line - @sizeOf(std.atomic.Value(u64))]u8 = undefined, +}; + +// Per-thread counters without false sharing +const ThreadCounters = struct { + counters: [MAX_THREADS]PaddedCounter = [_]PaddedCounter{.{}} ** MAX_THREADS, +}; +``` + +Typical values by architecture: +- x86_64, AArch64: 128 bytes (big cores) +- ARM, MIPS: 32 bytes +- Most others: 64 bytes + +## Common Patterns + +### Thread-Safe Counter + +```zig +const Counter = struct { + value: std.atomic.Value(u64) = std.atomic.Value(u64).init(0), + + pub fn increment(self: *@This()) void { + _ = self.value.fetchAdd(1, .monotonic); + } + + pub fn decrement(self: *@This()) void { + _ = self.value.fetchSub(1, .monotonic); + } + + pub fn get(self: *const @This()) u64 { + return self.value.load(.monotonic); + } +}; +``` + +### Reference Counting + +```zig +const RefCounted = struct { + ref_count: std.atomic.Value(usize), + data: *Data, + + pub fn retain(self: *@This()) void { + _ = self.ref_count.fetchAdd(1, .monotonic); + } + + pub fn release(self: *@This()) void { + // Release ensures writes before release are visible to thread that sees 1 + if (self.ref_count.fetchSub(1, .release) == 1) { + // Acquire synchronizes with all previous releases + _ = self.ref_count.load(.acquire); + self.destroy(); + } + } +}; +``` + +### Lock-Free Stack (Treiber Stack) + +**Note:** This simplified implementation only supports single-consumer (one thread calling `pop()` at a time). Multiple concurrent poppers require ABA protection (hazard pointers, epoch-based reclamation, or tagged pointers) to safely read `node.next` without use-after-free. + +```zig +fn Stack(comptime T: type) type { + return struct { + const Node = struct { + value: T, + next: ?*Node, + }; + + head: std.atomic.Value(?*Node) = std.atomic.Value(?*Node).init(null), + + pub fn push(self: *@This(), node: *Node) void { + var current_head = self.head.load(.acquire); + while (true) { + node.next = current_head; + if (self.head.cmpxchgWeak(current_head, node, .release, .acquire)) |actual| { + current_head = actual; + } else { + break; // Success + } + } + } + + // Single-consumer only! See note above. + pub fn pop(self: *@This()) ?*Node { + var current_head = self.head.load(.acquire); + while (current_head) |node| { + if (self.head.cmpxchgWeak(current_head, node.next, .acq_rel, .acquire)) |actual| { + current_head = actual; + } else { + return node; // Success + } + } + return null; // Empty + } + }; +} +``` + +### Once Initialization (Double-Checked Locking) + +```zig +var initialized = std.atomic.Value(bool).init(false); +var init_mutex: std.Thread.Mutex = .{}; +var global_resource: ?*Resource = null; + +fn getResource() *Resource { + // Fast path: already initialized + if (initialized.load(.acquire)) { + return global_resource.?; + } + + // Slow path: initialize with lock + init_mutex.lock(); + defer init_mutex.unlock(); + + if (!initialized.load(.acquire)) { + global_resource = initializeResource(); + initialized.store(true, .release); + } + + return global_resource.?; +} +``` + +### Spin Lock + +```zig +const SpinLock = struct { + locked: std.atomic.Value(bool) = std.atomic.Value(bool).init(false), + + pub fn lock(self: *@This()) void { + while (self.locked.swap(true, .acquire)) { + while (self.locked.load(.monotonic)) { + std.atomic.spinLoopHint(); + } + } + } + + pub fn unlock(self: *@This()) void { + self.locked.store(false, .release); + } + + pub fn tryLock(self: *@This()) bool { + return !self.locked.swap(true, .acquire); + } +}; +``` + +### Progress Flag (SeqLock) + +**Note:** `Data` must be a type that supports atomic load/store (integers, bools, enums, floats, pointers). For larger structs, use a pointer or different synchronization. + +```zig +const SeqLock = struct { + seq: std.atomic.Value(u64) = std.atomic.Value(u64).init(0), + data: Data = .{}, + + // Single-writer only! + pub fn write(self: *@This(), new_data: Data) void { + // Odd sequence = write in progress + _ = self.seq.fetchAdd(1, .release); + @atomicStore(Data, &self.data, new_data, .unordered); + _ = self.seq.fetchAdd(1, .release); + } + + pub fn read(self: *@This()) Data { + while (true) { + const seq1 = self.seq.load(.acquire); + if (seq1 & 1 != 0) { + std.atomic.spinLoopHint(); + continue; // Write in progress + } + const data = @atomicLoad(Data, &self.data, .unordered); + const seq2 = self.seq.load(.acquire); + if (seq1 == seq2) return data; + std.atomic.spinLoopHint(); + } + } +}; +``` + +### Barrier Synchronization + +```zig +const Barrier = struct { + counter: std.atomic.Value(usize), + generation: std.atomic.Value(usize) = std.atomic.Value(usize).init(0), + total: usize, + + pub fn init(count: usize) @This() { + return .{ + .counter = std.atomic.Value(usize).init(count), + .total = count, + }; + } + + pub fn wait(self: *@This()) void { + const gen = self.generation.load(.acquire); + if (self.counter.fetchSub(1, .acq_rel) == 1) { + // Last thread to arrive + self.counter.store(self.total, .release); + _ = self.generation.fetchAdd(1, .release); + } else { + // Wait for generation to change + while (self.generation.load(.acquire) == gen) { + std.atomic.spinLoopHint(); + } + } + } +}; +``` + +## See Also + +- **[std.Thread](std-thread.md)** - Higher-level synchronization (Mutex, RwLock, Condition, Semaphore) +- **[std.Thread.Futex](std-thread.md)** - OS-level blocking primitives diff --git a/references/std-base64.md b/references/std-base64.md new file mode 100644 index 0000000..9c3073a --- /dev/null +++ b/references/std-base64.md @@ -0,0 +1,144 @@ +# std.base64 (Zig 0.16.0) + +Base64 encoding/decoding per RFC 4648. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +For examples that write encoded output to files/stdout, use `std.Io.Writer` and `std.Io.File.stdout().writer(io, &buf)`. + +## Quick Reference + +| Codec | Use Case | +|-------|----------| +| `standard` | Standard Base64 with `=` padding (email, MIME) | +| `standard_no_pad` | Standard Base64 without padding | +| `url_safe` | URL-safe Base64 with `=` padding | +| `url_safe_no_pad` | URL-safe Base64 without padding (JWT, URLs) | + +## Encoding + +```zig +const std = @import("std"); +const base64 = std.base64; + +const data = "Hello, World!"; + +// Standard Base64 (with padding) +var buf: [100]u8 = undefined; +const encoded = base64.standard.Encoder.encode(&buf, data); +// "SGVsbG8sIFdvcmxkIQ==" + +// URL-safe without padding (common for JWTs) +const encoded = base64.url_safe_no_pad.Encoder.encode(&buf, data); +// "SGVsbG8sIFdvcmxkIQ" + +// Calculate required buffer size +const size = base64.standard.Encoder.calcSize(data.len); +``` + +## Decoding + +```zig +const encoded = "SGVsbG8sIFdvcmxkIQ=="; + +// Decode to buffer +var buf: [100]u8 = undefined; +const decoded_len = try base64.standard.Decoder.calcSizeForSlice(encoded); +const decoded = buf[0..decoded_len]; +try base64.standard.Decoder.decode(decoded, encoded); +// decoded = "Hello, World!" + +// Calculate max decoded size (before knowing padding) +const max_size = try base64.standard.Decoder.calcSizeUpperBound(encoded.len); +``` + +## Decoding with Ignored Characters + +Decode Base64 that contains whitespace or other characters to ignore: + +```zig +const encoded = "SGVs bG8s\nIFdv cmxk IQ=="; // with spaces and newlines + +// Create decoder that ignores whitespace +const decoder = base64.standard.decoderWithIgnore(" \n"); + +var buf: [100]u8 = undefined; +const max_size = try decoder.calcSizeUpperBound(encoded.len); +const decoded_len = try decoder.decode(buf[0..max_size], encoded); +const decoded = buf[0..decoded_len]; +// "Hello, World!" +``` + +## Streaming Encoding + +```zig +var buf: [4096]u8 = undefined; +var writer = std.Io.File.stdout().writer(io, &buf); + +try base64.standard.Encoder.encodeWriter(&writer.interface, data); +try writer.interface.flush(); +``` + +## Codecs Detail + +```zig +// Standard alphabet: A-Z, a-z, 0-9, +, / +base64.standard // with = padding +base64.standard_no_pad // without padding + +// URL-safe alphabet: A-Z, a-z, 0-9, -, _ +base64.url_safe // with = padding +base64.url_safe_no_pad // without padding + +// Access alphabet characters directly +base64.standard_alphabet_chars // [64]u8 +base64.url_safe_alphabet_chars // [64]u8 +``` + +## Error Handling + +```zig +base64.standard.Decoder.decode(dest, source) catch |err| switch (err) { + error.InvalidCharacter => // character not in alphabet + error.InvalidPadding => // incorrect padding + error.NoSpaceLeft => // dest buffer too small (DecoderWithIgnore only) +}; +``` + +## Common Patterns + +### Encode binary data for JSON/URLs +```zig +fn encodeForUrl(data: []const u8, buf: []u8) []const u8 { + return std.base64.url_safe_no_pad.Encoder.encode(buf, data); +} +``` + +### Decode JWT payload +```zig +fn decodeJwtPayload(payload: []const u8, buf: []u8) ![]u8 { + const decoder = std.base64.url_safe_no_pad.Decoder; + const size = try decoder.calcSizeForSlice(payload); + try decoder.decode(buf[0..size], payload); + return buf[0..size]; +} +``` + +### Handle multi-line Base64 (PEM format) +```zig +fn decodePem(pem_data: []const u8, buf: []u8) ![]u8 { + // Skip header/footer, decode with newline ignoring + const decoder = std.base64.standard.decoderWithIgnore("\n\r"); + const max = try decoder.calcSizeUpperBound(pem_data.len); + const len = try decoder.decode(buf[0..max], pem_data); + return buf[0..len]; +} +``` + +## Notes + +- Standard uses `+` and `/` which need URL encoding +- URL-safe uses `-` and `_` which are safe in URLs +- Padding (`=`) makes length divisible by 4 +- `calcSizeForSlice` gives exact size; `calcSizeUpperBound` gives max (ignores padding) +- All codecs use little-endian byte order internally diff --git a/references/std-bit-set.md b/references/std-bit-set.md new file mode 100644 index 0000000..e064e06 --- /dev/null +++ b/references/std-bit-set.md @@ -0,0 +1,211 @@ +# std.bit_set + +Densely stored sets of integers with efficient set operations (union, intersection, complement). Each integer gets a single bit. + +## When to Use + +- Track presence/absence of items from a known finite set +- Set operations (union, intersection, difference) +- Bit flags with variable size +- Compact storage when max value is known + +## Variants + +| Type | Size | Allocation | +|------|------|------------| +| `IntegerBitSet(N)` | Compile-time, N <= 128 | None (single integer) | +| `ArrayBitSet(usize, N)` | Compile-time, any N | None (array) | +| `StaticBitSet(N)` | Compile-time | Auto-selects Integer or Array | +| `DynamicBitSet` | Runtime | Allocator (managed) | +| `DynamicBitSetUnmanaged` | Runtime | Allocator (unmanaged) | + +## Static Bit Set (Compile-Time Size) + +```zig +const std = @import("std"); + +// StaticBitSet auto-selects best implementation +const Flags = std.StaticBitSet(64); + +var flags = Flags.initEmpty(); +flags.set(5); +flags.set(10); + +if (flags.isSet(5)) { + // bit 5 is set +} + +flags.unset(5); +flags.toggle(10); + +const count = flags.count(); // number of set bits +``` + +## Dynamic Bit Set (Runtime Size) + +```zig +var bits = try std.DynamicBitSet.initEmpty(allocator, 1000); +defer bits.deinit(); + +bits.set(42); +bits.set(100); + +// Resize dynamically +try bits.resize(2000, false); // false = new bits are 0 +try bits.resize(2000, true); // true = new bits are 1 + +// Clone +var copy = try bits.clone(allocator); +defer copy.deinit(); +``` + +## Set Operations + +```zig +var a = Flags.initEmpty(); +var b = Flags.initEmpty(); +a.set(1); a.set(2); +b.set(2); b.set(3); + +// In-place operations (modify a) +a.setUnion(b); // a = a | b (bits in either) +a.setIntersection(b); // a = a & b (bits in both) +a.toggleSet(b); // a = a ^ b (flip bits that are in b) + +// Return new set (pure functions) +const union_set = a.unionWith(b); +const intersection = a.intersectWith(b); +const xor_set = a.xorWith(b); +const diff = a.differenceWith(b); // a - b +const comp = a.complement(); // ~a +``` + +## Comparison + +```zig +if (a.eql(b)) { + // same bits set +} + +if (a.subsetOf(b)) { + // all bits in a are also in b +} + +if (a.supersetOf(b)) { + // all bits in b are also in a +} +``` + +## Iteration + +```zig +var flags = Flags.initEmpty(); +flags.set(1); flags.set(5); flags.set(10); + +// Iterate set bits (ascending order by default) +var it = flags.iterator(.{}); +while (it.next()) |index| { + std.debug.print("bit {} is set\n", .{index}); +} + +// Iterate unset bits +var unset_it = flags.iterator(.{ .kind = .unset }); + +// Reverse order +var rev_it = flags.iterator(.{ .direction = .reverse }); +``` + +## Range Operations + +```zig +// Set/unset a range of bits +flags.setRangeValue(.{ .start = 10, .end = 20 }, true); // set bits 10-19 +flags.setRangeValue(.{ .start = 10, .end = 20 }, false); // unset bits 10-19 +``` + +## Find Operations + +```zig +// Find first/last set bit +if (flags.findFirstSet()) |index| { + // index of lowest set bit +} + +if (flags.findLastSet()) |index| { + // index of highest set bit +} + +// Find and toggle (atomic-like) +if (flags.toggleFirstSet()) |index| { + // returns index and unsets the bit +} +``` + +## Toggle All + +```zig +flags.toggleAll(); // flip every bit +``` + +## Unmanaged Dynamic BitSet + +```zig +// For when you don't want to store the allocator +var bits: std.DynamicBitSetUnmanaged = .{}; +try bits.resize(allocator, 100, false); +defer bits.deinit(allocator); + +bits.set(50); +``` + +## Complete Example: Permission Flags + +```zig +const std = @import("std"); + +const Permission = enum(u8) { + read = 0, + write = 1, + execute = 2, + delete = 3, + admin = 4, +}; + +const Permissions = std.StaticBitSet(8); + +fn hasPermission(perms: Permissions, p: Permission) bool { + return perms.isSet(@intFromEnum(p)); +} + +fn grant(perms: *Permissions, p: Permission) void { + perms.set(@intFromEnum(p)); +} + +fn revoke(perms: *Permissions, p: Permission) void { + perms.unset(@intFromEnum(p)); +} + +pub fn main() void { + var user_perms = Permissions.initEmpty(); + grant(&user_perms, .read); + grant(&user_perms, .write); + + var admin_perms = Permissions.initFull(); + + // Check if user has all admin permissions + if (user_perms.subsetOf(admin_perms)) { + // user can do everything admin can (not in this case) + } + + // Grant user all of admin's permissions + user_perms.setUnion(admin_perms); +} +``` + +## Notes + +- `StaticBitSet` is zero-allocation, copyable by value +- `DynamicBitSet` requires allocation, call `deinit()` +- `initFull()` creates set with all bits set +- Iteration order is index order, not insertion order +- Use `std.enums.EnumSet` for enum-based bit flags diff --git a/references/std-buf-map.md b/references/std-buf-map.md new file mode 100644 index 0000000..db02007 --- /dev/null +++ b/references/std-buf-map.md @@ -0,0 +1,176 @@ +# std.BufMap / std.BufSet + +String-keyed maps and sets that own their strings. Automatically copy and free string keys/values. + +## When to Use + +- Environment variable storage +- String-to-string mapping with ownership +- Set of unique strings with automatic memory management +- When you don't want to manage string lifetime manually + +## BufMap (String -> String) + +```zig +const std = @import("std"); + +var map = std.BufMap.init(allocator); +defer map.deinit(); // frees all stored strings + +// Put (copies both key and value) +try map.put("HOME", "/Users/alice"); +try map.put("PATH", "/usr/bin"); + +// Get +if (map.get("HOME")) |home| { + std.debug.print("home: {s}\n", .{home}); +} + +// Get pointer (invalidated on resize) +if (map.getPtr("PATH")) |path_ptr| { + path_ptr.* = try map.copy("/new/path"); // update in place +} + +// Remove (frees both key and value) +map.remove("PATH"); + +// Count +const n = map.count(); +``` + +## BufMap: Move Ownership + +```zig +// putMove takes ownership instead of copying +const key = try allocator.dupe(u8, "MY_KEY"); +const value = try allocator.dupe(u8, "my_value"); +try map.putMove(key, value); +// Don't free key/value - map owns them now +``` + +## BufMap: Iteration + +```zig +var it = map.iterator(); +while (it.next()) |entry| { + const key = entry.key_ptr.*; + const value = entry.value_ptr.*; + std.debug.print("{s}={s}\n", .{ key, value }); +} +``` + +## BufSet (Set of Strings) + +```zig +var set = std.BufSet.init(allocator); +defer set.deinit(); // frees all stored strings + +// Insert (copies the string) +try set.insert("apple"); +try set.insert("banana"); +try set.insert("apple"); // no-op, already exists + +// Check membership +if (set.contains("apple")) { + // it's in the set +} + +// Remove (frees the string) +set.remove("banana"); + +// Count +const n = set.count(); +``` + +## BufSet: Iteration + +```zig +var it = set.iterator(); +while (it.next()) |key| { + std.debug.print("{s}\n", .{key.*}); +} +``` + +## BufSet: Clone + +```zig +// Create independent copy +var copy = try set.clone(); +defer copy.deinit(); + +// Clone with different allocator +var arena_copy = try set.cloneWithAllocator(arena.allocator()); +// No need to deinit if using arena +``` + +## Complete Example: Environment Variables + +```zig +const std = @import("std"); + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + const alloc = gpa.allocator(); + + var env = std.BufMap.init(alloc); + defer env.deinit(); + + // Set some variables + try env.put("APP_NAME", "MyApp"); + try env.put("APP_VERSION", "1.0.0"); + try env.put("DEBUG", "true"); + + // Update a value + try env.put("DEBUG", "false"); // replaces, frees old value + + // Print all + var it = env.iterator(); + while (it.next()) |entry| { + std.debug.print("{s}={s}\n", .{ entry.key_ptr.*, entry.value_ptr.* }); + } + + // Check and use + if (env.get("DEBUG")) |debug| { + if (std.mem.eql(u8, debug, "true")) { + std.debug.print("Debug mode enabled\n", .{}); + } + } +} +``` + +## Complete Example: Unique Words + +```zig +const std = @import("std"); + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + + var words = std.BufSet.init(gpa.allocator()); + defer words.deinit(); + + const text = "the quick brown fox jumps over the lazy dog"; + var tokens = std.mem.tokenizeScalar(u8, text, ' '); + + while (tokens.next()) |word| { + try words.insert(word); // duplicates automatically ignored + } + + std.debug.print("Unique words: {}\n", .{words.count()}); // 8 + + var it = words.iterator(); + while (it.next()) |word| { + std.debug.print(" {s}\n", .{word.*}); + } +} +``` + +## Notes + +- All strings are copied on insert/put, freed on remove/deinit +- Use `putMove` to transfer ownership instead of copying +- `get()` returns borrowed slice - don't store long-term +- Iteration order is not insertion order (hash map) +- For non-owning string maps, use `std.StringHashMap` diff --git a/references/std-build.md b/references/std-build.md new file mode 100644 index 0000000..00e680d --- /dev/null +++ b/references/std-build.md @@ -0,0 +1,1135 @@ + +# std.Build - Zig Build System Reference (Zig 0.16.0) + +The Zig build system models projects as directed acyclic graphs (DAG) of build steps. Build scripts are written in Zig itself (`build.zig`), providing full language features during configuration. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +## Zig 0.16 Update Notes + +Keep the `root_module = b.createModule(...)` style from Zig 0.15; it remains the baseline in 0.16. + +0.16 build-system changes to apply during upgrades: + +- Prefer build-system C translation with `b.addTranslateC(...)`; `@cImport` is deprecated as the long-term API. +- Unit tests can be bounded with `zig build test --test-timeout `. +- `--prominent-compile-errors` was removed; use `--error-style minimal`. +- `--error-style` supports `verbose`, `minimal`, `verbose_clear`, and `minimal_clear`. +- `--multiline-errors` supports `indent`, `newline`, and `none`. +- `Build.makeTempPath` and the RemoveDir step are gone. +- Use `b.addTempFiles`, `b.addMutateFiles`, `b.tmpPath`, and `std.Build.Step.WriteFile` temp/mutate modes for temporary build outputs. +- Use local package overrides or project-local package fetching instead of editing global dependency cache contents. + +## Table of Contents +- [Quick Start](#quick-start) +- [Core Concepts](#core-concepts) +- [Custom Targets](#custom-targets) +- [Creating Executables and Libraries](#creating-executables-and-libraries) +- [Modules](#modules) +- [Build Options](#build-options) +- [Build Steps](#build-steps) +- [Dependencies](#dependencies) +- [Running Commands](#running-commands) +- [Installing Artifacts](#installing-artifacts) +- [Testing](#testing) +- [Generating Files](#generating-files) +- [C/C++ Integration](#cc-integration) +- [LazyPath](#lazypath) +- [LazyPath Deep Dive](#lazypath-deep-dive) +- [Build Allocation](#build-allocation) +- [build.zig.zon](#buildzon) +- [CLI Reference](#cli-reference) + +## Quick Start + +### Minimal build.zig (0.16.x) +```zig +const std = @import("std"); + +pub fn build(b: *std.Build) void { + const target = b.standardTargetOptions(.{}); + const optimize = b.standardOptimizeOption(.{}); + + const exe = b.addExecutable(.{ + .name = "myapp", + .root_module = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, + }), + }); + + b.installArtifact(exe); + + const run_cmd = b.addRunArtifact(exe); + run_cmd.step.dependOn(b.getInstallStep()); + if (b.args) |args| run_cmd.addArgs(args); + + const run_step = b.step("run", "Run the application"); + run_step.dependOn(&run_cmd.step); +} +``` + +**CRITICAL:** `root_source_file` at the artifact level is removed. Use `root_module` with `b.createModule()`. + +## Core Concepts + +### Build Graph +The build system is a DAG where: +- **Nodes** are `Step` objects (compile, run, install, etc.) +- **Edges** are dependencies between steps +- Steps run concurrently when dependencies allow +- Caching prevents redundant work + +### Key Types +```zig +// Main build context +const Build = std.Build; + +// Build steps +const Step = std.Build.Step; + +// Compile artifacts (exe, lib, obj, test) +const Compile = std.Build.Step.Compile; + +// Module (compilation unit with source, target, optimize) +const Module = std.Build.Module; + +// Lazy path reference (resolved at build time) +const LazyPath = std.Build.LazyPath; + +// Target configuration +const ResolvedTarget = std.Build.ResolvedTarget; +``` + +### Standard Options +```zig +// Get target (defaults to native, user can override with -Dtarget) +const target = b.standardTargetOptions(.{}); + +// Get optimization (defaults to Debug, user can override with -Doptimize) +const optimize = b.standardOptimizeOption(.{}); + +// With defaults/constraints +const target = b.standardTargetOptions(.{ + .default_target = .{ .cpu_arch = .x86_64, .os_tag = .linux }, + .whitelist = &.{ + .{ .cpu_arch = .x86_64, .os_tag = .linux }, + .{ .cpu_arch = .aarch64, .os_tag = .macos }, + }, +}); + +const optimize = b.standardOptimizeOption(.{ + .preferred_optimize_mode = .ReleaseFast, // Default for --release +}); +``` + +## Custom Targets + +### Programmatic Target Configuration +For cross-compilation or building with specific CPU features, use `std.Target.Query`: + +```zig +// Build WebAssembly with specific features +const wasm_query: std.Target.Query = .{ + .cpu_arch = .wasm32, + .os_tag = .freestanding, + .cpu_features_add = std.Target.wasm.featureSet(&.{ .bulk_memory, .multivalue }), +}; +const wasm_target = b.resolveTargetQuery(wasm_query); + +const wasm_module = b.addExecutable(.{ + .name = "module", + .root_module = b.createModule(.{ + .root_source_file = b.path("src/wasm.zig"), + .target = wasm_target, + .optimize = .ReleaseSmall, + }), +}); +``` + +### Host Target for Build Tools +When building tools that run during the build (code generators, asset processors), use the host target: + +```zig +// Build a code generator that runs on the host machine +const codegen_tool = b.addExecutable(.{ + .name = "codegen", + .root_module = b.createModule(.{ + .root_source_file = b.path("tools/codegen.zig"), + .target = b.graph.host, // Always builds for the machine running the build + .optimize = .ReleaseFast, + }), +}); + +// This tool can be run even when cross-compiling the main project +const run_codegen = b.addRunArtifact(codegen_tool); +``` + +### CPU Feature Sets +Enable specific CPU features for optimized builds: + +```zig +// x86_64 with AVX2 and FMA +const x86_query: std.Target.Query = .{ + .cpu_arch = .x86_64, + .os_tag = .linux, + .cpu_features_add = std.Target.x86.featureSet(&.{ .avx2, .fma }), +}; + +// ARM with NEON +const arm_query: std.Target.Query = .{ + .cpu_arch = .aarch64, + .os_tag = .linux, + .cpu_features_add = std.Target.aarch64.featureSet(&.{ .neon, .crypto }), +}; +``` + +## Creating Executables and Libraries + +### Executable +```zig +const exe = b.addExecutable(.{ + .name = "myapp", + .root_module = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, + }), + .version = .{ .major = 1, .minor = 0, .patch = 0 }, // optional + .linkage = .dynamic, // optional: .static (default) or .dynamic +}); +``` + +### Static Library +```zig +const lib = b.addLibrary(.{ + .name = "mylib", + .linkage = .static, // default + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = target, + .optimize = optimize, + }), +}); +``` + +### Dynamic/Shared Library +```zig +const lib = b.addLibrary(.{ + .name = "mylib", + .linkage = .dynamic, + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = target, + .optimize = optimize, + }), + .version = .{ .major = 1, .minor = 2, .patch = 3 }, +}); +``` + +### Object File +```zig +const obj = b.addObject(.{ + .name = "myobj", + .root_module = b.createModule(.{ + .root_source_file = b.path("src/obj.zig"), + .target = target, + .optimize = optimize, + }), +}); +``` + +## Modules + +### Creating Modules +```zig +// Private module (not exposed to dependents) +const private_mod = b.createModule(.{ + .root_source_file = b.path("src/helper.zig"), + .target = target, + .optimize = optimize, +}); + +// Public module (exposed to packages depending on this one) +const public_mod = b.addModule("mymodule", .{ + .root_source_file = b.path("src/mymodule.zig"), + .target = target, + .optimize = optimize, +}); +``` + +### Adding Imports to Modules +```zig +// Add module import +exe.root_module.addImport("helper", private_mod); + +// Add module from dependency +const dep = b.dependency("some_lib", .{ .target = target, .optimize = optimize }); +exe.root_module.addImport("some_lib", dep.module("some_lib")); + +// Anonymous import (inline module) +exe.root_module.addAnonymousImport("generated", .{ + .root_source_file = generated_file, + .target = target, + .optimize = optimize, +}); +``` + +### Build Options as Module +```zig +const options = b.addOptions(); +options.addOption(bool, "enable_feature", enable_feature); +options.addOption([]const u8, "version", "1.0.0"); + +exe.root_module.addOptions("config", options); + +// In Zig code: +// const config = @import("config"); +// if (config.enable_feature) { ... } +``` + +## Build Options + +### Declaring Options +```zig +// Boolean option +const enable_debug = b.option(bool, "debug", "Enable debug mode") orelse false; + +// String option +const output_name = b.option([]const u8, "name", "Output name") orelse "default"; + +// Enum option +const Backend = enum { opengl, vulkan, metal }; +const backend = b.option(Backend, "backend", "Graphics backend") orelse .opengl; + +// Integer option +const threads = b.option(u32, "threads", "Number of threads") orelse 4; + +// List option +const features = b.option([]const []const u8, "features", "Features to enable") orelse &.{}; + +// Path option +const config_path = b.option(std.Build.LazyPath, "config", "Path to config file"); +``` + +### Using Options +```zig +// Pass options to compile step +if (enable_debug) { + exe.root_module.addCMacro("DEBUG", "1"); +} + +// Create options module for runtime access +const options = b.addOptions(); +options.addOption(bool, "debug", enable_debug); +options.addOption(Backend, "backend", backend); +exe.root_module.addOptions("build_options", options); +``` + +## Build Steps + +### Creating Custom Steps +```zig +// Named top-level step (visible in `zig build --list-steps`) +const test_step = b.step("test", "Run unit tests"); +const bench_step = b.step("bench", "Run benchmarks"); + +// Add dependencies +test_step.dependOn(&run_tests.step); +bench_step.dependOn(&run_benchmarks.step); + +// Get built-in steps +const install_step = b.getInstallStep(); // default step +const uninstall_step = b.getUninstallStep(); +``` + +### Step Dependencies +```zig +// Make step B depend on step A (A runs first) +step_b.dependOn(&step_a.step); + +// Chain multiple dependencies +const build_step = b.step("all", "Build everything"); +build_step.dependOn(&exe.step); +build_step.dependOn(&lib.step); +build_step.dependOn(&tests.step); +``` + +### Fail Step +```zig +// Fail with message (useful for unsupported configurations) +if (!target.result.os.tag.isDarwin()) { + const fail = b.addFail("This project only supports macOS"); + build_step.dependOn(&fail.step); + return; +} +``` + +### Compile Step Outputs +Access various outputs from compile steps: + +```zig +const lib = b.addLibrary(.{ + .name = "mylib", + .linkage = .static, + .root_module = b.createModule(.{ + .root_source_file = b.path("src/lib.zig"), + .target = target, + .optimize = optimize, + }), +}); + +// Get LazyPath to the compiled binary in .zig-cache +const bin_path = lib.getEmittedBin(); + +// Get assembly output for debugging codegen (like godbolt.org but for your project) +const asm_path = lib.getEmittedAsm(); +const install_asm = b.addInstallFile(asm_path, "debug/lib.s"); +const asm_step = b.step("asm", "View generated assembly"); +asm_step.dependOn(&install_asm.step); +``` + +### Run Step Data Dependencies +Pass file and directory paths as arguments while establishing proper data dependencies: + +```zig +const gen_tool = b.addExecutable(.{ + .name = "gen", + .root_module = b.createModule(.{ + .root_source_file = b.path("tools/gen.zig"), + .target = b.graph.host, + }), +}); + +const run_gen = b.addRunArtifact(gen_tool); + +// Directory input with prefix (--shader-dir=/path/to/shaders) +run_gen.addPrefixedDirectoryArg("--shader-dir=", b.path("data/shaders")); + +// Output file with prefix (--out=/path/to/output.h) +// Returns LazyPath to the generated file +const generated_header = run_gen.addPrefixedOutputFileArg("--out=", "shaders.h"); + +// The step now has data dependencies: if shaders change, it re-runs +``` + +### UpdateSourceFiles Step +Copy generated files back into the source tree (for committing generated code): + +```zig +const update_src = b.addUpdateSourceFiles(); +update_src.addCopyFileToSource(generated_file, "src/generated.zig"); + +const update_step = b.step("update-generated", "Update generated source files"); +update_step.dependOn(&update_src.step); +``` + +## Dependencies + +### Declaring in build.zig.zon +```zig +.{ + .name = "myproject", + .version = "1.0.0", + .dependencies = .{ + .zlib = .{ + .url = "https://github.com/user/zlib-zig/archive/v1.0.0.tar.gz", + .hash = "1220abc123...", + }, + .local_lib = .{ + .path = "../local-lib", + }, + }, + .paths = .{ "build.zig", "build.zig.zon", "src" }, +} +``` + +### Using Dependencies +```zig +// Get dependency +const zlib = b.dependency("zlib", .{ + .target = target, + .optimize = optimize, +}); + +// Get module from dependency +exe.root_module.addImport("zlib", zlib.module("zlib")); + +// Get artifact from dependency +exe.linkLibrary(zlib.artifact("z")); + +// Get path from dependency +const include_path = zlib.path("include"); +exe.root_module.addIncludePath(include_path); +``` + +### Lazy Dependencies +Lazy dependencies are only fetched when actually used, avoiding unnecessary downloads for platform-specific or optional dependencies. + +**In build.zig.zon:** +```zig +.dependencies = .{ + // Mark platform-specific dependency as lazy + .@"dawn-windows-x64" = .{ + .url = "https://github.com/example/dawn/releases/download/v1.0/dawn-windows-x64.tar.gz", + .hash = "1220abc...", + .lazy = true, // Only fetched when lazyDependency() is called + }, + .@"dawn-linux-x64" = .{ + .url = "https://github.com/example/dawn/releases/download/v1.0/dawn-linux-x64.tar.gz", + .hash = "1220def...", + .lazy = true, + }, +}, +``` + +**In build.zig:** +```zig +// lazyDependency returns ?*Dependency (null if not yet fetched) +const dawn_dep = switch (target.result.os.tag) { + .windows => b.lazyDependency("dawn-windows-x64", .{}), + .linux => b.lazyDependency("dawn-linux-x64", .{}), + else => null, +}; + +if (dawn_dep) |dep| { + // Dependency is available, use normally + exe.addLibraryPath(dep.path("lib")); + exe.linkSystemLibrary("dawn"); +} +``` + +**How it works:** After the build graph is constructed, `zig build` checks if any `lazyDependency()` calls require unfetched dependencies (returned null). If so, it fetches them and re-runs the build script with the dependencies now available. + +### Passing Options to Dependencies +```zig +const dep = b.dependency("configurable_lib", .{ + .target = target, + .optimize = optimize, + .enable_feature = true, + .backend = @as([]const u8, "vulkan"), +}); +``` + +## Running Commands + +### Run Compiled Artifact +```zig +const run_cmd = b.addRunArtifact(exe); + +// Pass command line arguments +if (b.args) |args| { + run_cmd.addArgs(args); +} + +// Fixed arguments +run_cmd.addArgs(&.{ "--config", "debug.json" }); + +// Change working directory +run_cmd.setCwd(b.path("data")); + +// Set environment variables +run_cmd.setEnvironmentVariable("DEBUG", "1"); + +// Create run step +const run_step = b.step("run", "Run the application"); +run_step.dependOn(&run_cmd.step); +``` + +### Run System Command +```zig +const cmd = b.addSystemCommand(&.{ "git", "describe", "--tags" }); + +// Capture output +const version = cmd.captureStdOut(); + +// Use output as file +const version_file = b.addInstallFile(version, "version.txt"); +``` + +### Run Project Tool +```zig +// Build and run a tool from the project +const tool = b.addExecutable(.{ + .name = "codegen", + .root_module = b.createModule(.{ + .root_source_file = b.path("tools/codegen.zig"), + .target = b.graph.host, // Build for host + .optimize = .ReleaseFast, + }), +}); + +const run_tool = b.addRunArtifact(tool); +run_tool.addArgs(&.{ "input.json" }); + +// Capture generated output +const generated = run_tool.addOutputFileArg("generated.zig"); + +// Use as source file +exe.root_module.addAnonymousImport("generated", .{ + .root_source_file = generated, +}); +``` + +## Installing Artifacts + +### Basic Installation +```zig +// Install to default location (zig-out/bin or zig-out/lib) +b.installArtifact(exe); +b.installArtifact(lib); +``` + +### Custom Installation +```zig +// Install with custom options +const install = b.addInstallArtifact(exe, .{ + .dest_dir = .{ .custom = "tools" }, // zig-out/tools/ +}); +b.getInstallStep().dependOn(&install.step); + +// Install file +b.installFile("assets/config.json", "share/config.json"); +b.installBinFile("scripts/run.sh", "run.sh"); + +// Install directory +b.installDirectory(.{ + .source_dir = b.path("assets"), + .install_dir = .{ .custom = "share" }, + .install_subdir = "assets", +}); +``` + +### Install Generated Files +```zig +// Install documentation +const docs_install = b.addInstallDirectory(.{ + .source_dir = lib.getEmittedDocs(), + .install_dir = .prefix, + .install_subdir = "docs", +}); + +const docs_step = b.step("docs", "Generate documentation"); +docs_step.dependOn(&docs_install.step); +``` + +## Testing + +### Unit Tests +```zig +const tests = b.addTest(.{ + .root_module = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, + }), +}); + +// Add imports (same as executable) +tests.root_module.addImport("helper", helper_mod); + +// Run tests +const run_tests = b.addRunArtifact(tests); + +const test_step = b.step("test", "Run unit tests"); +test_step.dependOn(&run_tests.step); +``` + +### Test Multiple Files +```zig +const test_files = &[_][]const u8{ + "src/parser.zig", + "src/lexer.zig", + "src/codegen.zig", +}; + +const test_step = b.step("test", "Run all tests"); + +for (test_files) |file| { + const t = b.addTest(.{ + .root_module = b.createModule(.{ + .root_source_file = b.path(file), + .target = target, + .optimize = optimize, + }), + }); + test_step.dependOn(&b.addRunArtifact(t).step); +} +``` + +### Filtering Tests +```zig +const tests = b.addTest(.{ + .root_module = b.createModule(.{ + .root_source_file = b.path("src/tests.zig"), + .target = target, + }), + .filters = &.{"specific_test_name"}, // Only run matching tests +}); +``` + +### Cross-Platform Testing +```zig +const run_tests = b.addRunArtifact(tests); +run_tests.skip_foreign_checks = true; // Don't fail for non-native targets +``` + +## Generating Files + +### WriteFiles Step +```zig +const wf = b.addWriteFiles(); + +// Add string content +_ = wf.add("config.json", + \\{ + \\ "version": "1.0" + \\} +); + +// Copy file +_ = wf.addCopyFile(b.path("template.txt"), "output.txt"); + +// Get directory for use elsewhere +const dir = wf.getDirectory(); +``` + +### ConfigHeader (for C headers) +```zig +// CMake-style config.h generation +const config_h = b.addConfigHeader(.{ + .style = .{ .cmake = b.path("config.h.in") }, +}, .{ + .HAVE_FEATURE = true, + .VERSION_STRING = "1.0.0", +}); + +exe.addConfigHeader(config_h); +``` + +### Code Generation with Zig Tool +```zig +const gen_tool = b.addExecutable(.{ + .name = "gen", + .root_module = b.createModule(.{ + .root_source_file = b.path("tools/gen.zig"), + .target = b.graph.host, + }), +}); + +const gen_run = b.addRunArtifact(gen_tool); +gen_run.addFileArg(b.path("schema.json")); +const generated = gen_run.addOutputFileArg("generated.zig"); + +exe.root_module.addAnonymousImport("schema", .{ + .root_source_file = generated, +}); +``` + +## C/C++ Integration + +> **Note:** Compile-level methods like `exe.addCSourceFiles()`, `exe.linkSystemLibrary()`, +> `exe.addIncludePath()`, `exe.linkLibC()` are **deprecated** (to be removed after 0.15.0). +> Use `exe.root_module.*` equivalents shown below. + +### Adding C Sources +```zig +exe.root_module.addCSourceFiles(.{ + .root = b.path("src/c"), + .files = &.{ "foo.c", "bar.c" }, + .flags = &.{ "-Wall", "-O2" }, +}); + +exe.root_module.addCSourceFile(.{ + .file = b.path("src/main.c"), + .flags = &.{"-std=c11"}, +}); +``` + +### Include Paths and Macros +```zig +exe.root_module.addIncludePath(b.path("include")); +exe.root_module.addSystemIncludePath(b.path("deps/include")); +exe.root_module.addCMacro("DEBUG", "1"); +exe.root_module.addCMacro("VERSION", "\"1.0.0\""); +``` + +### Linking Libraries +```zig +// System library +exe.root_module.linkSystemLibrary("pthread", .{}); +exe.root_module.linkSystemLibrary("ssl", .{}); + +// Static library file +exe.addObjectFile(b.path("lib/libfoo.a")); + +// Library search path +exe.addLibraryPath(b.path("lib")); +exe.addRPath(b.path("lib")); + +// Link libc (set via createModule options or directly) +exe.root_module.link_libc = true; +exe.root_module.link_libcpp = true; +``` + +### pkg-config Integration +```zig +// Use pkg-config to find library +exe.root_module.linkSystemLibrary("openssl", .{}); +exe.root_module.linkSystemLibrary("libcurl", .{}); +``` + +### Best Practices: Prefer Zig APIs Over Clang Flags +When building C/C++ code, prefer Zig's explicit APIs over raw Clang flags for better type safety and build graph visibility: + +```zig +// PREFERRED: Use Zig APIs +exe.root_module.addIncludePath(b.path("include")); // Instead of -I +exe.root_module.addSystemIncludePath(dep.path("include")); // Instead of -isystem + +// PREFERRED: Use Module.CreateOptions +const mod = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + .target = target, + .optimize = optimize, + .link_libc = false, // Instead of -nolibc flag + .sanitize_c = .off, // Type is ?std.zig.SanitizeC (.off/.trap/.full) +}); + +// AVOID: Raw Clang flags (use only when no Zig API exists) +exe.root_module.addCSourceFiles(.{ + .files = &.{"foo.c"}, + .flags = &.{"-DDEBUG"}, // Use addCMacro instead when possible +}); +``` + +This approach: +- Provides type-checked configuration +- Gives the build graph visibility into dependencies +- Supports LazyPaths for generated/dependency paths +- Prevents flags from conflicting with each other + +## LazyPath + +LazyPath represents paths that may not exist until build time. + +### Types of LazyPath +```zig +// Source file relative to build root +const src = b.path("src/main.zig"); + +// Generated file from a step +const generated = step.getOutput(); + +// From dependency +const dep_file = dep.path("include/header.h"); + +// CWD-relative (avoid when possible) +const cwd_path: std.Build.LazyPath = .{ .cwd_relative = "/absolute/path" }; +``` + +### Using LazyPath +```zig +// As source file +exe.root_module.root_source_file = b.path("src/main.zig"); + +// As include path +exe.addIncludePath(dep.path("include")); + +// Install +b.installFile(generated_file, "share/output.txt"); +``` + +## LazyPath Deep Dive + +LazyPath is central to how data flows between build steps. Understanding its variants and methods is key to advanced build scripts. + +### The Four Variants +```zig +const LazyPath = union(enum) { + // Path relative to build root (most common) + src_path: struct { root: ?*Build, sub_path: []const u8 }, + + // Output from a build step (e.g., compiled binary, generated file) + generated: struct { file: *GeneratedFile, sub_path: ?[]const u8 }, + + // Absolute or CWD-relative path (use sparingly) + cwd_relative: []const u8, + + // Path inside a dependency package + dependency: struct { dep: *Dependency, sub_path: []const u8 }, +}; +``` + +### Creating LazyPaths +```zig +// From build root (returns src_path variant) +const src = b.path("src/main.zig"); + +// From build step output (returns generated variant) +const bin = exe.getEmittedBin(); +const generated = run_step.addOutputFileArg("output.txt"); + +// From dependency (returns dependency variant) +const dep_header = zlib.path("include/zlib.h"); + +// Absolute path (returns cwd_relative variant - avoid when possible) +const abs: LazyPath = .{ .cwd_relative = "/usr/include" }; +``` + +### Establishing Data Dependencies +When passing LazyPaths between steps, use `addStepDependencies()` to ensure proper ordering: + +```zig +const gen_step = b.addRunArtifact(generator); +const generated_file = gen_step.addOutputFileArg("data.bin"); + +// IMPORTANT: Establish that compile step depends on gen_step's output +generated_file.addStepDependencies(&exe.step); + +// Now exe will wait for gen_step to complete before compiling +exe.root_module.addAnonymousImport("data", .{ + .root_source_file = generated_file, +}); +``` + +### Path Navigation +```zig +const base = b.path("src/modules/parser"); + +// Navigate to parent directory +const parent = base.dirname(); // "src/modules" + +// Concatenate subpath +const file = base.join("lexer.zig"); // "src/modules/parser/lexer.zig" + +// Chain operations +const sibling = base.dirname().join("utils/helpers.zig"); // "src/modules/utils/helpers.zig" +``` + +## Build Allocation + +The Build struct provides an arena allocator for convenient memory management in build scripts. + +### Using the Arena Allocator +```zig +pub fn build(b: *std.Build) void { + // b.allocator is an arena - no need to free allocations + const items = b.allocator.alloc(u8, 1024) catch @panic("OOM"); + // No need to call b.allocator.free(items) + + // All allocations are freed when the build completes +} +``` + +### Convenience Functions +```zig +// Join paths without allocator noise +const full_path = b.pathJoin(&.{ "src", "modules", "parser.zig" }); + +// Format strings without worrying about storage +const name = b.fmt("myapp-{s}-{s}", .{ + @tagName(target.result.cpu_arch), + @tagName(target.result.os_tag), +}); + +// Both return arena-allocated strings that don't need freeing +``` + +## build.zig.zon + +ZON (Zig Object Notation) is used for package manifests. + +### Full Example +```zig +.{ + // Package name (required) + .name = "my_project", + + // Semantic version (required) + .version = "1.2.3", + + // Minimum Zig version (optional) + .minimum_zig_version = "0.15.0", + + // Dependencies (optional) + .dependencies = .{ + // URL dependency with hash + .@"zig-network" = .{ + .url = "https://github.com/user/zig-network/archive/v1.0.0.tar.gz", + .hash = "12205f17c...", + }, + + // Local path dependency + .local_dep = .{ + .path = "../other-project", + }, + + // Lazy dependency (only fetched if used) + .optional_dep = .{ + .url = "https://example.com/dep.tar.gz", + .hash = "1220abc...", + .lazy = true, + }, + }, + + // Files included in package (required for publishing) + .paths = .{ + "build.zig", + "build.zig.zon", + "src", + "LICENSE", + "README.md", + }, +} +``` + +### Getting Dependency Hash +```bash +# Zig will tell you the expected hash on first build +zig build + +# Or fetch and display hash +zig fetch https://github.com/user/repo/archive/v1.0.0.tar.gz +``` + +## CLI Reference + +### Common Commands +```bash +# Build (runs install step) +zig build + +# Run specific step +zig build run +zig build test + +# List available steps +zig build --list-steps + +# Build with options +zig build -Dtarget=x86_64-linux -Doptimize=ReleaseFast + +# Release build +zig build --release=fast +zig build --release=safe +zig build --release=small + +# Verbose output +zig build --verbose + +# Parallel jobs +zig build -j4 + +# Watch mode (rebuild on changes) +zig build --watch + +# Web UI for build visualization +zig build --webui + +# Fetch dependencies +zig build --fetch +``` + +### Debug Options +```bash +# Debug compiler output +zig build --verbose-link +zig build --verbose-cc +zig build --verbose-air +zig build --verbose-llvm-ir + +# Reference trace for errors +zig build -freference-trace=10 + +# Time report +zig build --time-report +``` + +### Installation Paths +```bash +# Custom install prefix +zig build -p /usr/local +zig build --prefix=/opt/myapp + +# Custom subdirectories +zig build --prefix-lib-dir=lib64 +zig build --prefix-exe-dir=sbin +``` + +## Common Patterns + +### Build for Multiple Targets +```zig +const targets = [_]std.Target.Query{ + .{ .cpu_arch = .x86_64, .os_tag = .linux }, + .{ .cpu_arch = .aarch64, .os_tag = .linux }, + .{ .cpu_arch = .x86_64, .os_tag = .windows }, + .{ .cpu_arch = .aarch64, .os_tag = .macos }, +}; + +for (targets) |t| { + const resolved = b.resolveTargetQuery(t); + const exe = b.addExecutable(.{ + .name = b.fmt("myapp-{s}-{s}", .{ + @tagName(t.cpu_arch.?), + @tagName(t.os_tag.?), + }), + .root_module = b.createModule(.{ + .root_source_file = b.path("src/main.zig"), + .target = resolved, + .optimize = .ReleaseFast, + }), + }); + b.installArtifact(exe); +} +``` + +### Example Suite +```zig +const examples = [_][]const u8{ "basic", "advanced", "demo" }; + +const examples_step = b.step("examples", "Build examples"); + +for (examples) |name| { + const exe = b.addExecutable(.{ + .name = name, + .root_module = b.createModule(.{ + .root_source_file = b.path(b.fmt("examples/{s}.zig", .{name})), + .target = target, + .optimize = optimize, + }), + }); + exe.root_module.addImport("mylib", lib_mod); + + const install = b.addInstallArtifact(exe, .{ + .dest_dir = .{ .custom = "examples" }, + }); + examples_step.dependOn(&install.step); +} +``` + +### Format Check Step +```zig +const fmt_step = b.step("fmt", "Check formatting"); + +const fmt = b.addFmt(.{ + .paths = &.{ "src/", "build.zig" }, + .check = true, +}); + +fmt_step.dependOn(&fmt.step); +``` + +### Clean Step +```zig +const clean_step = b.step("clean", "Clean build artifacts"); + +clean_step.dependOn(&b.addRemoveDirTree(b.path("zig-out")).step); + +// Note: zig-cache deletion may fail on Windows while build is running +if (@import("builtin").os.tag != .windows) { + clean_step.dependOn(&b.addRemoveDirTree(b.path(".zig-cache")).step); +} +``` diff --git a/references/std-c.md b/references/std-c.md new file mode 100644 index 0000000..587e12c --- /dev/null +++ b/references/std-c.md @@ -0,0 +1,1005 @@ +# std.c - C ABI Types and Libc Bindings Reference (Zig 0.16.0) + +Platform-specific C ABI types and libc function bindings. Provides cross-platform type definitions that match the C ABI for each target OS, enabling FFI with C libraries and direct syscall access. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Prefer `std.Io` for portable file/network/process work. Use `std.c` only for explicit C ABI or platform-specific requirements, and keep ABI layout assertions next to C bindings. + +## Table of Contents +- [When to Use std.c](#when-to-use-stdc) +- [Fundamental Types](#fundamental-types) +- [File and I/O Types](#file-and-io-types) +- [Time Types](#time-types) +- [Process Types](#process-types) +- [Socket Types](#socket-types) +- [Signal Types](#signal-types) +- [Memory Types](#memory-types) +- [Constants and Flags](#constants-and-flags) +- [Libc Function Bindings](#libc-function-bindings) +- [Platform-Specific Submodules](#platform-specific-submodules) +- [Common Patterns](#common-patterns) + +## When to Use std.c + +Use `std.c` when: +- Interfacing with C libraries via FFI +- Making direct syscalls that require C-compatible types +- Working with platform-specific kernel interfaces +- Need exact type layout matching C headers + +Prefer higher-level alternatives when available: +```zig +// High-level (recommended) +const file = try std.fs.cwd().openFile("data.txt", .{}); + +// POSIX-level +const fd = try std.posix.open("data.txt", .{}, 0); + +// C-level (direct libc, lowest level) +const fd = std.c.open("data.txt", .{}, 0); +``` + +## Fundamental Types + +### Integer Types +```zig +const c = std.c; + +// Fixed-size types (same as C) +c.c_char // char (usually i8) +c.c_short // short +c.c_int // int +c.c_long // long (platform-dependent size) +c.c_longlong // long long +c.c_uchar // unsigned char +c.c_ushort // unsigned short +c.c_uint // unsigned int +c.c_ulong // unsigned long +c.c_ulonglong // unsigned long long + +// Size types +c.size_t // size_t +c.ssize_t // ssize_t (signed size) +c.intptr_t // intptr_t +c.uintptr_t // uintptr_t +c.ptrdiff_t // ptrdiff_t +c.intmax_t // i64 +c.uintmax_t // u64 +c.max_align_t // maximum alignment type +``` + +### File Descriptor Types +```zig +c.fd_t // File descriptor (i32 on POSIX, HANDLE on Windows) +c.ino_t // Inode number +c.off_t // File offset (i64) +c.dev_t // Device ID +c.mode_t // File mode/permissions +c.nlink_t // Link count +c.blksize_t // Block size +c.blkcnt_t // Block count +``` + +### User/Group Types +```zig +c.uid_t // User ID (u32) +c.gid_t // Group ID (u32) +c.pid_t // Process ID (platform-specific) +``` + +## File and I/O Types + +### Stat Structure +```zig +const c = std.c; + +// Platform-specific stat structure +const stat: c.Stat = undefined; +// Fields (vary by platform): +stat.dev // Device ID +stat.ino // Inode number +stat.mode // File mode +stat.nlink // Number of hard links +stat.uid // Owner UID +stat.gid // Owner GID +stat.size // File size in bytes +stat.atim // Last access time (timespec) +stat.mtim // Last modification time +stat.ctim // Last status change time +stat.blksize // Preferred block size +stat.blocks // Number of 512-byte blocks +``` + +### I/O Vectors +```zig +// Scatter/gather I/O +c.iovec // { .base: [*]u8, .len: usize } +c.iovec_const // { .base: [*]const u8, .len: usize } + +c.IOV_MAX // Maximum iovec count per operation (platform-specific) +``` + +### Directory Types +```zig +c.DIR // Directory stream (opaque) +c.dirent // Directory entry structure +c.dirent64 // 64-bit directory entry (Linux) +c.MAXNAMLEN // Max filename length in dirent +c.DT // Directory entry type constants + .UNKNOWN, .FIFO, .CHR, .DIR, .BLK, .REG, .LNK, .SOCK, .WHT +``` + +### File Locking +```zig +c.Flock // File lock structure + .start // Starting offset + .len // Length (0 = to EOF) + .pid // Process ID holding lock + .type // Lock type (F.RDLCK, F.WRLCK, F.UNLCK) + .whence // SEEK.SET/CUR/END + +c.LOCK // Lock operations (from std.posix) + .SH // Shared lock + .EX // Exclusive lock + .NB // Non-blocking + .UN // Unlock +``` + +## Time Types + +### timespec +```zig +c.timespec // High-resolution time + .sec // Seconds (time_t or isize) + .nsec // Nanoseconds (isize or c_long) + +c.time_t // Calendar time in seconds +c.clock_t // CPU time +c.suseconds_t // Microseconds component +``` + +### timeval +```zig +c.timeval // Time with microsecond resolution + .sec // Seconds + .usec // Microseconds (suseconds_t) + +c.timezone // Timezone info (deprecated on most systems) + .minuteswest + .dsttime +``` + +### Interval Timer +```zig +c.itimerspec // Timer specification + .interval // timespec - repeat interval + .value // timespec - initial expiration +``` + +### Clock IDs +```zig +c.clockid_t // Clock identifier enum +c.CLOCK // Clock ID constants (platform-specific) + .REALTIME // Wall clock + .MONOTONIC // Monotonic clock + .PROCESS_CPUTIME_ID // Process CPU time + .THREAD_CPUTIME_ID // Thread CPU time + // Platform-specific additions: + // macOS: MONOTONIC_RAW, UPTIME_RAW + // Linux: BOOTTIME, REALTIME_COARSE, MONOTONIC_COARSE + // FreeBSD: UPTIME, REALTIME_FAST, MONOTONIC_FAST +``` + +## Process Types + +### Process and Thread IDs +```zig +c.pid_t // Process ID +c.pthread_t // POSIX thread handle + +// Pthread synchronization primitives +c.pthread_mutex_t // Mutex +c.pthread_cond_t // Condition variable +c.pthread_rwlock_t // Read-write lock +c.pthread_attr_t // Thread attributes +c.pthread_key_t // Thread-local storage key +c.pthread_spin_t // Spinlock (some platforms) +c.sem_t // Semaphore +``` + +### Mutex Initializers +```zig +c.PTHREAD_MUTEX_INITIALIZER // Static mutex initializer +c.PTHREAD_COND_INITIALIZER // Static condvar initializer +``` + +### User/Group Info +```zig +c.passwd // Password entry structure + .name // Username + .passwd // Encrypted password + .uid // User ID + .gid // Group ID + .gecos // User info (real name) + .dir // Home directory + .shell // Login shell + // BSD/macOS additions: .change, .class, .expire + +c.group // Group entry structure + .name // Group name + .passwd // Group password + .gid // Group ID + .mem // Null-terminated member list +``` + +### Resource Limits +```zig +c.rlimit // Resource limit structure + .cur // Current (soft) limit + .max // Maximum (hard) limit + +c.rlim_t // Resource limit value type +c.rlimit_resource // Resource type enum + .NOFILE // Max open files + .STACK // Stack size + .DATA // Data segment size + .AS // Address space + // ... (many more platform-specific) + +c.RLIM // Special limit values + .INFINITY // No limit + .SAVED_MAX, .SAVED_CUR +``` + +### Resource Usage +```zig +c.rusage // Resource usage statistics + .utime // User CPU time (timeval) + .stime // System CPU time + .maxrss // Maximum resident set size + .minflt // Minor page faults + .majflt // Major page faults + .nvcsw // Voluntary context switches + .nivcsw // Involuntary context switches + // ... additional fields +``` + +## Socket Types + +### Address Structures +```zig +c.sockaddr // Generic socket address + .family // Address family (sa_family_t) + .data // Address data + +c.sockaddr_in // IPv4 address (from std.posix) +c.sockaddr_in6 // IPv6 address +c.sockaddr_un // Unix domain socket +c.sockaddr_storage // Large enough for any address + +c.socklen_t // Socket address length type +c.sa_family_t // Address family type +c.in_port_t // Port number (u16) +``` + +### Address Families +```zig +c.AF // Address family constants + .UNSPEC // Unspecified + .UNIX, .LOCAL // Unix domain + .INET // IPv4 + .INET6 // IPv6 + .PACKET // Packet (Linux) + .NETLINK // Netlink (Linux) + // Many platform-specific families + +c.PF // Protocol families (same values as AF) +``` + +### Socket Options +```zig +c.SOCK // Socket types + .STREAM // TCP + .DGRAM // UDP + .RAW // Raw socket + .SEQPACKET // Sequenced packet + .CLOEXEC // Set close-on-exec + .NONBLOCK // Non-blocking + +c.SOL // Socket level for options + .SOCKET // Socket-level options + .IP, .IPV6 // IP-level options + .TCP, .UDP // Protocol-level options + +c.SO // Socket options (SOL_SOCKET level) + .REUSEADDR, .REUSEPORT + .KEEPALIVE + .BROADCAST + .LINGER + .RCVBUF, .SNDBUF + .RCVTIMEO, .SNDTIMEO + .ERROR + // Many more + +c.TCP // TCP options (SOL_TCP level) + .NODELAY // Disable Nagle + .KEEPIDLE // Idle time before keepalive + .KEEPINTVL // Keepalive interval + .KEEPCNT // Keepalive count + +c.IPPROTO // IP protocols + .IP, .ICMP, .TCP, .UDP, .IPV6, .RAW +``` + +### Message Flags +```zig +c.MSG // Send/recv flags + .OOB // Out-of-band data + .PEEK // Peek at incoming data + .DONTROUTE // Bypass routing + .DONTWAIT // Non-blocking + .NOSIGNAL // Don't generate SIGPIPE + .TRUNC, .CTRUNC + .WAITALL // Wait for full request +``` + +### Message Headers +```zig +c.msghdr // Message header for sendmsg/recvmsg + .name // Optional address + .namelen // Address length + .iov // Scatter/gather array + .iovlen // Elements in iov + .control // Ancillary data + .controllen // Ancillary data length + .flags // Flags on received message + +c.msghdr_const // Const version for sendmsg +``` + +### Address Info (DNS) +```zig +c.addrinfo // Address info structure + .flags // AI flags + .family // Address family + .socktype // Socket type + .protocol // Protocol + .addrlen // Address length + .addr // Socket address + .canonname // Canonical name + .next // Next in list + +c.AI // getaddrinfo flags + .PASSIVE // For bind() + .CANONNAME // Request canonical name + .NUMERICHOST // Don't resolve hostname + .NUMERICSERV // Don't resolve service + .V4MAPPED, .ALL, .ADDRCONFIG + +c.NI // getnameinfo flags + .NUMERICHOST, .NUMERICSERV + .NOFQDN, .NAMEREQD + .DGRAM + +c.EAI // getaddrinfo errors + .AGAIN, .BADFLAGS, .FAIL, .FAMILY + .MEMORY, .NONAME, .SERVICE, .SOCKTYPE + .SYSTEM, .OVERFLOW +``` + +## Signal Types + +### Signal Numbers +```zig +c.SIG // Signal constants (platform-specific enum) + .HUP, .INT, .QUIT, .ILL, .TRAP, .ABRT + .FPE, .KILL, .BUS, .SEGV, .SYS, .PIPE + .ALRM, .TERM, .URG, .STOP, .TSTP, .CONT + .CHLD, .TTIN, .TTOU, .IO, .XCPU, .XFSZ + .VTALRM, .PROF, .WINCH, .USR1, .USR2 +``` + +### Signal Handling +```zig +c.Sigaction // Signal action structure + .handler // Handler function or SIG_DFL/SIG_IGN + .mask // Signals to block during handler + .flags // SA flags + +c.SA // Sigaction flags + .NOCLDSTOP // Don't notify on child stop + .NOCLDWAIT // Don't create zombies + .SIGINFO // Use sa_sigaction handler + .RESTART // Restart interrupted syscalls + .NODEFER // Don't block signal in handler + .RESETHAND // Reset to default after handling + .ONSTACK // Use alternate signal stack + +c.siginfo_t // Signal info (with SA_SIGINFO) +c.sigset_t // Signal set +c.sigval // Signal value union +c.sig_atomic_t // Async-signal-safe integer + +c.NSIG // Number of signals +c.MINSIGSTKSZ // Minimum signal stack size +c.SIGSTKSZ // Default signal stack size +``` + +### Signal Stack +```zig +c.stack_t // Alternate signal stack + .sp // Stack pointer + .flags // SS flags + .size // Stack size + +c.SS // Stack flags + .ONSTACK // Currently on signal stack + .DISABLE // Signal stack disabled +``` + +## Memory Types + +### Memory Protection +```zig +c.PROT // Memory protection flags + .NONE // No access + .READ // Read permission + .WRITE // Write permission + .EXEC // Execute permission +``` + +### Memory Mapping +```zig +c.MAP // mmap flags + .SHARED // Share changes + .PRIVATE // Private copy-on-write + .FIXED // Use exact address + .ANONYMOUS // No file backing (Linux/BSD) + .ANON // Alias for ANONYMOUS + .NORESERVE // Don't reserve swap + .STACK // Stack mapping + // Platform-specific flags + +c.MAP_FAILED // mmap failure return value + +c.MREMAP // mremap flags (Linux) + .MAYMOVE + .FIXED +``` + +### Memory Advice +```zig +c.MADV // madvise hints + .NORMAL // Default behavior + .RANDOM // Random access pattern + .SEQUENTIAL // Sequential access + .WILLNEED // Will need soon + .DONTNEED // Won't need soon + .FREE // Can free pages (Linux) + .REMOVE // Remove from memory + // Platform-specific hints +``` + +### Memory Sync +```zig +c.MSF // msync flags + .ASYNC // Asynchronous sync + .SYNC // Synchronous sync + .INVALIDATE // Invalidate caches +``` + +## Constants and Flags + +### Error Numbers +```zig +c.E // errno values (platform-specific enum) + .SUCCESS // No error (0) + .PERM // Operation not permitted + .NOENT // No such file or directory + .SRCH // No such process + .INTR // Interrupted system call + .IO // I/O error + .NXIO // No such device + .@"2BIG" // Arg list too long + .NOEXEC // Exec format error + .BADF // Bad file descriptor + .CHILD // No child processes + .AGAIN // Try again (also WOULDBLOCK) + .NOMEM // Out of memory + .ACCES // Permission denied + .FAULT // Bad address + .BUSY // Device busy + .EXIST // File exists + .XDEV // Cross-device link + .NODEV // No such device + .NOTDIR // Not a directory + .ISDIR // Is a directory + .INVAL // Invalid argument + .NFILE // File table overflow + .MFILE // Too many open files + .NOTTY // Not a typewriter + .FBIG // File too large + .NOSPC // No space on device + .SPIPE // Illegal seek + .ROFS // Read-only filesystem + .PIPE // Broken pipe + .DEADLK // Deadlock avoided + .NAMETOOLONG // Name too long + .NOSYS // Function not implemented + .NOTEMPTY // Directory not empty + .TIMEDOUT // Connection timed out + .CONNREFUSED // Connection refused + // Many more platform-specific errors +``` + +### File Control (fcntl) +```zig +c.F // fcntl commands + .DUPFD // Duplicate fd + .GETFD // Get fd flags + .SETFD // Set fd flags + .GETFL // Get file flags + .SETFL // Set file flags + .GETLK // Get lock info + .SETLK // Set lock (non-blocking) + .SETLKW // Set lock (blocking) + .DUPFD_CLOEXEC // Dup with close-on-exec + // Lock types: + .RDLCK // Read lock + .WRLCK // Write lock + .UNLCK // Unlock + +c.FD_CLOEXEC // Close-on-exec flag (1) +``` + +### Access Mode +```zig +c.F_OK // Test existence (0) +c.X_OK // Test execute (1) +c.W_OK // Test write (2) +c.R_OK // Test read (4) +``` + +### Open Flags +```zig +c.O // open() flags (struct on some platforms) + .RDONLY // Read only + .WRONLY // Write only + .RDWR // Read/write + .CREAT // Create if not exists + .EXCL // Exclusive create + .TRUNC // Truncate + .APPEND // Append mode + .NONBLOCK // Non-blocking + .SYNC // Synchronous writes + .CLOEXEC // Close on exec + .DIRECTORY // Must be directory + .NOFOLLOW // Don't follow symlinks + // Platform-specific flags +``` + +### Seek Whence +```zig +c.SEEK // lseek whence values + .SET // From beginning + .CUR // From current position + .END // From end +c.whence_t // Whence type (c_int or wasi.whence_t) +``` + +### File Mode Bits +```zig +c.S // File mode constants + .IFMT // File type mask + .IFSOCK // Socket + .IFLNK // Symbolic link + .IFREG // Regular file + .IFBLK // Block device + .IFDIR // Directory + .IFCHR // Character device + .IFIFO // FIFO + .ISUID // Set-user-ID + .ISGID // Set-group-ID + .ISVTX // Sticky bit + .IRWXU // Owner rwx + .IRUSR, .IWUSR, .IXUSR + .IRWXG // Group rwx + .IRGRP, .IWGRP, .IXGRP + .IRWXO // Other rwx + .IROTH, .IWOTH, .IXOTH +``` + +### Poll Events +```zig +c.POLL // poll() events + .IN // Data to read + .PRI // Priority data + .OUT // Writing possible + .ERR // Error condition + .HUP // Hang up + .NVAL // Invalid fd + .RDNORM, .RDBAND + .WRNORM, .WRBAND + +c.pollfd // poll file descriptor + .fd // File descriptor + .events // Requested events + .revents // Returned events + +c.nfds_t // Number of poll fds +``` + +### AT Flags (openat, etc.) +```zig +c.AT // *at() function flags + .FDCWD // Use current directory + .SYMLINK_NOFOLLOW + .REMOVEDIR + .SYMLINK_FOLLOW + .EACCESS + .EMPTY_PATH // Linux +``` + +### Terminal I/O +```zig +c.termios // Terminal attributes + .iflag // Input flags + .oflag // Output flags + .cflag // Control flags + .lflag // Local flags + .cc // Control characters + +c.NCCS // Number of control chars +c.V // Control character indices + .INTR, .QUIT, .ERASE, .KILL + .EOF, .TIME, .MIN, .START, .STOP + // ... (many more) + +c.tc_iflag_t // Input flag type +c.tc_oflag_t // Output flag type +c.tc_cflag_t // Control flag type +c.tc_lflag_t // Local flag type +c.speed_t // Baud rate type +c.cc_t // Control character type + +c.TCSA // tcsetattr actions (from std.posix) +c.CSIZE // Character size mask +``` + +### Wait Flags +```zig +c.W // waitpid flags + .NOHANG // Don't block + .UNTRACED // Report stopped children + .CONTINUED // Report continued children (Linux) +``` + +### Shutdown +```zig +c.SHUT // shutdown() how values + .RD // Stop receiving + .WR // Stop sending + .RDWR // Stop both +``` + +### Kqueue (BSD/macOS) +```zig +c.Kevent // kevent structure + .ident // Identifier + .filter // Filter type + .flags // Action flags + .fflags // Filter-specific flags + .data // Filter data + .udata // User data + +c.EV // Event flags + .ADD, .DELETE, .ENABLE, .DISABLE + .ONESHOT, .CLEAR, .RECEIPT + .EOF, .ERROR + +c.EVFILT // Event filters + .READ, .WRITE, .AIO + .VNODE, .PROC, .SIGNAL + .TIMER, .USER, .MACHPORT (macOS) + +c.NOTE // Filter-specific notes +``` + +### Port Events (Solaris/illumos) +```zig +c.port_t // Event port handle +c.port_event // Port event structure +``` + +## Libc Function Bindings + +### File Operations +```zig +c.close // Close file descriptor +c.fstat // Get file status by fd +c.fstatat // Get file status relative to directory fd +c.stat // Get file status by path +c.readdir // Read directory entry +c.realpath // Resolve canonical path +c.flock // Apply/remove advisory lock +``` + +### Time Functions +```zig +c.clock_getres // Get clock resolution +c.clock_gettime // Get clock time +c.nanosleep // High-resolution sleep +c.gettimeofday // Get wall clock time +``` + +### Memory Functions +```zig +c.msync // Synchronize mapped memory +c.posix_memalign // Aligned memory allocation +c.malloc_size // macOS: get allocation size +c.malloc_usable_size // Linux: get allocation size +c._msize // Windows: get allocation size +``` + +### Process Functions +```zig +c.fork // Create child process +c.getrusage // Get resource usage +c.sched_yield // Yield processor +c.sysconf // Get system configuration +c.getentropy // Get random bytes (secure) +c.arc4random_buf // Get random bytes (BSD) +c.getrandom // Get random bytes (Linux) +``` + +### Signal Functions +```zig +c.sigaction // Examine/change signal action +c.sigaltstack // Set alternate signal stack +c.sigfillset // Fill signal set +c.sigemptyset // Empty signal set +c.sigaddset // Add signal to set +c.sigdelset // Remove signal from set +c.sigismember // Test signal membership +c.sigprocmask // Block/unblock signals +c.sigrtmin() // First real-time signal +c.sigrtmax() // Last real-time signal +``` + +### Network Functions +```zig +c.socket // Create socket +c.sendfile // Efficient file-to-socket copy +``` + +### I/O Functions +```zig +c.pipe2 // Create pipe with flags +c.copy_file_range // Efficient file copy (Linux) +c.getdirentries // Read directory entries (BSD) +c.getdents // Read directory entries (Linux) +``` + +### Thread Functions +```zig +c.pthread_setname_np // Set thread name +c.pthread_threadid_np // Get thread ID (macOS) +c.getcontext // Get current context (some platforms) +``` + +## Platform-Specific Submodules + +### darwin (macOS/iOS) +```zig +const darwin = std.c.darwin; // Internal, re-exported via std.c + +// Mach types and functions +darwin.mach_port_t +darwin.mach_task_self() +darwin.mach_msg() +darwin.mach_host_self() +darwin.mach_timebase_info() +darwin.mach_absolute_time() + +// Exception handling +darwin.EXC, darwin.EXCEPTION +darwin.task_set_exception_ports() +darwin.task_get_exception_ports() + +// Thread state +darwin.thread_state +darwin.thread_get_state() +darwin.thread_set_state() + +// VM operations +darwin.mach_vm_read() +darwin.mach_vm_write() +darwin.mach_vm_protect() +darwin.mach_vm_region() + +// Dispatch/GCD semaphores +darwin.dispatch_semaphore_create() +darwin.dispatch_semaphore_wait() +darwin.dispatch_semaphore_signal() + +// Unfair locks +darwin.os_unfair_lock +darwin.os_unfair_lock_lock() +darwin.os_unfair_lock_unlock() + +// Process spawning +darwin.posix_spawn() +darwin.posix_spawn_file_actions_* + +// File copy +darwin.fcopyfile() +darwin.COPYFILE +``` + +### freebsd +```zig +// Futex-like operations +std.c._umtx_op() +std.c.UMTX_OP, std.c.UMTX_ABSTIME + +// Capability rights +std.c.cap_rights + +// Memory file descriptors +std.c.MFD + +// Process info +std.c.kinfo_file +std.c.kinfo_getfile() +``` + +### openbsd +```zig +// Security +std.c.pledge() +std.c.unveil() + +// Password hashing +std.c.bcrypt() +std.c.bcrypt_newhash() +std.c.bcrypt_checkpass() + +// BSD authentication +std.c.auth_* // Various auth functions + +// Login capabilities +std.c.login_getclass() +std.c.login_getcapstr() +``` + +### netbsd +```zig +std.c._lwp_self() +std.c.lwpid_t +std.c._ksiginfo +``` + +### dragonfly +```zig +std.c.lwp_gettid() +std.c.umtx_sleep() +std.c.umtx_wakeup() +``` + +### solaris/illumos +```zig +// Port-based events +std.c.port_t +std.c.port_event +std.c.PORT_SOURCE +std.c.FILE_EVENT + +// Types +std.c.zoneid_t +std.c.ctid_t +std.c.projid_t +``` + +### haiku +```zig +std.c.team_id +std.c.thread_id +std.c.area_id +std.c.area_info +std.c._get_next_area_info() +std.c._get_next_image_info() +std.c.find_directory() +std.c.get_system_info() +``` + +### serenity +```zig +std.c.PERF_EVENT +std.c.profiling_enable() +std.c.profiling_disable() +std.c.futex_wait() +std.c.futex_wake() +std.c.anon_create() +``` + +## Common Patterns + +### Version Checking (glibc/musl) +```zig +// Check if glibc version supports a feature +if (std.c.versionCheck(.{ .major = 2, .minor = 28, .patch = 0 })) { + // Use newer glibc feature +} +// Returns true for musl (always "current") +// Returns false if not linking libc +``` + +### FFI with C Libraries +```zig +// Declare external C function +extern "c" fn c_function(fd: std.c.fd_t, buf: [*]u8, len: std.c.size_t) std.c.ssize_t; + +// Use std.c types for ABI compatibility +pub fn wrapper(fd: std.posix.fd_t, buf: []u8) !usize { + const result = c_function(fd, buf.ptr, buf.len); + if (result < 0) { + const err = std.posix.errno(std.c._errno().*); + return std.posix.unexpectedErrno(err); + } + return @intCast(result); +} +``` + +### Platform-Specific Type Handling +```zig +const builtin = @import("builtin"); + +fn platformSpecificCall() void { + switch (builtin.os.tag) { + .linux => { + // Linux uses c_int for some syscalls + const result: std.c.c_int = ...; + }, + .macos, .ios => { + // Darwin uses different types + const port = std.c.darwin.mach_port_t; + }, + .windows => { + // Windows uses HANDLE + const fd: std.c.fd_t = ...; // This is HANDLE on Windows + }, + else => @compileError("Unsupported"), + } +} +``` + +### Getting errno +```zig +// std.c._errno returns a pointer to errno +const c = std.c; + +fn checkError(result: c.c_int) !void { + if (result < 0) { + const errno_val = c._errno().*; + // Convert to std.posix error + return std.posix.unexpectedErrno(@enumFromInt(errno_val)); + } +} +``` + +### Standard File Descriptors +```zig +const c = std.c; + +c.STDIN_FILENO // 0 (or equivalent) +c.STDOUT_FILENO // 1 +c.STDERR_FILENO // 2 +``` + +### Path Limits +```zig +c.PATH_MAX // Maximum path length +c.NAME_MAX // Maximum filename length +c.HOST_NAME_MAX // Maximum hostname length +``` diff --git a/references/std-compress.md b/references/std-compress.md new file mode 100644 index 0000000..4ee1f34 --- /dev/null +++ b/references/std-compress.md @@ -0,0 +1,457 @@ +# std.compress - Compression API Reference (Zig 0.16.0) + +Compression and decompression algorithms. Zig 0.16 adds Deflate compression, simplifies decompression, and continues migrating compression APIs to `std.Io.Reader` / `std.Io.Writer`. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Old examples below may use removed `std.io.GenericReader` or `fixedBufferStream` patterns. Translate them to `std.Io.Reader` / `std.Io.Writer` before using them in Zig 0.16 code. + +## Table of Contents +- [Module Structure](#module-structure) +- [DEFLATE (gzip/zlib)](#deflate-gzipzlib) +- [Zstandard](#zstandard) +- [LZMA](#lzma) +- [LZMA2](#lzma2) +- [XZ](#xz) +- [Common Patterns](#common-patterns) + +## Module Structure + +```zig +std.compress.flate // DEFLATE: gzip, zlib, raw deflate +std.compress.zstd // Zstandard compression +std.compress.lzma // LZMA compression +std.compress.lzma2 // LZMA2 compression +std.compress.xz // XZ format (LZMA2 container) +``` + +## DEFLATE (gzip/zlib) + +DEFLATE compression with gzip, zlib, or raw containers. Defined in RFC 1951 (deflate), RFC 1950 (zlib), RFC 1952 (gzip). + +### Container Types + +```zig +const Container = enum { + raw, // No header/footer, raw deflate stream + gzip, // gzip header (10+ bytes) + deflate + CRC32 + size footer (8 bytes) + zlib, // zlib header (2 bytes) + deflate + Adler32 footer (4 bytes) +}; +``` + +### Decompression + +```zig +const flate = std.compress.flate; + +// Decompress gzip data +var input: std.Io.Reader = .fixed(compressed_data); +var output: std.Io.Writer.Allocating = .init(allocator); +defer output.deinit(); + +var decompress: flate.Decompress = .init(&input, .gzip, &.{}); +_ = try decompress.reader.streamRemaining(&output.writer); + +const decompressed = output.written(); +``` + +### With History Buffer + +For streaming decompression with backref support: + +```zig +var buffer: [flate.max_window_len]u8 = undefined; +var decompress: flate.Decompress = .init(&input, .zlib, &buffer); +``` + +### Decompress Constants + +```zig +flate.max_window_len // 65536 - Maximum window size (32768 * 2) +flate.history_len // 32768 - History buffer length +``` + +### Compression + +```zig +const flate = std.compress.flate; + +var output: std.Io.Writer.Allocating = .init(allocator); +defer output.deinit(); + +var buffer: [flate.max_window_len]u8 = undefined; +var compress: flate.Compress = .init(&output.writer, &buffer, .{ + .level = .default, + .container = .gzip, +}); + +try compress.writer.writeAll(data); +try compress.end(); + +const compressed = output.written(); +``` + +### Compression Levels + +```zig +const Level = enum { + level_4, // Fastest + level_5, + level_6, // Default + level_7, + level_8, + level_9, // Best compression + + fast, // Alias for level_4 + default, // Alias for level_6 + best, // Alias for level_9 +}; +``` + +### Huffman-Only Compression + +For faster compression without LZ77 match searching: + +```zig +const HuffmanEncoder = flate.HuffmanEncoder; +// Used internally for Huffman-only encoding (bigger output, faster compression) +``` + +## Zstandard + +Zstandard (zstd) decompression. High compression ratio with fast decompression. + +### Decompression + +```zig +const zstd = std.compress.zstd; + +var input: std.Io.Reader = .fixed(compressed_data); +var output: std.Io.Writer.Allocating = .init(allocator); +defer output.deinit(); + +var decompress: zstd.Decompress = .init(&input, &.{}, .{}); +_ = try decompress.reader.streamRemaining(&output.writer); + +const decompressed = output.written(); +``` + +### With Custom Window Size + +```zig +var buffer: [zstd.default_window_len + zstd.block_size_max]u8 = undefined; +var decompress: zstd.Decompress = .init(&input, &buffer, .{ + .window_len = zstd.default_window_len, + .verify_checksum = false, // Not yet implemented +}); +``` + +### Zstd Constants + +```zig +zstd.default_window_len // 8 * 1024 * 1024 (8 MB) +zstd.block_size_max // 1 << 17 (128 KB) +``` + +### Options + +```zig +pub const Options = struct { + verify_checksum: bool = false, // Not yet implemented + window_len: u32 = zstd.default_window_len, +}; +``` + +## LZMA + +LZMA decompression with streaming reader interface. + +### Decompression + +```zig +const lzma = std.compress.lzma; + +var decompress = try lzma.decompress(allocator, reader); +defer decompress.deinit(); + +var buf: [4096]u8 = undefined; +while (true) { + const n = try decompress.read(&buf); + if (n == 0) break; + // Process buf[0..n] +} +``` + +### With Options + +```zig +var decompress = try lzma.decompressWithOptions(allocator, reader, .{ + .memlimit = 128 * 1024 * 1024, // 128 MB memory limit +}); +``` + +### Decompress Type + +```zig +pub fn Decompress(comptime ReaderType: type) type { + return struct { + pub const Reader = std.io.GenericReader(*Self, Error, read); + + pub fn init(allocator: Allocator, source: ReaderType, params: Params, memlimit: ?usize) !Self; + pub fn deinit(self: *Self) void; + pub fn reader(self: *Self) Reader; + pub fn read(self: *Self, output: []u8) Error!usize; + }; +} +``` + +## LZMA2 + +LZMA2 decompression (improved LZMA with better streaming support). + +### Decompression + +```zig +const lzma2 = std.compress.lzma2; + +var output = std.ArrayList(u8).empty; +defer output.deinit(allocator); + +var stream = std.io.fixedBufferStream(compressed_data); +try lzma2.decompress(allocator, stream.reader(), output.writer(allocator)); +``` + +## XZ + +XZ format decompression (LZMA2 in a container with checksums). + +### Decompression + +```zig +const xz = std.compress.xz; + +var decompress = try xz.decompress(allocator, reader); +defer decompress.deinit(); + +var buf: [4096]u8 = undefined; +while (true) { + const n = try decompress.read(&buf); + if (n == 0) break; + // Process buf[0..n] +} +``` + +### Check Types + +XZ supports multiple integrity check types: + +```zig +pub const Check = enum(u4) { + none = 0x00, + crc32 = 0x01, + crc64 = 0x04, + sha256 = 0x0A, + _, +}; +``` + +## Common Patterns + +### Decompress gzip File + +```zig +fn decompressGzip(allocator: Allocator, compressed: []const u8) ![]u8 { + const flate = std.compress.flate; + + var input: std.Io.Reader = .fixed(compressed); + var output: std.Io.Writer.Allocating = .init(allocator); + errdefer output.deinit(); + + var decompress: flate.Decompress = .init(&input, .gzip, &.{}); + _ = try decompress.reader.streamRemaining(&output.writer); + + return output.toOwnedSlice(); +} +``` + +### Decompress zlib Data + +```zig +fn decompressZlib(allocator: Allocator, compressed: []const u8) ![]u8 { + const flate = std.compress.flate; + + var input: std.Io.Reader = .fixed(compressed); + var output: std.Io.Writer.Allocating = .init(allocator); + errdefer output.deinit(); + + var decompress: flate.Decompress = .init(&input, .zlib, &.{}); + _ = try decompress.reader.streamRemaining(&output.writer); + + return output.toOwnedSlice(); +} +``` + +### Decompress Zstandard + +```zig +fn decompressZstd(allocator: Allocator, compressed: []const u8) ![]u8 { + const zstd = std.compress.zstd; + + var input: std.Io.Reader = .fixed(compressed); + var output: std.Io.Writer.Allocating = .init(allocator); + errdefer output.deinit(); + + var decompress: zstd.Decompress = .init(&input, &.{}, .{}); + _ = try decompress.reader.streamRemaining(&output.writer); + + return output.toOwnedSlice(); +} +``` + +### Stream Decompression to File + +```zig +fn decompressToFile( + input_path: []const u8, + output_path: []const u8, + container: std.compress.flate.Container, +) !void { + const flate = std.compress.flate; + + const input_file = try std.fs.cwd().openFile(input_path, .{}); + defer input_file.close(); + + const output_file = try std.fs.cwd().createFile(output_path, .{}); + defer output_file.close(); + + var input_buf: [4096]u8 = undefined; + var input_reader = input_file.reader(&input_buf); + + var output_buf: [4096]u8 = undefined; + var output_writer = output_file.writer(&output_buf); + + var decompress: flate.Decompress = .init(&input_reader.interface, container, &.{}); + _ = try decompress.reader.streamRemaining(&output_writer.interface); + try output_writer.interface.flush(); +} +``` + +### Detect Compression Format + +```zig +fn detectFormat(data: []const u8) ?enum { gzip, zlib, zstd, xz } { + if (data.len < 2) return null; + + // gzip: 0x1f 0x8b + if (data[0] == 0x1f and data[1] == 0x8b) return .gzip; + + // zlib: CMF byte with CM=8, CINFO<=7 + const cmf = data[0]; + if ((cmf & 0x0f) == 8 and (cmf >> 4) <= 7) { + // Check FCHECK makes header divisible by 31 + const header: u16 = (@as(u16, data[0]) << 8) | data[1]; + if (header % 31 == 0) return .zlib; + } + + // zstd: magic 0xFD2FB528 + if (data.len >= 4) { + const magic = std.mem.readInt(u32, data[0..4], .little); + if (magic == 0xFD2FB528) return .zstd; + } + + // xz: magic 0xFD377A585A00 + if (data.len >= 6) { + if (std.mem.eql(u8, data[0..6], &.{ 0xFD, '7', 'z', 'X', 'Z', 0x00 })) return .xz; + } + + return null; +} +``` + +## Error Types + +### DEFLATE Errors + +```zig +pub const Error = Container.Error || error{ + InvalidCode, + InvalidMatch, + WrongStoredBlockNlen, + InvalidBlockType, + InvalidDynamicBlockHeader, + ReadFailed, + OversubscribedHuffmanTree, + IncompleteHuffmanTree, + MissingEndOfBlockCode, + EndOfStream, +}; + +pub const Container.Error = error{ + BadGzipHeader, + BadZlibHeader, + WrongGzipChecksum, + WrongGzipSize, + WrongZlibChecksum, +}; +``` + +### Zstandard Errors + +```zig +pub const Error = error{ + BadMagic, + BlockOversize, + ChecksumFailure, + ContentOversize, + DictionaryIdFlagUnsupported, + EndOfStream, + HuffmanTreeIncomplete, + InvalidBitStream, + MalformedAccuracyLog, + MalformedBlock, + MalformedCompressedBlock, + MalformedFrame, + MalformedFseBits, + MalformedFseTable, + MalformedHuffmanTree, + MalformedLiteralsHeader, + MalformedLiteralsLength, + MalformedLiteralsSection, + MalformedSequence, + MissingStartBit, + OutputBufferUndersize, + InputBufferUndersize, + ReadFailed, + RepeatModeFirst, + ReservedBitSet, + ReservedBlock, + SequenceBufferUndersize, + TreelessLiteralsFirst, + UnexpectedEndOfLiteralStream, + WindowOversize, + WindowSizeUnknown, +}; +``` + +## Supported Features + +**DEFLATE (flate)**: +- Decompression: gzip, zlib, raw deflate +- Compression: gzip, zlib, raw deflate (levels 4-9) +- Streaming with history buffer + +**Zstandard (zstd)**: +- Decompression only +- Skippable frames +- Configurable window size +- Dictionary support: Not implemented + +**LZMA/LZMA2**: +- Decompression only +- Streaming interface +- Memory limit configuration + +**XZ**: +- Decompression only +- CRC32/CRC64/SHA256 integrity checks +- Multiple block support diff --git a/references/std-crypto.md b/references/std-crypto.md new file mode 100644 index 0000000..c952745 --- /dev/null +++ b/references/std-crypto.md @@ -0,0 +1,602 @@ +# std.crypto - Cryptography Library (Zig 0.16.0) + +Comprehensive cryptographic primitives: hashing, encryption, signatures, key exchange, password hashing, and secure utilities. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +## Zig 0.16 Entropy Rule + +Do not use `std.crypto.random` directly in new Zig 0.16 code. Entropy is owned by `std.Io`. + +```zig +var key: [32]u8 = undefined; +io.random(&key); + +const rng_source: std.Random.IoSource = .{ .io = io }; +const rng = rng_source.interface(); +``` + +Use `io.randomSecure(&bytes)` when fresh OS-backed secure entropy is required and failures should be reported. + +Zig 0.16 also adds AES-SIV, AES-GCM-SIV, and Ascon AEAD/hash constructions. + +## Quick Reference + +| Category | Types/Functions | +|----------|-----------------| +| **Hash** | `hash.sha2.Sha256`, `hash.sha2.Sha512`, `hash.sha3.*`, `hash.Blake3`, `hash.blake2.*`, `hash.Md5`, `hash.Sha1` | +| **AEAD** | `aead.aes_gcm.Aes256Gcm`, `aead.chacha_poly.ChaCha20Poly1305`, `aead.aegis.*` | +| **MAC** | `auth.hmac.*`, `auth.siphash.*`, `auth.cmac.*` | +| **Signatures** | `sign.Ed25519`, `sign.ecdsa.*` | +| **Key Exchange** | `dh.X25519` | +| **KEM** | `kem.ml_kem.*` (post-quantum) | +| **Password** | `pwhash.argon2`, `pwhash.scrypt`, `pwhash.bcrypt`, `pwhash.pbkdf2` | +| **KDF** | `kdf.hkdf.HkdfSha256`, `kdf.hkdf.HkdfSha512` | +| **Random** | Use `io.random`, `io.randomSecure`, or `std.Random.IoSource` | +| **Utilities** | `secureZero`, `timing_safe.*`, `codecs.*` | + +## Choosing Algorithms + +``` +Need encryption? +├─ With authentication → AEAD (Aes256Gcm, ChaCha20Poly1305) +└─ Stream only → stream.chacha.* (usually want AEAD instead) + +Need hashing? +├─ General purpose → Sha256, Sha512, Blake3 +├─ Password storage → argon2, scrypt, bcrypt +└─ Legacy compatibility → Md5, Sha1 (NOT secure for new designs) + +Need signatures? +├─ Standard choice → Ed25519 +└─ ECDSA compatibility → ecdsa.EcdsaP256Sha256 + +Need key exchange? +├─ Standard choice → X25519 +└─ Post-quantum → ml_kem.* (Kyber) + +Need MAC? +├─ With key → HmacSha256, HmacSha512 +└─ Hash table keying → siphash +``` + +## Hashing + +### SHA-2 Family + +```zig +const std = @import("std"); +const sha2 = std.crypto.hash.sha2; + +// One-shot hashing +var digest: [sha2.Sha256.digest_length]u8 = undefined; +sha2.Sha256.hash("hello world", &digest, .{}); + +// Streaming (incremental) +var hasher = sha2.Sha256.init(.{}); +hasher.update("hello "); +hasher.update("world"); +hasher.final(&digest); + +// Peek at intermediate digest without consuming state +const intermediate = hasher.peek(); +``` + +Available: `Sha224`, `Sha256`, `Sha384`, `Sha512`, `Sha512_224`, `Sha512_256` + +### SHA-3 Family + +```zig +const sha3 = std.crypto.hash.sha3; + +var digest: [sha3.Sha3_256.digest_length]u8 = undefined; +sha3.Sha3_256.hash("data", &digest, .{}); + +// SHAKE (extendable output) +var shake = sha3.Shake128.init(.{}); +shake.update("data"); +var output: [64]u8 = undefined; +shake.squeeze(&output); +``` + +Available: `Sha3_224`, `Sha3_256`, `Sha3_384`, `Sha3_512`, `Shake128`, `Shake256`, `Keccak256`, `Keccak512` + +### Blake3 + +```zig +const Blake3 = std.crypto.hash.Blake3; + +// Standard hashing +var digest: [Blake3.digest_length]u8 = undefined; +Blake3.hash("data", &digest, .{}); + +// Keyed hashing (MAC) +var keyed: [Blake3.digest_length]u8 = undefined; +Blake3.hash("data", &keyed, .{ .key = key }); + +// Key derivation +var derived: [32]u8 = undefined; +Blake3.hash("material", &derived, .{ .context = "my app v1 key derivation" }); +``` + +### Blake2 + +```zig +const blake2 = std.crypto.hash.blake2; + +// Blake2b (64-byte output) +var digest: [blake2.Blake2b256.digest_length]u8 = undefined; +blake2.Blake2b256.hash("data", &digest, .{}); + +// With key +blake2.Blake2b256.hash("data", &digest, .{ .key = key }); +``` + +Available: `Blake2b128`, `Blake2b256`, `Blake2b384`, `Blake2b512`, `Blake2s128`, `Blake2s224`, `Blake2s256` + +## AEAD (Authenticated Encryption) + +### AES-GCM + +```zig +const Aes256Gcm = std.crypto.aead.aes_gcm.Aes256Gcm; + +// Encryption +var ciphertext: [plaintext.len]u8 = undefined; +var tag: [Aes256Gcm.tag_length]u8 = undefined; +Aes256Gcm.encrypt(&ciphertext, &tag, plaintext, associated_data, nonce, key); + +// Decryption +var decrypted: [ciphertext.len]u8 = undefined; +try Aes256Gcm.decrypt(&decrypted, &ciphertext, tag, associated_data, nonce, key); +// Returns error.AuthenticationFailed if tag doesn't verify +``` + +Key constants: +- `key_length`: 32 bytes (256 bits) +- `nonce_length`: 12 bytes +- `tag_length`: 16 bytes + +### ChaCha20-Poly1305 + +```zig +const ChaCha20Poly1305 = std.crypto.aead.chacha_poly.ChaCha20Poly1305; + +var ciphertext: [msg.len]u8 = undefined; +var tag: [ChaCha20Poly1305.tag_length]u8 = undefined; + +ChaCha20Poly1305.encrypt(&ciphertext, &tag, msg, ad, nonce, key); +try ChaCha20Poly1305.decrypt(&decrypted, &ciphertext, tag, ad, nonce, key); +``` + +Key constants: +- `key_length`: 32 bytes +- `nonce_length`: 12 bytes (IETF) or 24 bytes (XChaCha) +- `tag_length`: 16 bytes + +Available variants: +- `ChaCha20Poly1305` - Standard IETF +- `XChaCha20Poly1305` - Extended nonce (24 bytes, better for random nonces) +- `ChaCha12Poly1305`, `ChaCha8Poly1305` - Reduced rounds (faster, lower security margin) + +### AEGIS + +High-performance AEAD designed for modern CPUs with AES-NI: + +```zig +const Aegis256 = std.crypto.aead.aegis.Aegis256; + +var ciphertext: [msg.len]u8 = undefined; +var tag: [Aegis256.tag_length]u8 = undefined; + +Aegis256.encrypt(&ciphertext, &tag, msg, ad, nonce, key); +try Aegis256.decrypt(&decrypted, &ciphertext, tag, ad, nonce, key); +``` + +## Message Authentication (MAC) + +### HMAC + +```zig +const HmacSha256 = std.crypto.auth.hmac.sha2.HmacSha256; + +// One-shot +var mac: [HmacSha256.mac_length]u8 = undefined; +HmacSha256.create(&mac, message, key); + +// Streaming +var hmac = HmacSha256.init(key); +hmac.update(data1); +hmac.update(data2); +hmac.final(&mac); +``` + +Available: `HmacMd5`, `HmacSha1`, `HmacSha224`, `HmacSha256`, `HmacSha384`, `HmacSha512` + +### SipHash + +Fast MAC for hash table keying (not for general authentication): + +```zig +const SipHash = std.crypto.auth.siphash.SipHash64(2, 4); + +const hash = SipHash.hash(key, data); +``` + +## Digital Signatures + +### Ed25519 + +```zig +const Ed25519 = std.crypto.sign.Ed25519; + +// Generate key pair +const kp = Ed25519.KeyPair.generate(); + +// Sign message +const sig = kp.sign(message, null); + +// Verify signature +try kp.public_key.verify(sig, message); +// Returns error.SignatureVerificationFailed on failure + +// Incremental signing (large messages) +var signer = try kp.signer(null); +signer.update(chunk1); +signer.update(chunk2); +const sig2 = signer.finalize(); +``` + +Key lengths: +- Secret key: 64 bytes +- Public key: 32 bytes +- Signature: 64 bytes + +### ECDSA + +```zig +const EcdsaP256Sha256 = std.crypto.sign.ecdsa.EcdsaP256Sha256; + +// Generate key pair +const kp = EcdsaP256Sha256.KeyPair.generate(); + +// Sign +const sig = try kp.sign(message, null); + +// Verify +try sig.verify(message, kp.public_key); +``` + +Available: `EcdsaP256Sha256`, `EcdsaP256Sha3_256`, `EcdsaP384Sha384`, `EcdsaP384Sha3_384`, `EcdsaSecp256k1Sha256` + +## Key Exchange + +### X25519 (Diffie-Hellman) + +```zig +const X25519 = std.crypto.dh.X25519; + +// Generate key pairs for Alice and Bob +const alice = X25519.KeyPair.generate(); +const bob = X25519.KeyPair.generate(); + +// Compute shared secret +const alice_shared = try X25519.scalarmult(alice.secret_key, bob.public_key); +const bob_shared = try X25519.scalarmult(bob.secret_key, alice.public_key); +// alice_shared == bob_shared + +// IMPORTANT: Hash the shared secret before use +var key: [32]u8 = undefined; +std.crypto.hash.sha2.Sha256.hash(&alice_shared, &key, .{}); +``` + +### ML-KEM (Post-Quantum) + +```zig +const MlKem768 = std.crypto.kem.ml_kem.MlKem768; + +// Key generation +const kp = MlKem768.KeyPair.generate(); + +// Encapsulation (sender) +const encaps = kp.public_key.encaps(null); +const shared_secret = encaps.shared_secret; +const ciphertext = encaps.ciphertext; + +// Decapsulation (receiver) +const decaps_secret = try kp.secret_key.decaps(ciphertext); +// shared_secret == decaps_secret +``` + +Available: `MlKem512`, `MlKem768`, `MlKem1024` + +## Key Derivation + +### HKDF + +```zig +const HkdfSha256 = std.crypto.kdf.hkdf.HkdfSha256; + +// Extract: derive pseudorandom key from input keying material +const prk = HkdfSha256.extract(salt, input_key_material); + +// Expand: derive output key from PRK +var output_key: [32]u8 = undefined; +HkdfSha256.expand(&output_key, context_info, prk); + +// Streaming extract (large IKM) +var hkdf = HkdfSha256.extractInit(salt); +hkdf.update(ikm_part1); +hkdf.update(ikm_part2); +var prk2: [HkdfSha256.prk_length]u8 = undefined; +hkdf.final(&prk2); +``` + +## Password Hashing + +### Argon2 + +Memory-hard password hashing (recommended for new applications): + +```zig +const argon2 = std.crypto.pwhash.argon2; + +// Hash password +var hash: [32]u8 = undefined; +try argon2.kdf( + allocator, + &hash, + password, + salt, + .{ + .t = 3, // time cost (iterations) + .m = 65536, // memory cost (KiB) + .p = 4, // parallelism + }, + .argon2id, // mode: argon2i, argon2d, or argon2id +); + +// Use preset parameters +try argon2.kdf(allocator, &hash, password, salt, argon2.Params.interactive_2id, .argon2id); + +// PHC string format (for storage) +var buf: [128]u8 = undefined; +const encoded = try argon2.strHash(password, salt, .interactive_2id, .argon2id, &buf); +// Returns: "$argon2id$v=19$m=65536,t=3,p=4$..." + +// Verify PHC-encoded hash +try argon2.strVerify(encoded, password, null); +``` + +Parameter presets: +- `interactive_2id`: Fast verification (login forms) +- `moderate_2id`: Balanced +- `sensitive_2id`: High security (key derivation) +- `owasp_2id`: OWASP recommended + +### Scrypt + +Memory-hard KDF: + +```zig +const scrypt = std.crypto.pwhash.scrypt; + +var hash: [32]u8 = undefined; +try scrypt.kdf( + allocator, + &hash, + password, + salt, + .{ .ln = 17, .r = 8, .p = 1 }, // N=2^17, r=8, p=1 +); + +// Presets +try scrypt.kdf(allocator, &hash, password, salt, scrypt.Params.interactive); +``` + +### bcrypt + +```zig +const bcrypt = std.crypto.pwhash.bcrypt; + +// Hash password +var hash: [bcrypt.hash_length]u8 = undefined; +try bcrypt.strHash(password, .{ .rounds = 10 }, &hash); + +// Verify +try bcrypt.strVerify(hash_str, password); +``` + +### PBKDF2 + +```zig +const pbkdf2 = std.crypto.pwhash.pbkdf2; +const HmacSha256 = std.crypto.auth.hmac.sha2.HmacSha256; + +var key: [32]u8 = undefined; +pbkdf2(HmacSha256, &key, password, salt, 100000); // 100k iterations +``` + +## Secure Random + +Thread-local cryptographically secure PRNG: + +```zig +const random = std.crypto.random; + +// Random bytes +var key: [32]u8 = undefined; +random.bytes(&key); + +// Random integers +const n = random.int(u64); +const bounded = random.intRangeLessThan(u32, 0, 100); // [0, 100) + +// Random float [0, 1) +const f = random.float(f64); + +// Shuffle +random.shuffle(u32, &items); +``` + +## Secure Utilities + +### secureZero + +Securely erase sensitive data (prevents optimizer from removing): + +```zig +var secret: [32]u8 = undefined; +// ... use secret ... +std.crypto.secureZero(u8, &secret); // guaranteed to zero +``` + +### Timing-Safe Operations + +```zig +const timing_safe = std.crypto.timing_safe; + +// Constant-time equality (for MACs, signatures) +const equal = timing_safe.eql([32]u8, mac1, mac2); + +// Constant-time comparison +const order = timing_safe.compare(u8, &a, &b, .big); // .lt, .eq, .gt + +// Constant-time arithmetic +const overflow = timing_safe.add(u8, &a, &b, &result, .big); +const underflow = timing_safe.sub(u8, &a, &b, &result, .big); +``` + +### Codecs (Constant-Time) + +```zig +const codecs = std.crypto.codecs; + +// Hex encoding (constant-time) +var hex: [64]u8 = undefined; +try codecs.hex.encode(&hex, &binary, .lower); + +// Hex decoding +var decoded: [32]u8 = undefined; +try codecs.hex.decode(&decoded, &hex); + +// Base64 +const base64 = codecs.base64; +// Similar API to hex +``` + +## Elliptic Curve Primitives + +Low-level curve operations (usually use higher-level APIs): + +```zig +const ecc = std.crypto.ecc; + +// Edwards25519 +const point = ecc.Edwards25519.basePoint; +const result = try point.mul(scalar); + +// P-256 (NIST) +const p256_point = ecc.P256.basePoint; + +// Ristretto255 (prime-order group) +const ristretto = ecc.Ristretto255.basePoint; +``` + +Available: `Curve25519`, `Edwards25519`, `Ristretto255`, `P256`, `P384`, `Secp256k1` + +## Error Handling + +```zig +const errors = std.crypto.errors; + +// Common errors +error.AuthenticationFailed // MAC/tag verification failed +error.SignatureVerificationFailed +error.IdentityElement // Degenerate point in ECC +error.NonCanonical // Input not in canonical form +error.InvalidEncoding // Malformed input +error.WeakPublicKey // Unsafe public key +error.PasswordVerificationFailed +``` + +## Common Patterns + +### Encrypt-then-MAC + +```zig +// Use AEAD instead - it handles this correctly +const Aes256Gcm = std.crypto.aead.aes_gcm.Aes256Gcm; +Aes256Gcm.encrypt(&ct, &tag, pt, ad, nonce, key); +``` + +### Key Generation + +```zig +// For symmetric keys +var key: [32]u8 = undefined; +std.crypto.random.bytes(&key); + +// For asymmetric keys +const kp = std.crypto.sign.Ed25519.KeyPair.generate(); +``` + +### Nonce Management + +```zig +// Option 1: Counter (deterministic, never reuse) +var nonce: [12]u8 = undefined; +std.mem.writeInt(u64, nonce[0..8], counter, .big); +@memset(nonce[8..], 0); +counter += 1; + +// Option 2: Random (safe with XChaCha's 24-byte nonce) +const XChaCha = std.crypto.aead.chacha_poly.XChaCha20Poly1305; +var nonce: [XChaCha.nonce_length]u8 = undefined; +std.crypto.random.bytes(&nonce); +``` + +### Secure Password Storage + +```zig +const argon2 = std.crypto.pwhash.argon2; + +// Registration: hash and store +var buf: [128]u8 = undefined; +const hash_str = try argon2.strHash(password, null, .interactive_2id, .argon2id, &buf); +// Store hash_str in database + +// Login: verify +argon2.strVerify(stored_hash, password, null) catch |err| { + if (err == error.PasswordVerificationFailed) { + // Invalid password + } +}; +``` + +## Side-Channel Protection + +Configure side-channel mitigations: + +```zig +const SideChannelsMitigations = std.crypto.SideChannelsMitigations; + +// Available levels: +// .none - Fastest, no mitigations +// .basic - Protects against most practical attacks +// .medium - Default, good balance (increased resistance) +// .full - Highest protection, significant performance impact + +// Default is .medium +const default = std.crypto.default_side_channels_mitigations; +``` + +## Notes + +- **Never use MD5 or SHA1 for security** - only for legacy compatibility +- **AEAD over separate encrypt+MAC** - AES-GCM or ChaCha20-Poly1305 handle this correctly +- **Hash shared secrets** - X25519 output should be passed through a KDF before use +- **Use argon2id for passwords** - it's the current best practice +- **XChaCha for random nonces** - 24-byte nonce has negligible collision probability +- **Timing attacks** - use `timing_safe.eql` for comparing secrets, not `==` or `std.mem.eql` +- **Zero secrets** - always `secureZero` sensitive data when done diff --git a/references/std-debug.md b/references/std-debug.md new file mode 100644 index 0000000..1eaa2a6 --- /dev/null +++ b/references/std-debug.md @@ -0,0 +1,470 @@ +# std.debug (Zig 0.16.0) + +Debugging utilities: panic handling, assertions, stack traces, hex dumps, and value tracing. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 reworked debug information and expanded target support for segfault handling/unwinding. When debug output writes to stderr/stdout directly, use `std.Io.File.stderr().writer(io, &buf)` / `std.Io.File.stdout().writer(io, &buf)` rather than old `std.io` or `std.fs.File` APIs. + +## Quick Reference + +| Function | Purpose | +|----------|---------| +| `print(fmt, args)` | Printf-style debug output to stderr | +| `panic(fmt, args)` | Format message and abort | +| `assert(bool)` | Crash if false (optimized out in ReleaseFast) | +| `dumpCurrentStackTrace(addr)` | Print stack trace to stderr | +| `dumpHex(bytes)` | Print hexdump to stderr | + +## Debug Printing + +```zig +const std = @import("std"); + +// Quick debug output (64-byte buffer, auto-flush) +std.debug.print("value: {}\n", .{x}); +std.debug.print("name: {s}, count: {d}\n", .{name, count}); + +// Print without newline +std.debug.print("loading...", .{}); +``` + +**Note:** `std.debug.print` silently ignores errors. For production logging, use `std.log`. + +## Format Specifiers + +Format string syntax: `{[arg]:[fill][alignment][width][.precision][specifier]}` + +### Type Specifiers + +| Specifier | Types | Output | +|-----------|-------|--------| +| `{}` | any | Default formatting | +| `{s}` | `[]const u8`, `[*:0]const u8` | String | +| `{d}` | int, float, enum | Decimal | +| `{b}` | int, enum | Binary | +| `{o}` | int, enum | Octal | +| `{x}` | int, float, `[]u8`, enum | Lowercase hex | +| `{X}` | int, float, `[]u8`, enum | Uppercase hex | +| `{c}` | u8, u21 | ASCII character | +| `{u}` | u21 | Unicode codepoint | +| `{e}` | float | Scientific notation | +| `{*}` | pointer | Address (`Type@0x...`) | +| `{f}` | has `format` method | Custom formatter | +| `{any}` | any | Debug representation with depth limit | + +### Examples + +```zig +std.debug.print("{d}\n", .{42}); // "42" +std.debug.print("{x}\n", .{255}); // "ff" +std.debug.print("{X}\n", .{255}); // "FF" +std.debug.print("{b}\n", .{5}); // "101" +std.debug.print("{o}\n", .{64}); // "100" +std.debug.print("{s}\n", .{"hello"}); // "hello" +std.debug.print("{c}\n", .{'A'}); // "A" +std.debug.print("{*}\n", .{&value}); // "i32@7fff5fbff8a0" + +// Floats +std.debug.print("{d}\n", .{3.14159}); // "3.14159" +std.debug.print("{e}\n", .{1234.5}); // "1.2345e+03" +std.debug.print("{x}\n", .{@as(f32, 1.0)}); // "0x1.0p0" + +// Hex dump of bytes +std.debug.print("{x}\n", .{"hello"}); // "68656c6c6f" +``` + +### Width and Alignment + +```zig +std.debug.print("{d:5}\n", .{42}); // " 42" (right-aligned, width 5) +std.debug.print("{d:<5}\n", .{42}); // "42 " (left-aligned) +std.debug.print("{d:^5}\n", .{42}); // " 42 " (center-aligned) +std.debug.print("{d:0>5}\n", .{42}); // "00042" (zero-padded) +std.debug.print("{s:_<10}\n", .{"hi"}); // "hi________" (custom fill) +``` + +### Precision + +```zig +std.debug.print("{d:.2}\n", .{3.14159}); // "3.14" +std.debug.print("{e:.3}\n", .{1234.5}); // "1.234e+03" +std.debug.print("{x:.4}\n", .{@as(f32, 1.0)}); // "0x1.0000p0" +``` + +### Named and Positional Arguments + +```zig +// Positional +std.debug.print("{0} {1} {0}\n", .{"a", "b"}); // "a b a" + +// Named (with struct) +std.debug.print("{name}: {value}\n", .{ .name = "x", .value = 42 }); + +// Runtime width/precision +std.debug.print("{d:[width]}\n", .{ .width = 5, 42 }); +std.debug.print("{d:.[precision]}\n", .{ .precision = 2, 3.14159 }); +``` + +### Escape Braces + +```zig +std.debug.print("{{literal braces}}\n", .{}); // "{literal braces}" +``` + +### Custom Format Method + +Types can implement a `format` method for `{f}`: + +```zig +const Point = struct { + x: f32, + y: f32, + + pub fn format(self: @This(), writer: *std.Io.Writer) std.Io.Writer.Error!void { + try writer.print("({d:.2}, {d:.2})", .{ self.x, self.y }); + } +}; + +const p = Point{ .x = 1.5, .y = 2.5 }; +std.debug.print("{f}\n", .{p}); // "(1.50, 2.50)" +``` + +### Any Format (Debug Representation) + +```zig +const data = .{ .x = 1, .list = &[_]u8{ 1, 2, 3 } }; +std.debug.print("{any}\n", .{data}); +// Prints struct with depth-limited recursion +``` + +## Assertions + +```zig +// Runtime assertion (triggers illegal instruction on failure) +std.debug.assert(x > 0); +std.debug.assert(ptr != null); + +// Debug/ReleaseSafe: generates check +// ReleaseFast/ReleaseSmall: optimized away (undefined behavior if false) +``` + +### Specialized Assertions + +```zig +// Assert slice is readable (checks memory mapping) +std.debug.assertReadable(slice); + +// Assert pointer alignment +std.debug.assertAligned(ptr, .@"16"); // 16-byte alignment +``` + +## Panic + +```zig +// Formatted panic message +std.debug.panic("invalid state: {}", .{state}); + +// With explicit return address +std.debug.panicExtra(@returnAddress(), "error: {s}", .{msg}); +``` + +Panic prints message + stack trace to stderr, then aborts. + +## Stack Traces + +### Dump Current Stack + +```zig +// Print current stack trace to stderr +std.debug.dumpCurrentStackTrace(null); + +// Skip frames until this address +std.debug.dumpCurrentStackTrace(@returnAddress()); +``` + +### Dump to Writer + +```zig +var buf: [4096]u8 = undefined; +var stderr = std.Io.File.stderr().writer(io, &buf); +try std.debug.dumpCurrentStackTraceToWriter(null, &stderr.interface); +``` + +### Capture Stack Trace + +```zig +var addrs: [32]usize = undefined; +var trace: std.builtin.StackTrace = .{ + .instruction_addresses = &addrs, + .index = 0, +}; +std.debug.captureStackTrace(@returnAddress(), &trace); + +// Later: print captured trace +std.debug.dumpStackTrace(trace); +``` + +### StackIterator + +Walk the stack manually: + +```zig +var it = std.debug.StackIterator.init(@returnAddress(), null); +defer it.deinit(); + +while (it.next()) |return_address| { + const addr = return_address -| 1; + std.debug.print("0x{x}\n", .{addr}); +} +``` + +## Hex Dump + +```zig +const data = "Hello, World!\x00\x01\x02"; + +// Quick dump to stderr +std.debug.dumpHex(data); +// Output: +// 7fff5fbff8a0 48 65 6C 6C 6F 2C 20 57 6F 72 6C 64 21 00 01 02 Hello, World!... + +// Dump to writer +var buf: [256]u8 = undefined; +var aw: std.io.Writer.Allocating = .init(allocator); +defer aw.deinit(); +try std.debug.dumpHexFallible(&aw.writer, .no_color, data); +``` + +Output format: +- Address (lowercase hex) +- 16 bytes per line (uppercase hex) +- ASCII representation (`.` for non-printable, special chars for `\n`, `\r`, `\t`) + +## Value Tracing + +Track where values originate and mutate during debugging: + +```zig +const Trace = std.debug.Trace; // Pre-configured: 2 traces, 4 stack frames + +const MyStruct = struct { + value: u32, + trace: Trace = .init, + + fn setValue(self: *@This(), v: u32) void { + self.value = v; + self.trace.add("setValue called"); + } +}; + +var s = MyStruct{ .value = 0 }; +s.setValue(42); +s.trace.dump(); // Prints stack traces with notes +``` + +### Configurable Trace + +```zig +// Custom configuration: 4 trace slots, 8 stack frames per trace +const MyTrace = std.debug.ConfigurableTrace(4, 8, true); + +var trace: MyTrace = .init; +trace.add("first mutation"); +trace.addAddr(@returnAddress(), "with explicit address"); + +// Check if tracing is enabled +if (MyTrace.enabled) { + trace.dump(); +} + +// Use in format strings +std.debug.print("trace: {}", .{trace}); +``` + +In release builds (`enabled = false`), all trace operations are no-ops with zero size. + +## SafetyLock + +Debug helper to detect concurrent access violations: + +```zig +const SafetyLock = std.debug.SafetyLock; + +var lock: SafetyLock = .{}; + +fn criticalSection() void { + lock.lock(); + defer lock.unlock(); + // ... protected code +} + +fn checkNotLocked() void { + lock.assertUnlocked(); // Panics if locked +} +``` + +- In Debug/ReleaseSafe: actively tracks lock state +- In ReleaseFast/ReleaseSmall: all methods are no-ops + +## Source Location + +```zig +const SourceLocation = std.debug.SourceLocation; + +const loc: SourceLocation = .{ + .line = 42, + .column = 10, + .file_name = "src/main.zig", +}; + +// Invalid/unknown location +const unknown = SourceLocation.invalid; +``` + +## Symbol Information + +```zig +const Symbol = std.debug.Symbol; + +// Symbol with resolved source location +const sym: Symbol = .{ + .name = "myFunction", + .compile_unit_name = "main.zig", + .source_location = .{ .line = 100, .column = 1, .file_name = "src/main.zig" }, +}; + +// Unknown symbol +const unknown: Symbol = .{}; // name = "???", compile_unit_name = "???" +``` + +## Segfault Handling + +```zig +// Check if platform supports segfault handling +if (std.debug.have_segfault_handling_support) { + // Attach handler (prints stack trace on SIGSEGV/SIGBUS/etc) + std.debug.attachSegfaultHandler(); + + // Later: reset to default handler + std.debug.resetSegfaultHandler(); +} + +// Check if handler is enabled by default +const enabled = std.debug.default_enable_segfault_handler; +``` + +**Note:** `maybeEnableSegfaultHandler()` is called automatically by the runtime if `std.options.enable_segfault_handler` is true. + +## Thread Context + +Platform-specific CPU register state for stack unwinding: + +```zig +const ThreadContext = std.debug.ThreadContext; + +var ctx: ThreadContext = undefined; +if (std.debug.getContext(&ctx)) { + // ctx now contains register state + std.debug.dumpStackTraceFromBase(&ctx, stderr); +} + +// Copy context (handles internal pointers) +var ctx_copy: ThreadContext = undefined; +std.debug.copyContext(&original_ctx, &ctx_copy); +``` + +## Valgrind Detection + +```zig +if (std.debug.inValgrind()) { + // Running under Valgrind - may want different behavior + std.debug.print("Valgrind detected\n", .{}); +} +``` + +## Debug Info Access + +```zig +// Get debug info for current executable +const info = try std.debug.getSelfDebugInfo(); + +// Get symbol at address +const symbol = try info.getSymbolAtAddress(allocator, address); +defer if (symbol.source_location) |sl| allocator.free(sl.file_name); + +std.debug.print("{s}:{d}: {s}\n", .{ + symbol.source_location.?.file_name, + symbol.source_location.?.line, + symbol.name, +}); +``` + +## Constants + +```zig +// Whether runtime safety checks are enabled +std.debug.runtime_safety // true in Debug/ReleaseSafe + +// Whether platform can produce stack traces +std.debug.sys_can_stack_trace // false on WASM, MIPS, etc. + +// Whether platform has ucontext_t +std.debug.have_ucontext +``` + +## Submodules + +| Module | Purpose | +|--------|---------| +| `std.debug.Dwarf` | DWARF debug info parser | +| `std.debug.Pdb` | Windows PDB debug info parser | +| `std.debug.SelfInfo` | Debug info for current executable | +| `std.debug.MemoryAccessor` | Safe memory access for unwinding | +| `std.debug.Coverage` | Code coverage support | + +## FullPanic + +Create custom panic handler with formatted safety messages: + +```zig +pub const panic = std.debug.FullPanic(myPanicFn); + +fn myPanicFn(msg: []const u8, ret_addr: ?usize) noreturn { + // Custom panic handling (log to file, send telemetry, etc.) + std.posix.abort(); +} + +// Now safety checks use myPanicFn with descriptive messages: +// - "sentinel mismatch: expected X, found Y" +// - "index out of bounds: index N, len M" +// - "attempt to unwrap error: ErrorName" +// etc. +``` + +## Locking stderr + +For multi-line debug output without interleaving: + +```zig +// Lock stderr and clear any progress indicators +std.debug.lockStdErr(); +defer std.debug.unlockStdErr(); + +// Safe to write multiple lines +var buf: [256]u8 = undefined; +var stderr = std.Io.File.stderr().writer(io, &buf); +try stderr.interface.writeAll("Line 1\n"); +try stderr.interface.writeAll("Line 2\n"); +try stderr.interface.flush(); +``` + +Or with a writer: + +```zig +var buf: [256]u8 = undefined; +const writer = std.debug.lockStderrWriter(&buf); +defer std.debug.unlockStderrWriter(); + +try writer.print("Complex output: {}\n", .{value}); +``` diff --git a/references/std-enums.md b/references/std-enums.md new file mode 100644 index 0000000..b46bf9d --- /dev/null +++ b/references/std-enums.md @@ -0,0 +1,300 @@ +# std.enums + +Utilities for working with enums: sets, maps, arrays, and iteration backed by bit operations. + +## EnumSet + +Bit-backed set of enum values. Zero allocation, copyable by value. + +```zig +const std = @import("std"); + +const Color = enum { red, green, blue, yellow }; +const ColorSet = std.enums.EnumSet(Color); + +// Initialize +var colors = ColorSet.initEmpty(); +var all = ColorSet.initFull(); + +// Struct-style init +var primary = ColorSet.init(.{ + .red = true, + .green = true, + .blue = true, + .yellow = false, +}); + +// From slice +var some = ColorSet.initMany(&.{ .red, .blue }); + +// Single element +var just_red = ColorSet.initOne(.red); +``` + +## EnumSet Operations + +```zig +// Insert/remove +colors.insert(.red); +colors.remove(.blue); +colors.toggle(.green); +colors.setPresent(.yellow, true); + +// Check +if (colors.contains(.red)) { + // red is in set +} + +const n = colors.count(); // number of elements + +// Set operations (in-place) +colors.setUnion(other); // add all from other +colors.setIntersection(other); // keep only common +colors.toggleSet(other); // XOR +colors.toggleAll(); // invert all + +// Set operations (return new set) +const u = colors.unionWith(other); +const i = colors.intersectWith(other); +const x = colors.xorWith(other); +const d = colors.differenceWith(other); // colors - other +const c = colors.complement(); // all except colors + +// Comparison +if (colors.eql(other)) { } +if (colors.subsetOf(other)) { } +if (colors.supersetOf(other)) { } +``` + +## EnumSet Iteration + +```zig +var it = colors.iterator(); +while (it.next()) |color| { + std.debug.print("{}\n", .{color}); +} +``` + +## EnumMap + +Map from enum to value. Fixed-size, zero allocation. + +```zig +const Color = enum { red, green, blue }; +const ColorMap = std.enums.EnumMap(Color, u32); + +// Empty map +var map = ColorMap{}; + +// Struct-style init (null = not present) +var scores = ColorMap.init(.{ + .red = 100, + .green = 50, + .blue = null, // not in map +}); +``` + +## EnumMap Operations + +```zig +// Insert +map.put(.red, 42); + +// Get +if (map.get(.red)) |value| { + std.debug.print("red = {}\n", .{value}); +} + +// Get with default +const value = map.getOrDefault(.blue, 0); + +// Get pointer +if (map.getPtr(.red)) |ptr| { + ptr.* += 1; // modify in place +} + +// Remove +map.remove(.red); + +// Check +if (map.contains(.red)) { } + +// Count +const n = map.count(); +``` + +## EnumMap Iteration + +```zig +// Iterate entries +var it = map.iterator(); +while (it.next()) |entry| { + std.debug.print("{}: {}\n", .{ entry.key, entry.value.* }); +} + +// Iterate keys only +var key_it = map.keyIterator(); +while (key_it.next()) |key| { + std.debug.print("{}\n", .{key}); +} +``` + +## EnumArray + +Dense array indexed by enum. All values always present. + +```zig +const Color = enum { red, green, blue }; +const ColorArray = std.enums.EnumArray(Color, u32); + +// Initialize all to same value +var arr = ColorArray.initFill(0); + +// Struct-style init (all must be present) +var rgb = ColorArray.init(.{ + .red = 255, + .green = 128, + .blue = 64, +}); + +// Access +const r = rgb.get(.red); // 255 +rgb.set(.green, 200); +rgb.getPtr(.blue).* = 100; +``` + +## EnumArray Iteration + +```zig +// By key +for (std.enums.values(Color)) |color| { + std.debug.print("{}: {}\n", .{ color, rgb.get(color) }); +} + +// Direct slice access +const slice = rgb.values; // [3]u32 +``` + +## EnumIndexer + +Convert between enum values and dense indices. + +```zig +const Indexer = std.enums.EnumIndexer(Color); + +const idx = Indexer.indexOf(.green); // 1 +const color = Indexer.keyForIndex(1); // .green +const count = Indexer.count; // 3 +``` + +## Utility Functions + +```zig +// Get all values as slice +const colors = std.enums.values(Color); // [3]Color + +// Safe tag name (works with non-exhaustive) +const name = std.enums.tagName(Color, .red); // "red" or null + +// Safe int-to-enum +const maybe = std.enums.fromInt(Color, 1); // ?.green +``` + +## Direct Enum Array (Sparse Enums) + +For enums with gaps in values: + +```zig +const Sparse = enum(u8) { a = 1, b = 5, c = 10 }; + +// Create array indexed by enum int value +const arr = std.enums.directEnumArray( + Sparse, + bool, + 8, // max_unused_slots (gaps allowed) + .{ .a = true, .b = false, .c = true }, +); +// arr is [11]bool, indexed by @intFromEnum +``` + +## Complete Example: Permission System + +```zig +const std = @import("std"); + +const Permission = enum { + read, + write, + execute, + admin, +}; + +const Permissions = std.enums.EnumSet(Permission); + +const User = struct { + name: []const u8, + perms: Permissions, +}; + +fn canAccess(user: User, required: Permissions) bool { + // User must have all required permissions + return required.subsetOf(user.perms); +} + +pub fn main() void { + const admin = User{ + .name = "admin", + .perms = Permissions.initFull(), + }; + + const reader = User{ + .name = "reader", + .perms = Permissions.initOne(.read), + }; + + const write_required = Permissions.initMany(&.{ .read, .write }); + + std.debug.print("admin can write: {}\n", .{canAccess(admin, write_required)}); // true + std.debug.print("reader can write: {}\n", .{canAccess(reader, write_required)}); // false +} +``` + +## Complete Example: State Machine Transitions + +```zig +const std = @import("std"); + +const State = enum { idle, running, paused, stopped }; +const Event = enum { start, pause, resume, stop }; + +const TransitionMap = std.enums.EnumMap(Event, State); +const StateTransitions = std.enums.EnumArray(State, TransitionMap); + +const transitions = StateTransitions.init(.{ + .idle = TransitionMap.init(.{ .start = .running, .stop = .stopped }), + .running = TransitionMap.init(.{ .pause = .paused, .stop = .stopped }), + .paused = TransitionMap.init(.{ .resume = .running, .stop = .stopped }), + .stopped = TransitionMap{}, // no transitions from stopped +}); + +fn nextState(current: State, event: Event) ?State { + return transitions.get(current).get(event); +} + +pub fn main() void { + var state = State.idle; + state = nextState(state, .start) orelse state; // -> running + state = nextState(state, .pause) orelse state; // -> paused + state = nextState(state, .resume) orelse state; // -> running + std.debug.print("Final state: {}\n", .{state}); +} +``` + +## Notes + +- `EnumSet`: Bit-backed, use for presence tracking +- `EnumMap`: Sparse, only stores present values +- `EnumArray`: Dense, all values always present +- All are fixed-size, zero-allocation, copyable by value +- Use `std.StaticBitSet` for non-enum integer sets +- Works with non-exhaustive enums (explicit fields only) diff --git a/references/std-fmt.md b/references/std-fmt.md new file mode 100644 index 0000000..7b351bc --- /dev/null +++ b/references/std-fmt.md @@ -0,0 +1,556 @@ +# std.fmt - String Formatting and Parsing (Zig 0.16.0) + +String formatting and parsing utilities: format strings, integer/float parsing, hex encoding/decoding, and custom formatters. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +## Zig 0.16 Formatting Notes + +- `std.fmt.format` is replaced by `std.Io.Writer.print`. +- `std.fmt.Formatter` is renamed to `std.fmt.Alt`. +- `std.fmt.FormatOptions` is renamed to `std.fmt.Options`. +- `std.fmt.bufPrintZ` is renamed to `std.fmt.bufPrintSentinel`. +- The `{D}` duration specifier was removed; format `std.Io.Duration` with `{f}`. + +```zig +try writer.print("{f}", .{std.Io.Duration.fromMilliseconds(250)}); +``` + +Custom formatter signature remains writer-centered: + +```zig +pub fn format(self: @This(), writer: *std.Io.Writer) std.Io.Writer.Error!void { + try writer.print("{s}", .{self.name}); +} +``` + +## Table of Contents +- [Format String Syntax](#format-string-syntax) +- [Format Specifiers](#format-specifiers) +- [Integer Parsing](#integer-parsing) +- [Float Parsing](#float-parsing) +- [Hex Encoding/Decoding](#hex-encodingdecoding) +- [Buffer Printing](#buffer-printing) +- [Allocating Print](#allocating-print) +- [Comptime Print](#comptime-print) +- [Custom Formatters](#custom-formatters) +- [Format String Parser](#format-string-parser) + +## Format String Syntax + +Full syntax: `{[arg]:[fill][alignment][width][.precision][specifier]}` + +### Components + +| Component | Description | Example | +|-----------|-------------|---------| +| `arg` | Argument index or name | `{0}`, `{name}` | +| `fill` | Padding character | `{:0>5}` uses `0` | +| `alignment` | `<` left, `^` center, `>` right | `{:<10}` | +| `width` | Minimum field width | `{:10}` | +| `precision` | Decimal places for floats | `{:.2}` | +| `specifier` | Output format | `{d}`, `{x}`, `{s}` | + +### Examples + +```zig +std.debug.print("{d:0>8}\n", .{42}); // "00000042" +std.debug.print("{s:_^10}\n", .{"hi"}); // "____hi____" +std.debug.print("{d:.2}\n", .{3.14159}); // "3.14" +std.debug.print("{0} {1} {0}\n", .{"a", "b"}); // "a b a" +``` + +### Named Arguments + +```zig +std.debug.print("{name}: {value}\n", .{ .name = "x", .value = 42 }); +``` + +### Runtime Width/Precision + +```zig +std.debug.print("{d:[width]}\n", .{ .width = @as(usize, 8), 42 }); +std.debug.print("{d:.[prec]}\n", .{ .prec = @as(usize, 2), 3.14159 }); +``` + +### Escape Braces + +```zig +std.debug.print("{{literal}}\n", .{}); // "{literal}" +``` + +## Format Specifiers + +### Type Specifiers + +| Specifier | Types | Output | +|-----------|-------|--------| +| `{}` | any | Default formatting | +| `{d}` | int, float, enum | Decimal | +| `{b}` | int, enum | Binary | +| `{o}` | int, enum | Octal | +| `{x}` | int, float, `[]u8`, enum | Lowercase hex | +| `{X}` | int, float, `[]u8`, enum | Uppercase hex | +| `{s}` | `[]const u8`, `[*:0]const u8` | String | +| `{c}` | u8 | ASCII character | +| `{u}` | u21 | UTF-8 codepoint | +| `{e}` | float | Scientific notation | +| `{f}` | has `format` method | Custom formatter | +| `{*}` | pointer | Address (`Type@0x...`) | +| `{?}` | optional | Value or `null` | +| `{!}` | error union | Value or `error.Name` | +| `{any}` | any | Debug representation | + +### Integer Examples + +```zig +std.debug.print("{d}\n", .{255}); // "255" +std.debug.print("{x}\n", .{255}); // "ff" +std.debug.print("{X}\n", .{255}); // "FF" +std.debug.print("{b}\n", .{5}); // "101" +std.debug.print("{o}\n", .{64}); // "100" +std.debug.print("{c}\n", .{'A'}); // "A" +std.debug.print("{u}\n", .{0x1F310}); // globe emoji +``` + +### Float Examples + +```zig +std.debug.print("{d}\n", .{3.14159}); // "3.14159" +std.debug.print("{d:.2}\n", .{3.14159}); // "3.14" +std.debug.print("{e}\n", .{1234.5}); // "1.2345e3" +std.debug.print("{e:.3}\n", .{1234.5}); // "1.234e3" +std.debug.print("{x}\n", .{@as(f32, 1.0)}); // "0x1p0" +std.debug.print("{x:.5}\n", .{@as(f32, 1.0)}); // "0x1.00000p0" +``` + +### Special Float Values + +```zig +std.debug.print("{}\n", .{std.math.nan(f64)}); // "nan" +std.debug.print("{}\n", .{std.math.inf(f64)}); // "inf" +std.debug.print("{}\n", .{-std.math.inf(f64)}); // "-inf" +``` + +### Slice/Array Formatting + +```zig +const bytes: []const u8 = "hello"; +std.debug.print("{s}\n", .{bytes}); // "hello" +std.debug.print("{x}\n", .{bytes}); // "68656c6c6f" +std.debug.print("{any}\n", .{bytes}); // "{ 104, 101, 108, 108, 111 }" +``` + +### Padding and Alignment + +```zig +std.debug.print("{d:5}\n", .{42}); // " 42" (right, default) +std.debug.print("{d:<5}\n", .{42}); // "42 " (left) +std.debug.print("{d:^5}\n", .{42}); // " 42 " (center) +std.debug.print("{d:0>5}\n", .{42}); // "00042" (zero-pad) +std.debug.print("{d:=>5}\n", .{42}); // "===42" (custom fill) +``` + +## Integer Parsing + +### parseInt + +Parse signed or unsigned integers with optional base detection. + +```zig +const std = @import("std"); + +// Explicit base +const a = try std.fmt.parseInt(i32, "-123", 10); // -123 +const b = try std.fmt.parseInt(u32, "ff", 16); // 255 +const c = try std.fmt.parseInt(u8, "101", 2); // 5 + +// Auto-detect base (base = 0) +const d = try std.fmt.parseInt(i32, "0x1f", 0); // 31 (hex) +const e = try std.fmt.parseInt(i32, "0b101", 0); // 5 (binary) +const f = try std.fmt.parseInt(i32, "0o17", 0); // 15 (octal) +const g = try std.fmt.parseInt(i32, "42", 0); // 42 (decimal) + +// Underscores allowed between digits +const h = try std.fmt.parseInt(u32, "1_000_000", 10); // 1000000 +const i = try std.fmt.parseInt(u32, "0xff_ff", 0); // 65535 +``` + +**Errors:** +- `error.InvalidCharacter` - Invalid digit for base, leading/trailing underscore, empty string +- `error.Overflow` - Result doesn't fit in type + +### parseUnsigned + +Parse unsigned integers only (rejects `+` and `-` signs). + +```zig +const a = try std.fmt.parseUnsigned(u16, "65535", 10); // 65535 +const b = try std.fmt.parseUnsigned(u8, "ff", 16); // 255 + +// These return error.InvalidCharacter: +// std.fmt.parseUnsigned(u8, "+10", 10) +// std.fmt.parseUnsigned(u8, "-10", 10) +``` + +### parseIntSizeSuffix + +Parse integers with SI size suffixes (K, M, G, T, P, E, Z, Y, R, Q). + +```zig +const std = @import("std"); + +const a = try std.fmt.parseIntSizeSuffix("2", 10); // 2 +const b = try std.fmt.parseIntSizeSuffix("2B", 10); // 2 +const c = try std.fmt.parseIntSizeSuffix("2k", 10); // 2000 +const d = try std.fmt.parseIntSizeSuffix("2kB", 10); // 2000 +const e = try std.fmt.parseIntSizeSuffix("2Ki", 10); // 2048 (binary) +const f = try std.fmt.parseIntSizeSuffix("2KiB", 10); // 2048 (binary) +const g = try std.fmt.parseIntSizeSuffix("1M", 10); // 1000000 +const h = try std.fmt.parseIntSizeSuffix("1Mi", 10); // 1048576 +const i = try std.fmt.parseIntSizeSuffix("aKiB", 16); // 10240 (hex base) +``` + +### charToDigit / digitToChar + +Convert between characters and digit values. + +```zig +const d = try std.fmt.charToDigit('a', 16); // 10 +const c = std.fmt.digitToChar(10, .lower); // 'a' +const C = std.fmt.digitToChar(10, .upper); // 'A' +``` + +## Float Parsing + +### parseFloat + +Parse floating-point numbers from strings. + +```zig +const std = @import("std"); + +// Decimal notation +const a = try std.fmt.parseFloat(f64, "3.14159"); // 3.14159 +const b = try std.fmt.parseFloat(f32, "-123.456"); // -123.456 +const c = try std.fmt.parseFloat(f64, "1e10"); // 1e10 +const d = try std.fmt.parseFloat(f64, "1.5e-3"); // 0.0015 +const e = try std.fmt.parseFloat(f64, "+0"); // 0.0 +const f = try std.fmt.parseFloat(f64, "-0"); // -0.0 + +// Hexadecimal notation +const g = try std.fmt.parseFloat(f64, "0x1p0"); // 1.0 +const h = try std.fmt.parseFloat(f64, "0x1.8p1"); // 3.0 +const i = try std.fmt.parseFloat(f32, "-0x1p-1"); // -0.5 + +// Special values +const nan = try std.fmt.parseFloat(f64, "nan"); // NaN +const inf = try std.fmt.parseFloat(f64, "inf"); // +Inf +const ninf = try std.fmt.parseFloat(f64, "-inf"); // -Inf + +// Underscores allowed between digits +const j = try std.fmt.parseFloat(f64, "1_234.567_8"); // 1234.5678 +``` + +**Supported types:** `f16`, `f32`, `f64`, `f80`, `f128` + +**Errors:** +- `error.InvalidCharacter` - Invalid format, empty string, invalid underscore placement + +## Hex Encoding/Decoding + +### bytesToHex + +Convert bytes to hexadecimal string. + +```zig +const input = "hello"; +const hex_lower = std.fmt.bytesToHex(input, .lower); // "68656c6c6f" +const hex_upper = std.fmt.bytesToHex(input, .upper); // "68656C6C6F" +``` + +### hexToBytes + +Decode hexadecimal string to bytes. + +```zig +var buf: [32]u8 = undefined; +const decoded = try std.fmt.hexToBytes(&buf, "48656c6c6f"); // "Hello" +``` + +**Errors:** +- `error.InvalidCharacter` - Non-hex character +- `error.InvalidLength` - Odd number of hex digits +- `error.NoSpaceLeft` - Output buffer too small + +### hex + +Convert unsigned integer to little-endian hex bytes. + +```zig +const h = std.fmt.hex(@as(u32, 0xdeadbeef)); // "efbeadde" +``` + +## Buffer Printing + +### bufPrint + +Format into a fixed buffer, returns slice of written data. + +```zig +var buf: [256]u8 = undefined; +const result = try std.fmt.bufPrint(&buf, "Hello {s}!", .{"world"}); +// result = "Hello world!" +``` + +**Errors:** +- `error.NoSpaceLeft` - Buffer too small + +### bufPrintZ + +Format into buffer with null terminator. + +```zig +var buf: [256]u8 = undefined; +const result = try std.fmt.bufPrintZ(&buf, "Hello {s}!", .{"world"}); +// result is [:0]u8 = "Hello world!" (null-terminated) +``` + +### count + +Count characters needed for format (without allocating). + +```zig +const len = std.fmt.count("Value: {d}, Name: {s}", .{ 42, "test" }); +// len = 21 +``` + +## Allocating Print + +### allocPrint + +Format with dynamic allocation. + +```zig +const allocator = std.heap.page_allocator; +const result = try std.fmt.allocPrint(allocator, "Hello {s}!", .{"world"}); +defer allocator.free(result); +// result = "Hello world!" +``` + +### allocPrintSentinel + +Format with allocation and sentinel terminator. + +```zig +const result = try std.fmt.allocPrintSentinel(allocator, "Hello {s}", .{"world"}, 0); +defer allocator.free(result); +// result is [:0]u8 = "Hello world" (null-terminated) +``` + +## Comptime Print + +### comptimePrint + +Format at compile time, returns pointer to comptime-known string. + +```zig +const msg = comptime std.fmt.comptimePrint("Value: {d}", .{100}); +// msg: *const [10:0]u8 = "Value: 100" +``` + +## Custom Formatters + +### Using `{f}` Specifier + +Types with a `format` method use `{f}`: + +```zig +const Point = struct { + x: f32, + y: f32, + + pub fn format(self: @This(), writer: *std.Io.Writer) std.Io.Writer.Error!void { + try writer.print("({d:.2}, {d:.2})", .{ self.x, self.y }); + } +}; + +const p = Point{ .x = 1.5, .y = 2.5 }; +std.debug.print("{f}\n", .{p}); // "(1.50, 2.50)" +``` + +### Alt (Formatter Wrapper) + +Create a type that wraps data with a custom format function. + +```zig +const std = @import("std"); + +fn formatReversed(data: []const u8, writer: *std.Io.Writer) std.Io.Writer.Error!void { + var i = data.len; + while (i > 0) { + i -= 1; + try writer.writeByte(data[i]); + } +} + +const Reversed = std.fmt.Alt([]const u8, formatReversed); + +pub fn main() !void { + const rev = Reversed{ .data = "hello" }; + std.debug.print("{f}\n", .{rev}); // "olleh" +} +``` + +### alt Helper + +Call alternate format methods by name. + +```zig +const Example = struct { + number: u8, + + pub fn asHex(self: @This(), writer: *std.Io.Writer) std.Io.Writer.Error!void { + try writer.print("0x{x:0>2}", .{self.number}); + } +}; + +const ex = Example{ .number = 42 }; +std.debug.print("{f}\n", .{std.fmt.alt(ex, .asHex)}); // "0x2a" +``` + +## Format String Parser + +For implementing custom formatters compatible with std.fmt. + +### Parser + +Stream-based parser for format strings. + +```zig +const std = @import("std"); + +var parser: std.fmt.Parser = .{ .bytes = "hello:world", .i = 0 }; + +// Parse until delimiter +const before = parser.until(':'); // "hello" + +// Consume delimiter +_ = parser.char(); // ':' + +// Check for character +if (parser.maybe('w')) { + // consumed 'w' +} + +// Parse number +parser = .{ .bytes = "42abc", .i = 0 }; +const num = parser.number(); // 42 + +// Peek without consuming +const next = parser.peek(0); // 'a' +``` + +### Placeholder + +Parse format placeholder syntax. + +```zig +const ph = std.fmt.Placeholder.parse("0d:0>8.2"); +// ph.arg = .{ .number = 0 } +// ph.specifier_arg = "d" +// ph.fill = '0' +// ph.alignment = .right +// ph.width = .{ .number = 8 } +// ph.precision = .{ .number = 2 } +``` + +### Specifier + +Argument reference in format string. + +```zig +const Specifier = union(enum) { + none, // {} - auto-increment + number: usize, // {0} - positional + named: []const u8, // {name} - named +}; +``` + +## Utility Functions + +### digits2 + +Fast conversion of 0-99 to two-digit string. + +```zig +const d = std.fmt.digits2(42); // "42" +const z = std.fmt.digits2(7); // "07" +``` + +### printInt + +Print integer to buffer, returns end index. + +```zig +var buf: [32]u8 = undefined; +const end = std.fmt.printInt(&buf, @as(i32, -42), 10, .lower, .{}); +const result = buf[0..end]; // "-42" +``` + +## Types + +### Options + +Formatting options for numbers. + +```zig +const Options = struct { + precision: ?usize = null, + width: ?usize = null, + alignment: Alignment = .right, + fill: u8 = ' ', +}; +``` + +### Number + +Extended options for numeric formatting. + +```zig +const Number = struct { + mode: Mode = .decimal, // .decimal, .binary, .octal, .hex, .scientific + case: Case = .lower, // .lower, .upper + precision: ?usize = null, + width: ?usize = null, + alignment: Alignment = .right, + fill: u8 = ' ', +}; +``` + +### Alignment + +```zig +const Alignment = enum { left, center, right }; +``` + +### Case + +```zig +const Case = enum { lower, upper }; +``` + +## Error Types + +```zig +const ParseIntError = error{ Overflow, InvalidCharacter }; +const ParseFloatError = error{ InvalidCharacter }; +const BufPrintError = error{ NoSpaceLeft }; +``` + +## Constants + +```zig +const default_max_depth = 3; // Default recursion depth for {any} +const hex_charset = "0123456789abcdef"; +``` diff --git a/references/std-fs.md b/references/std-fs.md new file mode 100644 index 0000000..88745a0 --- /dev/null +++ b/references/std-fs.md @@ -0,0 +1,238 @@ +# File System API Reference (Zig 0.16.0) + +Primary release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 migrates file-system operations to `std.Io`. Use `std.Io.Dir`, `std.Io.File`, and an explicit `std.Io` parameter. `std.fs` is now mostly path helpers and deprecated compatibility names. + +## Core Types + +```zig +std.Io.Dir // Directory handle and file-system operations +std.Io.File // File handle, readers, writers, stat, locking +std.Io.File.Atomic +std.Io.File.MemoryMap +std.Io.Dir.path // Path helpers; std.fs.path is deprecated compatibility +``` + +## Current Directory + +```zig +const cwd = std.Io.Dir.cwd(); +``` + +Do not use old `std.fs.cwd()` in new 0.16 code. + +## Opening Files + +```zig +const file = try std.Io.Dir.cwd().openFile(io, "data.txt", .{}); +defer file.close(io); + +const rw = try std.Io.Dir.cwd().openFile(io, "data.txt", .{ + .mode = .read_write, +}); +defer rw.close(io); +``` + +## Creating Files + +```zig +const file = try std.Io.Dir.cwd().createFile(io, "out.txt", .{}); +defer file.close(io); +``` + +Common options remain conceptually similar: truncate, exclusive create, read access, mode/permissions, and locking where supported. + +## Reading Files + +### Stream With Reader + +```zig +const file = try std.Io.Dir.cwd().openFile(io, "data.txt", .{}); +defer file.close(io); + +var buffer: [4096]u8 = undefined; +var file_reader = file.reader(io, &buffer); +const reader = &file_reader.interface; + +while (try reader.takeDelimiter('\n')) |line| { + _ = line; +} +``` + +### Allocate Whole File + +```zig +const bytes = try std.Io.Dir.cwd().readFileAlloc(io, "data.txt", allocator, .limited(1024 * 1024)); +defer allocator.free(bytes); +``` + +The limit uses `std.Io.Limit`. Hitting the limit returns `error.StreamTooLong`. + +### Read To End From Existing File + +```zig +var buffer: [4096]u8 = undefined; +var file_reader = file.reader(io, &buffer); +const bytes = try file_reader.interface.allocRemaining(allocator, .limited(max_size)); +defer allocator.free(bytes); +``` + +## Writing Files + +### Stream With Writer + +```zig +const file = try std.Io.Dir.cwd().createFile(io, "out.txt", .{}); +defer file.close(io); + +var buffer: [4096]u8 = undefined; +var file_writer = file.writer(io, &buffer); +const writer = &file_writer.interface; + +try writer.print("value={d}\n", .{42}); +try writer.writeAll("raw bytes\n"); +try writer.flush(); +``` + +### Write Whole File + +```zig +try std.Io.Dir.cwd().writeFile(io, .{ + .sub_path = "out.txt", + .data = "hello\n", +}); +``` + +## Standard I/O + +```zig +var stdout_buffer: [4096]u8 = undefined; +var stdout_writer = std.Io.File.stdout().writer(io, &stdout_buffer); +try stdout_writer.interface.print("hello\n", .{}); +try stdout_writer.interface.flush(); + +var stderr_buffer: [4096]u8 = undefined; +var stderr_writer = std.Io.File.stderr().writer(io, &stderr_buffer); +try stderr_writer.interface.print("error: {s}\n", .{"message"}); +try stderr_writer.interface.flush(); +``` + +```zig +var stdin_buffer: [4096]u8 = undefined; +var stdin_reader = std.Io.File.stdin().reader(io, &stdin_buffer); + +if (try stdin_reader.interface.takeDelimiter('\n')) |line| { + _ = line; +} +``` + +## Directories + +```zig +var dir = try std.Io.Dir.cwd().openDir(io, "assets", .{}); +defer dir.close(io); + +try std.Io.Dir.cwd().createDir(io, "new-dir", .default_dir); +try std.Io.Dir.cwd().createDirPath(io, "path/to/nested"); +``` + +Use `openDir` options for iteration/access/no-follow behavior as needed. + +## Walking + +Use `walk` for full recursive walks and `walkSelectively` when you want to decide which directories to enter. + +```zig +var walker = try dir.walk(allocator); +defer walker.deinit(); + +while (try walker.next(io)) |entry| { + _ = entry; +} +``` + +```zig +var walker = try dir.walkSelectively(allocator); +defer walker.deinit(); + +while (try walker.next(io)) |entry| { + if (entry.kind == .directory and shouldEnter(entry)) { + try walker.enter(io, entry); + } +} +``` + +## Metadata + +```zig +const stat = try file.stat(io); + +const size = stat.size; +const kind = stat.kind; +const modified = stat.mtime; +const accessed = stat.atime orelse return error.FileAccessTimeUnavailable; +_ = .{ size, kind, modified, accessed }; +``` + +`atime` is optional in 0.16. + +Setting timestamps uses structured options: + +```zig +try file.setTimestamps(io, .{ + .access_timestamp = .init(stat.atime), + .modify_timestamp = .init(stat.mtime), +}); +``` + +## Paths + +`std.Io.Dir.path` / `std.fs.path` functions handle Windows paths more consistently in 0.16. + +`relative`, `relativeWindows`, and `relativePosix` are pure: pass the current directory and optional environment map instead of letting the function query the OS. + +```zig +const cwd_path = try std.process.currentPathAlloc(io, allocator); +defer allocator.free(cwd_path); + +const rel = try std.fs.path.relative(allocator, cwd_path, environ_map, from, to); +defer allocator.free(rel); +``` + +## Atomic Files + +Use `std.Io.File.Atomic` or directory atomic helpers instead of hand-rolled random temporary names. The 0.16 implementation is routed through `std.Io`, including entropy. + +## Absolute Operations + +Many old `std.fs.*Absolute` functions moved to `std.Io.Dir.*Absolute`, for example: + +- `std.fs.openFileAbsolute` -> `std.Io.Dir.openFileAbsolute` +- `std.fs.createFileAbsolute` -> `std.Io.Dir.createFileAbsolute` +- `std.fs.deleteFileAbsolute` -> `std.Io.Dir.deleteFileAbsolute` +- `std.fs.renameAbsolute` -> `std.Io.Dir.renameAbsolute` +- `std.fs.accessAbsolute` -> `std.Io.Dir.accessAbsolute` + +Many `Z` and `W` path-specific helpers were removed. + +## Migration Map + +| Old 0.15-style API | Zig 0.16 API | +|--------------------|--------------| +| `std.fs.cwd()` | `std.Io.Dir.cwd()` | +| `std.fs.File.stdout()` | `std.Io.File.stdout()` | +| `file.close()` | `file.close(io)` | +| `dir.close()` | `dir.close(io)` | +| `file.reader(&buf)` | `file.reader(io, &buf)` | +| `file.writer(&buf)` | `file.writer(io, &buf)` | +| `dir.readFileAlloc(allocator, path, max)` | `dir.readFileAlloc(io, path, allocator, .limited(max))` | +| `file.readToEndAlloc(allocator, max)` | `file.reader(io, &buf).interface.allocRemaining(allocator, .limited(max))` | +| `std.process.getCwdAlloc(allocator)` | `std.process.currentPathAlloc(io, allocator)` | + +## Application Guidance + +- Prefer an application's virtual or asset file-system abstraction for runtime reads when one exists. +- New direct file operations still need an `io` parameter at the API boundary. +- Writes should be explicit and use `std.Io.Dir` / `std.Io.File` until a project abstraction owns write policy. +- Avoid local fallback `Io.Threaded` construction in low-level file helpers; pass or store `io`. diff --git a/references/std-hash.md b/references/std-hash.md new file mode 100644 index 0000000..49b5f08 --- /dev/null +++ b/references/std-hash.md @@ -0,0 +1,512 @@ +# std.hash - Hash Functions (Zig 0.16.0) + +Non-cryptographic hash functions for hash tables, checksums, and data integrity. For cryptographic hashing, use `std.crypto.hash`. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Examples that hash file contents should use `std.Io.Dir`/`std.Io.File` and explicit `std.Io` for file access. + +## Quick Reference + +| Category | Types | +|----------|-------| +| General Purpose | `Wyhash` (default for HashMap), `XxHash64`, `XxHash32`, `XxHash3` | +| Classic | `Fnv1a_32`, `Fnv1a_64`, `Fnv1a_128`, `Murmur2_32`, `Murmur2_64`, `Murmur3_32` | +| Checksum | `Crc32`, `Adler32`, `crc.*` (100+ CRC variants) | +| City | `CityHash32`, `CityHash64` | +| SipHash | `SipHash64`, `SipHash128` (from `std.crypto.siphash`) | +| Auto-hashing | `autoHash`, `autoHashStrat`, `hash.int` | + +## Choosing a Hash Function + +``` +Need a hash for HashMap? +├─ Yes → Use default (Wyhash via std.hash.autoHash) +└─ No → Need checksum/integrity? + ├─ Yes → Crc32 or Adler32 + └─ No → Need speed? + ├─ Yes → XxHash3, XxHash64, or Wyhash + └─ No → Fnv1a (simple), Murmur (portable) +``` + +| Hash | Output | Speed | Use Case | +|------|--------|-------|----------| +| `Wyhash` | 64-bit | Very fast | Default for HashMap, general hashing | +| `XxHash3` | 64-bit | Fastest | Large data, streaming | +| `XxHash64` | 64-bit | Fast | General purpose | +| `Fnv1a_64` | 64-bit | Moderate | Simple, small code size | +| `Murmur3_32` | 32-bit | Fast | Portable, well-tested | +| `CityHash64` | 64-bit | Fast | Google's hash, good distribution | +| `Crc32` | 32-bit | Fast | Checksums, error detection | +| `Adler32` | 32-bit | Very fast | Lightweight checksums | + +## Basic Usage + +### One-Shot Hashing + +```zig +const std = @import("std"); +const hash = std.hash; + +// Wyhash (recommended default) +const h1 = hash.Wyhash.hash(0, "hello world"); + +// XxHash +const h2 = hash.XxHash64.hash(0, "hello world"); +const h3 = hash.XxHash3.hash(0, "hello world"); + +// FNV-1a +const h4 = hash.Fnv1a_64.hash("hello world"); + +// Murmur +const h5 = hash.Murmur2_64.hash("hello world"); +const h6 = hash.Murmur3_32.hash("hello world"); + +// CityHash +const h7 = hash.CityHash64.hash("hello world"); +``` + +### Streaming/Incremental Hashing + +All hashers support incremental updates for large or streaming data: + +```zig +const std = @import("std"); + +// Initialize hasher +var hasher = std.hash.Wyhash.init(0); // 0 is the seed + +// Update with data incrementally +hasher.update("hello "); +hasher.update("world"); + +// Get final hash +const result = hasher.final(); +``` + +### With Seed + +```zig +// Different seeds produce different hashes +const seed: u64 = 12345; + +const h1 = std.hash.Wyhash.hash(seed, "data"); +const h2 = std.hash.XxHash64.hash(seed, "data"); + +// Fnv1a doesn't take a seed in hash(), use init() for streaming +var fnv = std.hash.Fnv1a_64.init(); +fnv.update("data"); +const h3 = fnv.final(); +``` + +## Auto-Hashing (Generic Types) + +`std.hash.autoHash` automatically hashes any Zig type: + +```zig +const std = @import("std"); + +const Point = struct { + x: i32, + y: i32, +}; + +fn hashPoint(p: Point) u64 { + var hasher = std.hash.Wyhash.init(0); + std.hash.autoHash(&hasher, p); + return hasher.final(); +} + +// Works with any hashable type +fn hashAny(value: anytype) u64 { + var hasher = std.hash.Wyhash.init(0); + std.hash.autoHash(&hasher, value); + return hasher.final(); +} + +// Usage +const h1 = hashAny(Point{ .x = 10, .y = 20 }); +const h2 = hashAny(@as(u32, 42)); +const h3 = hashAny(MyEnum.value); +``` + +### Hash Strategy for Pointers + +`autoHashStrat` controls how pointers are hashed: + +```zig +const std = @import("std"); +const Strategy = std.hash.Strategy; + +var hasher = std.hash.Wyhash.init(0); +const data: []const u8 = "hello"; + +// Shallow: hash pointer address only (default for autoHash) +std.hash.autoHashStrat(&hasher, data, .Shallow); + +// Deep: follow pointer, hash contents (one level) +std.hash.autoHashStrat(&hasher, data, .Deep); + +// DeepRecursive: follow all pointers recursively +std.hash.autoHashStrat(&hasher, data, .DeepRecursive); +``` + +| Strategy | Behavior | +|----------|----------| +| `.Shallow` | Hash pointer address, not contents | +| `.Deep` | Follow pointer one level, hash contents | +| `.DeepRecursive` | Follow all pointers, hash all contents | + +## Integer Hashing + +`std.hash.int` provides optimized integer-to-integer hashing: + +```zig +const std = @import("std"); + +// Hash integers directly (preserves type) +const h1: u32 = std.hash.int(@as(u32, 12345)); +const h2: u64 = std.hash.int(@as(u64, 12345)); +const h3: i32 = std.hash.int(@as(i32, -42)); + +// Useful for hash table probing +fn probe(key: u64, attempt: usize) u64 { + return std.hash.int(key +% @as(u64, attempt)); +} +``` + +## Checksum Functions + +### CRC32 + +```zig +const std = @import("std"); +const Crc32 = std.hash.Crc32; + +// One-shot +const checksum = Crc32.hash("data to checksum"); + +// Streaming +var crc = Crc32.init(); +crc.update("data "); +crc.update("to checksum"); +const result = crc.final(); +``` + +### CRC Variants + +Over 100 CRC variants available: + +```zig +const crc = std.hash.crc; + +// Common variants +const Crc32IsoHdlc = crc.Crc32IsoHdlc; // Standard CRC-32 (default) +const Crc32Iscsi = crc.Crc32Iscsi; // CRC-32C (Castagnoli) +const Crc16Usb = crc.Crc16Usb; +const Crc16Modbus = crc.Crc16Modbus; +const Crc8Bluetooth = crc.Crc8Bluetooth; + +// Usage +const checksum = crc.Crc32Iscsi.hash("data"); +``` + +### Custom CRC + +```zig +const std = @import("std"); +const Crc = std.hash.crc.Crc; + +// Define custom CRC +const MyCrc = Crc(u16, .{ + .polynomial = 0x8005, + .initial = 0xFFFF, + .reflect_input = true, + .reflect_output = true, + .xor_output = 0x0000, +}); + +const checksum = MyCrc.hash("data"); +``` + +### Adler32 + +Faster than CRC but weaker error detection: + +```zig +const std = @import("std"); +const Adler32 = std.hash.Adler32; + +const checksum = Adler32.hash("data"); + +// Streaming +var adler = Adler32.init(); +adler.update("data"); +const result = adler.final(); +``` + +## Using with HashMap + +HashMap uses `std.hash.autoHash` by default: + +```zig +const std = @import("std"); + +// Default string HashMap (uses Wyhash internally) +var map = std.StringHashMap(u32).init(allocator); +defer map.deinit(); + +try map.put("key", 42); + +// Custom key type +const Point = struct { + x: i32, + y: i32, +}; + +// AutoHashMap handles hashing automatically +var point_map = std.AutoHashMap(Point, []const u8).init(allocator); +defer point_map.deinit(); + +try point_map.put(.{ .x = 1, .y = 2 }, "origin-ish"); +``` + +### Custom Hash Function for HashMap + +```zig +const std = @import("std"); + +const MyKey = struct { + id: u64, + name: []const u8, +}; + +const MyContext = struct { + pub fn hash(self: @This(), key: MyKey) u64 { + _ = self; + var h = std.hash.Wyhash.init(0); + h.update(std.mem.asBytes(&key.id)); + h.update(key.name); + return h.final(); + } + + pub fn eql(self: @This(), a: MyKey, b: MyKey) bool { + _ = self; + return a.id == b.id and std.mem.eql(u8, a.name, b.name); + } +}; + +var map = std.HashMap(MyKey, u32, MyContext, 80).init(allocator); +``` + +## Hash Function Details + +### Wyhash + +Default hash for `std.HashMap`. Very fast with excellent distribution. + +```zig +const std = @import("std"); +const Wyhash = std.hash.Wyhash; + +// One-shot (most efficient for single use) +const h = Wyhash.hash(seed, data); + +// Streaming +var hasher = Wyhash.init(seed); +hasher.update(chunk1); +hasher.update(chunk2); +const result = hasher.final(); // idempotent, can call multiple times +``` + +### XxHash Family + +High-performance hash functions by Yann Collet: + +```zig +const std = @import("std"); + +// XxHash3 - fastest for large inputs +const h1 = std.hash.XxHash3.hash(0, data); + +// XxHash64 - 64-bit output +const h2 = std.hash.XxHash64.hash(0, data); + +// XxHash32 - 32-bit output +const h3 = std.hash.XxHash32.hash(0, data); + +// Streaming +var hasher = std.hash.XxHash64.init(0); +hasher.update(chunk); +const result = hasher.final(); +``` + +### FNV-1a + +Simple, portable hash. Good for small data: + +```zig +const std = @import("std"); + +// One-shot +const h32 = std.hash.Fnv1a_32.hash("data"); +const h64 = std.hash.Fnv1a_64.hash("data"); +const h128 = std.hash.Fnv1a_128.hash("data"); + +// Streaming +var hasher = std.hash.Fnv1a_64.init(); +hasher.update("hello "); +hasher.update("world"); +const result = hasher.final(); +``` + +### Murmur Hash + +Well-tested, portable hash functions: + +```zig +const std = @import("std"); +const murmur = std.hash.murmur; + +// Murmur2 +const h1 = murmur.Murmur2_32.hash("data"); +const h2 = murmur.Murmur2_64.hash("data"); + +// Murmur3 +const h3 = murmur.Murmur3_32.hash("data"); + +// With seed +const h4 = murmur.Murmur2_32.hashWithSeed("data", 12345); +const h5 = murmur.Murmur2_64.hashWithSeed("data", 12345); + +// Direct integer hashing +const h6 = murmur.Murmur2_32.hashUint32(12345); +const h7 = murmur.Murmur2_64.hashUint64(12345); +``` + +### CityHash + +Google's fast hash function: + +```zig +const std = @import("std"); +const cityhash = std.hash.cityhash; + +const h32 = cityhash.CityHash32.hash("data"); +const h64 = cityhash.CityHash64.hash("data"); + +// With seed +const h64_seeded = cityhash.CityHash64.hashWithSeed("data", 12345); +``` + +### SipHash + +Cryptographically strong for hash table protection: + +```zig +const std = @import("std"); + +// Requires 128-bit key +const key: [16]u8 = .{0} ** 16; + +const h64 = std.hash.SipHash64(2, 4).hash(&key, "data"); +const h128 = std.hash.SipHash128(2, 4).hash(&key, "data"); + +// Default parameters (2-4 rounds) +const SipHash = std.hash.SipHash64(2, 4); +``` + +## Common Patterns + +### Combine Multiple Values + +```zig +fn combineHashes(a: u64, b: u64) u64 { + var hasher = std.hash.Wyhash.init(0); + hasher.update(std.mem.asBytes(&a)); + hasher.update(std.mem.asBytes(&b)); + return hasher.final(); +} + +// Or use autoHash for any type +fn hashPair(comptime T: type, a: T, b: T) u64 { + var hasher = std.hash.Wyhash.init(0); + std.hash.autoHash(&hasher, a); + std.hash.autoHash(&hasher, b); + return hasher.final(); +} +``` + +### File Checksum + +```zig +fn checksumFile(path: []const u8) !u32 { + const file = try std.fs.cwd().openFile(path, .{}); + defer file.close(); + + var crc = std.hash.Crc32.init(); + var buf: [4096]u8 = undefined; + + while (true) { + const n = try file.read(&buf); + if (n == 0) break; + crc.update(buf[0..n]); + } + + return crc.final(); +} +``` + +### Bloom Filter Hash + +```zig +fn bloomHashes(data: []const u8, k: usize) []u64 { + var hashes: [16]u64 = undefined; + const h1 = std.hash.Wyhash.hash(0, data); + const h2 = std.hash.Wyhash.hash(h1, data); + + for (0..k) |i| { + hashes[i] = h1 +% @as(u64, i) *% h2; + } + return hashes[0..k]; +} +``` + +### Consistent Hashing + +```zig +fn consistentHash(key: []const u8, num_buckets: u32) u32 { + const hash = std.hash.XxHash64.hash(0, key); + // Jump consistent hash + var b: i64 = -1; + var j: i64 = 0; + var h = hash; + while (j < num_buckets) { + b = j; + h = h *% 2862933555777941757 +% 1; + j = @intFromFloat(@as(f64, @floatFromInt(b + 1)) * + (@as(f64, 1 << 31) / @as(f64, @floatFromInt((h >> 33) + 1)))); + } + return @intCast(b); +} +``` + +## Performance Notes + +- **Wyhash**: Fastest general-purpose hash, excellent for hash tables +- **XxHash3**: Fastest for large inputs (>256 bytes), uses SIMD when available +- **XxHash64/32**: Good balance of speed and portability +- **Fnv1a**: Simple, small code size, slower for large data +- **Murmur**: Widely compatible, good for cross-platform consistency +- **CityHash**: Fast, optimized for x86 +- **Crc32**: Hardware-accelerated on many platforms +- **Adler32**: Fastest checksum, weaker error detection + +## Notes + +- All hash functions are deterministic (same input = same output) +- Non-cryptographic hashes are NOT suitable for security (use `std.crypto.hash`) +- `autoHash` rejects slices by default to avoid ambiguity; use `autoHashStrat` with explicit strategy +- Streaming APIs (`init`/`update`/`final`) allow hashing data incrementally +- `final()` is idempotent on most hashers (can be called multiple times) +- For HashMap keys, implement custom `hash` and `eql` in a context struct diff --git a/references/std-hashmap.md b/references/std-hashmap.md new file mode 100644 index 0000000..e62952e --- /dev/null +++ b/references/std-hashmap.md @@ -0,0 +1,181 @@ +# std.HashMap / std.AutoHashMap + +Hash maps for key-value storage. Use `AutoHashMap` for simple keys, `StringHashMap` for string keys. + +## Types Overview + +```zig +// AutoHashMap - automatic hash/eql for simple types +std.AutoHashMap(KeyType, ValueType) +std.AutoHashMapUnmanaged(KeyType, ValueType) // no stored allocator + +// StringHashMap - optimized for string keys +std.StringHashMap(ValueType) +std.StringHashMapUnmanaged(ValueType) + +// ArrayHashMap - preserves insertion order, fast iteration +std.ArrayHashMap(K, V, Context, store_hash) +std.StringArrayHashMap(V) +``` + +## AutoHashMap Usage + +```zig +// Initialization +var map = std.AutoHashMap(u32, []const u8).init(allocator); +defer map.deinit(); + +// Insert +try map.put(42, "answer"); + +// Get +if (map.get(42)) |value| { + // value is []const u8 +} + +// Get pointer (for modification) +if (map.getPtr(42)) |ptr| { + ptr.* = "new value"; +} + +// Remove +if (map.fetchRemove(42)) |kv| { + // kv.key, kv.value - removed entry +} +_ = map.remove(42); // returns bool + +// Check existence +const exists = map.contains(42); + +// Count +const n = map.count(); +``` + +## Unmanaged Variant + +```zig +// No stored allocator - pass to each method +var map: std.AutoHashMapUnmanaged(u32, []const u8) = .empty; +defer map.deinit(allocator); + +try map.put(allocator, 42, "answer"); +const val = map.get(42); +``` + +## StringHashMap + +```zig +var map = std.StringHashMap(i32).init(allocator); +defer map.deinit(); + +try map.put("foo", 123); +const val = map.get("foo"); // ?i32 +``` + +## getOrPut Pattern + +Efficient insert-or-update without double lookup: + +```zig +const gop = try map.getOrPut(key); +if (gop.found_existing) { + // Update existing + gop.value_ptr.* += 1; +} else { + // Initialize new entry + gop.value_ptr.* = 1; +} +``` + +## Iteration + +```zig +// Iterate entries +var iter = map.iterator(); +while (iter.next()) |entry| { + const key = entry.key_ptr.*; + const value = entry.value_ptr.*; +} + +// Keys only +for (map.keys()) |key| { } + +// Values only +for (map.values()) |value| { } +``` + +## Capacity + +```zig +try map.ensureTotalCapacity(100); +map.clearRetainingCapacity(); +map.clearAndFree(); +``` + +## Custom Context + +For custom hash/equality functions: + +```zig +const Context = struct { + pub fn hash(self: @This(), key: MyKey) u64 { + _ = self; + // compute hash + } + pub fn eql(self: @This(), a: MyKey, b: MyKey) bool { + _ = self; + // compare + } +}; + +var map = std.HashMap(MyKey, Value, Context, 80).init(allocator); +// Or with context instance: +var map = std.HashMap(MyKey, Value, Context, 80).initContext(allocator, context); +``` + +## ArrayHashMap (Ordered) + +Preserves insertion order, supports indexed access: + +```zig +var map = std.StringArrayHashMap(i32).init(allocator); +defer map.deinit(); + +try map.put("b", 2); +try map.put("a", 1); + +// Iterate in insertion order: "b", "a" +for (map.keys(), map.values()) |k, v| { } + +// Index access +const key = map.keys()[0]; // "b" +const val = map.values()[0]; // 2 + +// Swap remove (O(1) but changes order) +map.swapRemove("b"); + +// Ordered remove (O(n) but preserves order) +map.orderedRemove("a"); +``` + +## Common Patterns + +```zig +// Word frequency counter +var counts = std.StringHashMap(usize).init(allocator); +for (words) |word| { + const gop = try counts.getOrPut(word); + if (gop.found_existing) { + gop.value_ptr.* += 1; + } else { + gop.value_ptr.* = 1; + } +} + +// Cache with owned keys +var cache = std.StringHashMap(Data).init(allocator); +// When inserting, dupe the key if needed: +const key_copy = try allocator.dupe(u8, external_key); +errdefer allocator.free(key_copy); +try cache.put(key_copy, data); +``` diff --git a/references/std-http.md b/references/std-http.md new file mode 100644 index 0000000..73e6c4e --- /dev/null +++ b/references/std-http.md @@ -0,0 +1,709 @@ +# std.http Reference (Zig 0.16.0) + +HTTP client and server implementation with TLS, connection pooling, compression, and WebSocket support. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +HTTP uses the new `std.Io` interface in Zig 0.16. Construct clients with `.io = io` and pass `io` through APIs that perform networking. + +```zig +var client: std.http.Client = .{ + .allocator = allocator, + .io = io, +}; +defer client.deinit(); +``` + +Older examples below may still show 0.15 client construction. Add `.io = io` and prefer `std.Io.net` patterns in new code. + +## Table of Contents +- [HTTP Client](#http-client) +- [HTTP Server](#http-server) +- [WebSocket](#websocket) +- [Core Types](#core-types) +- [Common Patterns](#common-patterns) + +## HTTP Client + +### Quick Fetch (Simple Requests) + +```zig +const std = @import("std"); + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + const allocator = gpa.allocator(); + + var client: std.http.Client = .{ .allocator = allocator }; + defer client.deinit(); + + // Simple GET - response body discarded + const result = try client.fetch(.{ + .location = .{ .url = "https://example.com/api" }, + }); + std.debug.print("Status: {d}\n", .{@intFromEnum(result.status)}); +} +``` + +### Fetch with Response Body + +```zig +var client: std.http.Client = .{ .allocator = allocator }; +defer client.deinit(); + +// Create writer to capture response +var body_buf: [65536]u8 = undefined; +var body_writer: std.Io.Writer = .fixed(&body_buf); + +const result = try client.fetch(.{ + .location = .{ .url = "https://api.example.com/data" }, + .response_writer = &body_writer, +}); + +const body = body_writer.buffered(); +std.debug.print("Response ({d}): {s}\n", .{@intFromEnum(result.status), body}); +``` + +### Fetch with POST Body + +```zig +const result = try client.fetch(.{ + .location = .{ .url = "https://api.example.com/submit" }, + .method = .POST, + .payload = "{\"key\": \"value\"}", + .headers = .{ + .content_type = .{ .override = "application/json" }, + }, + .response_writer = &body_writer, +}); +``` + +### Full Request Control + +For more control over the request lifecycle: + +```zig +var client: std.http.Client = .{ .allocator = allocator }; +defer client.deinit(); + +const uri = try std.Uri.parse("https://api.example.com/resource"); + +var req = try client.request(.GET, uri, .{ + .keep_alive = true, + .headers = .{ + .authorization = .{ .override = "Bearer token123" }, + }, + .extra_headers = &.{ + .{ .name = "X-Custom-Header", .value = "custom-value" }, + }, +}); +defer req.deinit(); + +// Send request (no body for GET) +try req.sendBodiless(); + +// Receive response headers +var redirect_buf: [8192]u8 = undefined; +var response = try req.receiveHead(&redirect_buf); + +std.debug.print("Status: {d} {s}\n", .{ + @intFromEnum(response.head.status), + response.head.reason, +}); + +// Read response body +var reader_buf: [4096]u8 = undefined; +const body_reader = response.reader(&reader_buf); + +while (true) { + const chunk = body_reader.take(4096) catch |err| switch (err) { + error.EndOfStream => break, + else => return err, + }; + // process chunk... +} +``` + +### POST with Request Body + +```zig +var req = try client.request(.POST, uri, .{}); +defer req.deinit(); + +// Set content length and send body +const body = "request body content"; +try req.sendBodyComplete(@constCast(body)); + +// Or for streaming: +req.transfer_encoding = .{ .content_length = body.len }; +var body_writer_buf: [1024]u8 = undefined; +var bw = try req.sendBody(&body_writer_buf); +try bw.writer.writeAll(body); +try bw.end(); + +var response = try req.receiveHead(&.{}); +``` + +### Chunked Transfer Encoding + +```zig +var req = try client.request(.POST, uri, .{}); +defer req.deinit(); + +req.transfer_encoding = .chunked; +var body_writer_buf: [1024]u8 = undefined; +var bw = try req.sendBody(&body_writer_buf); + +// Write chunks +try bw.writer.writeAll("first chunk"); +try bw.writer.writeAll("second chunk"); +try bw.end(); // Sends final chunk marker + +var response = try req.receiveHead(&.{}); +``` + +### Decompressing Response Bodies + +```zig +var response = try req.receiveHead(&redirect_buf); + +var transfer_buf: [64]u8 = undefined; +var decompress: std.http.Decompress = undefined; + +// Allocate decompression buffer based on content encoding +const decompress_buf = switch (response.head.content_encoding) { + .identity => &.{}, + .zstd => try allocator.alloc(u8, std.compress.zstd.default_window_len), + .deflate, .gzip => try allocator.alloc(u8, std.compress.flate.max_window_len), + .compress => return error.UnsupportedCompression, +}; +defer if (decompress_buf.len > 0) allocator.free(decompress_buf); + +const reader = response.readerDecompressing(&transfer_buf, &decompress, decompress_buf); +// reader now returns decompressed bytes +``` + +### Redirect Handling + +```zig +var req = try client.request(.GET, uri, .{ + // Follow up to 5 redirects (default is 3) + .redirect_behavior = .init(5), + // Or disable redirects: + // .redirect_behavior = .not_allowed, + // Or handle manually: + // .redirect_behavior = .unhandled, +}); +defer req.deinit(); + +try req.sendBodiless(); + +// redirect_buf stores the redirect location URI +var redirect_buf: [8192]u8 = undefined; +var response = try req.receiveHead(&redirect_buf); + +// After redirects, req.uri contains the final URI +std.debug.print("Final URL: {s}\n", .{req.uri.path.raw}); +``` + +### Connection Pooling + +Connections are automatically pooled and reused: + +```zig +var client: std.http.Client = .{ .allocator = allocator }; +defer client.deinit(); + +// Configure pool size (default 32) +client.connection_pool.free_size = 64; + +// Configure buffer sizes +client.read_buffer_size = 16384; // default 8192 +client.write_buffer_size = 2048; // default 1024 + +// Connections are reused when host/port/protocol match +for (0..10) |_| { + var req = try client.request(.GET, uri, .{ .keep_alive = true }); + defer req.deinit(); + // ... same connection reused +} +``` + +### Proxy Configuration + +```zig +var client: std.http.Client = .{ .allocator = allocator }; +defer client.deinit(); + +// Load from environment (HTTP_PROXY, HTTPS_PROXY, etc.) +var arena = std.heap.ArenaAllocator.init(allocator); +defer arena.deinit(); +try client.initDefaultProxies(arena.allocator()); + +// Or configure manually: +var proxy: std.http.Proxy = .{ + .protocol = .plain, + .host = "proxy.example.com", + .port = 8080, + .authorization = null, // or "Basic base64credentials" + .supports_connect = true, +}; +client.http_proxy = &proxy; +``` + +### TLS Configuration + +```zig +var client: std.http.Client = .{ .allocator = allocator }; +defer client.deinit(); + +// TLS is enabled by default for https:// +// Configure TLS buffer size (affects memory usage) +client.tls_buffer_size = std.crypto.tls.Client.min_buffer_len; + +// Force certificate rescan on next HTTPS request +client.next_https_rescan_certs = true; + +// Disable TLS at compile time via std.options.http_disable_tls +``` + +## HTTP Server + +### Basic Server + +```zig +const std = @import("std"); +const net = std.net; +const http = std.http; + +pub fn main() !void { + const address = net.Address.initIp4(.{ 127, 0, 0, 1 }, 8080); + var tcp_server = try address.listen(.{}); + defer tcp_server.deinit(); + + while (true) { + const conn = try tcp_server.accept(); + defer conn.stream.close(); + + var read_buf: [8192]u8 = undefined; + var write_buf: [4096]u8 = undefined; + + var reader = conn.stream.reader(&read_buf); + var writer = conn.stream.writer(&write_buf); + + var server = http.Server.init(reader.interface(), &writer.interface); + + const request = server.receiveHead() catch |err| { + std.debug.print("Failed to receive: {}\n", .{err}); + continue; + }; + + try handleRequest(&request); + } +} + +fn handleRequest(request: *http.Server.Request) !void { + const head = request.head; + std.debug.print("{s} {s}\n", .{@tagName(head.method), head.target}); + + // Simple response + try request.respond("Hello, World!", .{ + .status = .ok, + .extra_headers = &.{ + .{ .name = "Content-Type", .value = "text/plain" }, + }, + }); +} +``` + +### Reading Request Body + +```zig +fn handleRequest(request: *http.Server.Request) !void { + // Handle Expect: 100-continue + var body_buf: [4096]u8 = undefined; + const body_reader = try request.readerExpectContinue(&body_buf); + + // Read entire body + var body: std.ArrayList(u8) = .empty; + defer body.deinit(allocator); + + while (true) { + const chunk = body_reader.take(1024) catch |err| switch (err) { + error.EndOfStream => break, + else => return err, + }; + try body.appendSlice(allocator, chunk); + } + + try request.respond("Received", .{}); +} +``` + +### Streaming Response + +```zig +fn handleRequest(request: *http.Server.Request) !void { + var response_buf: [4096]u8 = undefined; + + // Start streaming response (uses chunked transfer encoding by default) + var body = try request.respondStreaming(&response_buf, .{ + .respond_options = .{ + .status = .ok, + .extra_headers = &.{ + .{ .name = "Content-Type", .value = "text/event-stream" }, + }, + }, + }); + + // Write chunks + try body.writer.writeAll("data: chunk 1\n\n"); + try body.flush(); + + try body.writer.writeAll("data: chunk 2\n\n"); + try body.end(); // Finishes chunked response +} +``` + +### Response with Known Length + +```zig +fn handleRequest(request: *http.Server.Request) !void { + const content = "Fixed length response"; + + var response_buf: [1024]u8 = undefined; + var body = try request.respondStreaming(&response_buf, .{ + .content_length = content.len, // Uses Content-Length instead of chunked + .respond_options = .{ .status = .ok }, + }); + + try body.writer.writeAll(content); + try body.end(); +} +``` + +### Iterating Headers + +```zig +fn handleRequest(request: *http.Server.Request) !void { + var it = request.iterateHeaders(); + while (it.next()) |header| { + std.debug.print("{s}: {s}\n", .{header.name, header.value}); + } + + try request.respond("OK", .{}); +} +``` + +## WebSocket + +### Server-Side WebSocket Upgrade + +```zig +fn handleRequest(request: *http.Server.Request) !void { + const upgrade = request.upgradeRequested(); + + switch (upgrade) { + .websocket => |key| { + if (key) |k| { + var ws = try request.respondWebSocket(.{ .key = k }); + try handleWebSocket(&ws); + } else { + try request.respond("Missing key", .{ .status = .bad_request }); + } + }, + .other => |name| { + std.debug.print("Unknown upgrade: {s}\n", .{name}); + try request.respond("Not supported", .{ .status = .bad_request }); + }, + .none => { + try request.respond("Expected WebSocket", .{ .status = .bad_request }); + }, + } +} + +fn handleWebSocket(ws: *http.Server.WebSocket) !void { + try ws.flush(); // Send upgrade response + + while (true) { + const msg = ws.readSmallMessage() catch |err| switch (err) { + error.ConnectionClose => break, + else => return err, + }; + + switch (msg.opcode) { + .text, .binary => { + // Echo back + try ws.writeMessage(msg.data, msg.opcode); + }, + .ping => { + try ws.writeMessage(msg.data, .pong); + }, + else => {}, + } + } +} +``` + +### WebSocket Message Types + +```zig +// Opcodes +const Opcode = enum(u4) { + continuation = 0, + text = 1, + binary = 2, + connection_close = 8, + ping = 9, + pong = 10, +}; + +// Write different message types +try ws.writeMessage("Hello", .text); +try ws.writeMessage(&binary_data, .binary); +try ws.writeMessage("", .ping); + +// Unflushed writes (batch multiple messages) +try ws.writeMessageUnflushed("msg1", .text); +try ws.writeMessageUnflushed("msg2", .text); +try ws.flush(); +``` + +## Core Types + +### HTTP Methods + +```zig +const Method = enum { + GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE, PATCH, + + pub fn requestHasBody(m: Method) bool; // POST, PUT, PATCH + pub fn responseHasBody(m: Method) bool; // GET, POST, DELETE, CONNECT, OPTIONS, PATCH + pub fn safe(m: Method) bool; // GET, HEAD, OPTIONS, TRACE + pub fn idempotent(m: Method) bool; // GET, HEAD, PUT, DELETE, OPTIONS, TRACE + pub fn cacheable(m: Method) bool; // GET, HEAD +}; +``` + +### HTTP Status Codes + +```zig +const Status = enum(u10) { + // 1xx Informational + @"continue" = 100, + switching_protocols = 101, + + // 2xx Success + ok = 200, + created = 201, + accepted = 202, + no_content = 204, + + // 3xx Redirection + moved_permanently = 301, + found = 302, + see_other = 303, + not_modified = 304, + temporary_redirect = 307, + permanent_redirect = 308, + + // 4xx Client Error + bad_request = 400, + unauthorized = 401, + forbidden = 403, + not_found = 404, + method_not_allowed = 405, + too_many_requests = 429, + + // 5xx Server Error + internal_server_error = 500, + not_implemented = 501, + bad_gateway = 502, + service_unavailable = 503, + + _, // Non-exhaustive for custom codes + + pub fn phrase(self: Status) ?[]const u8; + pub fn class(self: Status) Class; +}; + +const Class = enum { informational, success, redirect, client_error, server_error }; +``` + +### Content Encoding + +```zig +const ContentEncoding = enum { + zstd, + gzip, + deflate, + compress, + identity, + + pub fn fromString(s: []const u8) ?ContentEncoding; + pub fn minBufferCapacity(ce: ContentEncoding) usize; +}; +``` + +### Transfer Encoding + +```zig +const TransferEncoding = enum { + chunked, + none, +}; +``` + +### Header Struct + +```zig +const Header = struct { + name: []const u8, + value: []const u8, +}; +``` + +## Common Patterns + +### JSON API Client + +```zig +fn fetchJson(comptime T: type, allocator: Allocator, url: []const u8) !T { + var client: std.http.Client = .{ .allocator = allocator }; + defer client.deinit(); + + var body_buf: [65536]u8 = undefined; + var body_writer: std.Io.Writer = .fixed(&body_buf); + + const result = try client.fetch(.{ + .location = .{ .url = url }, + .headers = .{ + .content_type = .{ .override = "application/json" }, + }, + .response_writer = &body_writer, + }); + + if (result.status != .ok) return error.HttpError; + + const parsed = try std.json.parseFromSlice(T, allocator, body_writer.buffered(), .{}); + return parsed.value; +} +``` + +### POST JSON Data + +```zig +fn postJson(allocator: Allocator, url: []const u8, data: anytype) !void { + const json = try std.json.stringifyAlloc(allocator, data, .{}); + defer allocator.free(json); + + var client: std.http.Client = .{ .allocator = allocator }; + defer client.deinit(); + + const result = try client.fetch(.{ + .location = .{ .url = url }, + .method = .POST, + .payload = json, + .headers = .{ + .content_type = .{ .override = "application/json" }, + }, + }); + + if (result.status.class() != .success) return error.HttpError; +} +``` + +### Download File + +```zig +fn downloadFile(allocator: Allocator, url: []const u8, path: []const u8) !void { + var client: std.http.Client = .{ .allocator = allocator }; + defer client.deinit(); + + const uri = try std.Uri.parse(url); + var req = try client.request(.GET, uri, .{}); + defer req.deinit(); + + try req.sendBodiless(); + + var redirect_buf: [8192]u8 = undefined; + var response = try req.receiveHead(&redirect_buf); + + if (response.head.status != .ok) return error.HttpError; + + const file = try std.fs.cwd().createFile(path, .{}); + defer file.close(); + + var file_buf: [4096]u8 = undefined; + var file_writer = file.writer(&file_buf); + + var reader_buf: [4096]u8 = undefined; + const body_reader = response.reader(&reader_buf); + + _ = body_reader.streamRemaining(&file_writer.interface) catch |err| switch (err) { + error.ReadFailed => return response.bodyErr().?, + else => return err, + }; + + try file_writer.interface.flush(); +} +``` + +### Simple REST Server + +```zig +fn handleApi(request: *http.Server.Request) !void { + const head = request.head; + + if (std.mem.eql(u8, head.target, "/api/health")) { + try request.respond("{\"status\":\"ok\"}", .{ + .extra_headers = &.{ + .{ .name = "Content-Type", .value = "application/json" }, + }, + }); + return; + } + + if (std.mem.startsWith(u8, head.target, "/api/users")) { + switch (head.method) { + .GET => try handleGetUsers(request), + .POST => try handleCreateUser(request), + else => try request.respond("", .{ .status = .method_not_allowed }), + } + return; + } + + try request.respond("Not Found", .{ .status = .not_found }); +} +``` + +### Error Handling + +```zig +fn makeRequest(client: *std.http.Client, uri: std.Uri) ![]const u8 { + var req = client.request(.GET, uri, .{}) catch |err| switch (err) { + error.ConnectionRefused => return error.ServerDown, + error.TlsInitializationFailed => return error.TlsError, + error.UnknownHostName => return error.DnsError, + else => return err, + }; + defer req.deinit(); + + try req.sendBodiless(); + + var buf: [8192]u8 = undefined; + var response = req.receiveHead(&buf) catch |err| switch (err) { + error.HttpHeadersOversize => return error.ResponseTooLarge, + error.HttpHeadersInvalid => return error.MalformedResponse, + error.TooManyHttpRedirects => return error.RedirectLoop, + else => return err, + }; + + if (response.head.status.class() != .success) { + return error.HttpError; + } + + // ... +} +``` diff --git a/references/std-io.md b/references/std-io.md new file mode 100644 index 0000000..eb3aa8a --- /dev/null +++ b/references/std-io.md @@ -0,0 +1,289 @@ +# std.Io - I/O API Reference (Zig 0.16.0) + +Primary release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 makes I/O an explicit interface. All file, network, process, time, entropy, cancelable synchronization, and task APIs that can block or interact with the outside world need a `std.Io`. + +## Ownership Rule + +Prefer this flow: + +1. Application entry receives `init: std.process.Init`. +2. It extracts `const io = init.io`. +3. It passes `io` through setup. +4. Long-lived systems that need I/O store `io`. +5. Tests use `std.testing.io`. + +```zig +pub fn main(init: std.process.Init) !void { + const gpa = init.gpa; + const io = init.io; + _ = .{ gpa, io }; +} +``` + +Temporary adapter only at a boundary: + +```zig +var threaded: std.Io.Threaded = .init_single_threaded; +const io = threaded.io(); +``` + +## Implementations + +- `std.Io.Threaded`: threaded, feature-complete, closest to old blocking behavior. +- `std.Io.Evented`: experimental evented/M:N implementation. +- `std.Io.Uring`, `std.Io.Kqueue`, `std.Io.Dispatch`: platform/proof-of-concept backends. +- `std.Io.failing`: backend that supports no operations. + +## Writer + +`std.Io.Writer` is non-generic. It contains its buffer and vtable. + +### Fixed Buffer Writer + +```zig +var buffer: [256]u8 = undefined; +var writer: std.Io.Writer = .fixed(&buffer); + +try writer.print("name={s} value={d}", .{ "answer", 42 }); +const bytes = writer.buffered(); +``` + +### File Writer + +```zig +const file = try std.Io.Dir.cwd().createFile(io, "out.txt", .{}); +defer file.close(io); + +var buffer: [4096]u8 = undefined; +var file_writer = file.writer(io, &buffer); +const writer = &file_writer.interface; + +try writer.writeAll("hello\n"); +try writer.print("value={d}\n", .{42}); +try writer.flush(); +``` + +`flush` is still required for buffered file/socket writers. + +### Standard Output + +```zig +var buffer: [4096]u8 = undefined; +var stdout_writer = std.Io.File.stdout().writer(io, &buffer); +const stdout = &stdout_writer.interface; + +try stdout.print("hello\n", .{}); +try stdout.flush(); +``` + +### Allocating Writer + +```zig +var aw: std.Io.Writer.Allocating = .init(allocator); +defer aw.deinit(); + +try aw.writer.print("hello {s}", .{"world"}); +const owned = try aw.toOwnedSlice(); +defer allocator.free(owned); +``` + +`Allocating` has an `alignment` field in 0.16. Prefer `.init`, `.initAligned`, or `.initCapacity`. + +### Discarding Writer + +Use this when old code used counting/discarding writer patterns. + +```zig +var buffer: [256]u8 = undefined; +var discarding: std.Io.Writer.Discarding = .init(&buffer); +try discarding.writer.print("ignored {d}", .{123}); +const count = discarding.fullCount(); +``` + +## Reader + +`std.Io.Reader` is non-generic and buffer-aware. + +### Fixed Reader + +```zig +var reader: std.Io.Reader = .fixed("alpha\nbeta\n"); + +while (try reader.takeDelimiter('\n')) |line| { + _ = line; +} +``` + +`takeDelimiter` returns `!?[]u8`: `null` means EOF, not an error. + +### File Reader + +```zig +const file = try std.Io.Dir.cwd().openFile(io, "data.txt", .{}); +defer file.close(io); + +var buffer: [4096]u8 = undefined; +var file_reader = file.reader(io, &buffer); +const reader = &file_reader.interface; + +while (try reader.takeDelimiter('\n')) |line| { + _ = line; +} +``` + +### Read Remaining + +```zig +const bytes = try reader.allocRemaining(allocator, .limited(1024 * 1024)); +defer allocator.free(bytes); +``` + +### Binary Reads + +```zig +const value = try reader.takeInt(u32, .little); +const header = try reader.takeStruct(Header, .little); +const leb = try reader.takeLeb128(u64); +``` + +## File Integration + +Use `std.Io.Dir` and `std.Io.File`. + +```zig +const cwd = std.Io.Dir.cwd(); + +const contents = try cwd.readFileAlloc(io, "data.txt", allocator, .limited(1024 * 1024)); +defer allocator.free(contents); + +try cwd.writeFile(io, .{ + .sub_path = "out.txt", + .data = contents, +}); +``` + +`close`, `stat`, `setTimestamps`, readers, writers, and most operations now take `io`. + +## Networking Integration + +Use `std.Io.net` for sockets/streams and pass `io` to high-level clients. + +```zig +var client: std.http.Client = .{ + .allocator = allocator, + .io = io, +}; +defer client.deinit(); +``` + +## Process Integration + +Use `std.process.run` and `std.process.spawn`. + +```zig +const result = try std.process.run(allocator, io, .{ + .argv = &.{ "zig", "version" }, + .stdout_limit = .limited(16 * 1024), + .stderr_limit = .limited(16 * 1024), +}); +defer allocator.free(result.stdout); +defer allocator.free(result.stderr); +``` + +## Entropy + +```zig +var bytes: [32]u8 = undefined; +io.random(&bytes); + +const source: std.Random.IoSource = .{ .io = io }; +const rng = source.interface(); +``` + +Use `io.randomSecure` for fresh secure entropy with error reporting. + +## Time + +The release notes map: + +- `std.time.Instant` -> `std.Io.Timestamp` +- `std.time.Timer` -> `std.Io.Timestamp` +- `std.time.timestamp` -> `std.Io.Timestamp.now` + +Use a shared application helper when common timestamp reads require consistent clock selection or conversion semantics. + +## Tasks and Cancelation + +`std.Io` provides function-level and operation-level concurrency: + +- `io.async(...)` +- `std.Io.Group` +- `std.Io.Select` +- `std.Io.Batch` +- `std.Io.Queue(T)` + +Cancelation guidance: + +- Propagate `error.Canceled` by default. +- Only the code that requested cancelation should ignore it. +- If handling it locally but continuing, use `io.recancel()` when cancelation should stay active. +- Use `errdefer group.cancel(io)` after creating grouped tasks. + +```zig +var group: std.Io.Group = .init; +errdefer group.cancel(io); + +group.async(io, worker, .{ io, arg }); +try group.await(io); +``` + +## Sync Primitives + +Blocking sync moved to `std.Io` equivalents: + +| Old | New | +|-----|-----| +| `std.Thread.Mutex` | `std.Io.Mutex` | +| `std.Thread.Condition` | `std.Io.Condition` | +| `std.Thread.Semaphore` | `std.Io.Semaphore` | +| `std.Thread.RwLock` | `std.Io.RwLock` | +| `std.Thread.ResetEvent` | `std.Io.Event` | +| `std.Thread.WaitGroup` | `std.Io.Group` | +| `std.Thread.Futex` | `std.Io.Futex` | + +```zig +try mutex.lock(io); +defer mutex.unlock(io); +``` + +```zig +mutex.lockUncancelable(io); +defer mutex.unlock(io); +``` + +## Removed Reader/Writer Names + +Use these replacements: + +| Old | New | +|-----|-----| +| `std.io` | `std.Io` | +| `std.Io.GenericReader` | `std.Io.Reader` | +| `std.Io.AnyReader` | `std.Io.Reader` | +| `std.Io.GenericWriter` | `std.Io.Writer` | +| `std.Io.AnyWriter` | `std.Io.Writer` | +| `std.io.fixedBufferStream(data).reader()` | `var r: std.Io.Reader = .fixed(data)` | +| `std.io.fixedBufferStream(buffer).writer()` | `var w: std.Io.Writer = .fixed(buffer)` | +| `std.leb.readUleb128` / `readIleb128` | `std.Io.Reader.takeLeb128` | + +## Review Checklist + +- Does this API need an `io: std.Io` parameter? +- Is `io` obtained at the application/test boundary rather than constructed deep inside? +- Are all file/socket/process/time/random operations routed through `std.Io`? +- Are blocking locks using `std.Io` primitives? +- Are buffered writers flushed? +- Are cancelation errors propagated unless this code requested cancelation? +- Are temporary `Io.Threaded.init_single_threaded` adapters isolated and documented? diff --git a/references/std-json.md b/references/std-json.md new file mode 100644 index 0000000..2654a4d --- /dev/null +++ b/references/std-json.md @@ -0,0 +1,428 @@ +# std.json - JSON Parsing and Serialization + +JSON RFC 8259 compliant parsing and stringification. In Zig 0.16, streaming/file examples should use `std.Io.Reader`, `std.Io.Writer`, `std.Io.Dir`, and an explicit `std.Io`. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +## Table of Contents +- [Parsing JSON](#parsing-json) +- [Serializing to JSON](#serializing-to-json) +- [Dynamic Values](#dynamic-values) +- [Custom Serialization](#custom-serialization) +- [Streaming API](#streaming-api) +- [Common Patterns](#common-patterns) + +## Parsing JSON + +### Parse into Struct + +```zig +const Config = struct { + name: []const u8, + port: u16, + enabled: bool = true, // default value for missing fields +}; + +const json_str = + \\{"name": "server", "port": 8080} +; + +const parsed = try std.json.parseFromSlice(Config, allocator, json_str, .{}); +defer parsed.deinit(); + +const config = parsed.value; +// config.name == "server" +// config.port == 8080 +// config.enabled == true (default) +``` + +### ParseOptions + +```zig +const parsed = try std.json.parseFromSlice(T, allocator, json_str, .{ + // What to do with duplicate fields + .duplicate_field_behavior = .@"error", // .use_first, .use_last, .@"error" (default) + + // Allow unknown fields (default: error) + .ignore_unknown_fields = true, + + // Max string/number length (default: input length for slices) + .max_value_len = 4096, + + // Parse numbers vs keep as strings + .parse_numbers = true, // default: true +}); +``` + +### Supported Types + +| Zig Type | JSON | +|----------|------| +| `bool` | `true`, `false` | +| `i32`, `u64`, etc. | number or string | +| `f32`, `f64` | number or string | +| `?T` | value or `null` | +| `[]const u8` | string | +| `[N]u8` | string (fixed length) | +| `[]T`, `[N]T` | array | +| `struct` | object | +| `union(enum)` | object with single field | +| `enum` | string | +| `std.json.Value` | any JSON value | + +### Parse into Dynamic Value + +Use `std.json.Value` when structure is unknown at compile time: + +```zig +const parsed = try std.json.parseFromSlice(std.json.Value, allocator, json_str, .{}); +defer parsed.deinit(); + +const value = parsed.value; +switch (value) { + .object => |obj| { + if (obj.get("name")) |name| { + std.debug.print("name: {s}\n", .{name.string}); + } + }, + .array => |arr| { + for (arr.items) |item| { ... } + }, + .string => |s| { ... }, + .integer => |i| { ... }, + .float => |f| { ... }, + .bool => |b| { ... }, + .null => { ... }, + .number_string => |s| { ... }, // unparsed number +} +``` + +### Leaky Parsing (Arena Allocator) + +When using an arena, skip the `Parsed` wrapper: + +```zig +var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator); +defer arena.deinit(); + +const config = try std.json.parseFromSliceLeaky( + Config, + arena.allocator(), + json_str, + .{}, +); +// No deinit needed - arena handles cleanup +``` + +## Serializing to JSON + +### Simple Serialization + +```zig +const config = Config{ .name = "app", .port = 3000 }; + +// To allocated string +const json = try std.json.Stringify.valueAlloc(allocator, config, .{}); +defer allocator.free(json); +// json == {"name":"app","port":3000} + +// To writer +var buf: [4096]u8 = undefined; +var writer = std.fs.File.stdout().writer(&buf); +try std.json.Stringify.value(config, .{}, &writer.interface); +try writer.interface.flush(); +``` + +### Stringify Options + +```zig +try std.json.Stringify.value(data, .{ + // Whitespace formatting + .whitespace = .minified, // default: no whitespace + // .whitespace = .indent_2, // 2-space indent + // .whitespace = .indent_4, // 4-space indent + // .whitespace = .indent_tab, + + // Include null optional fields? (default: true) + .emit_null_optional_fields = false, + + // Emit []u8 as array of numbers instead of string + .emit_strings_as_arrays = false, + + // Escape non-ASCII unicode as \uXXXX + .escape_unicode = false, + + // Large integers as strings for JS compatibility + .emit_nonportable_numbers_as_strings = false, +}, writer); +``` + +### Supported Types for Serialization + +- `bool` → `true`/`false` +- `?T` → value or `null` +- integers → number (or string if > 2^53 with option) +- floats → number (or string if not precisely representable as f64) +- `[]const u8` → string (or array with option) +- `[]T`, `[N]T` → array +- tuples → array +- `struct` → object (fields in declaration order) +- `union(enum)` → object with one field +- `enum` → string +- `*T` → serialization of `T` +- `error` → string + +## Dynamic Values + +### Value Type + +```zig +pub const Value = union(enum) { + null, + bool: bool, + integer: i64, + float: f64, + number_string: []const u8, // unparsed number + string: []const u8, + array: Array, // std.ArrayList(Value) + object: ObjectMap, // StringArrayHashMap(Value) +}; +``` + +### Building Values Manually + +```zig +var obj = std.json.ObjectMap.init(allocator); +try obj.put("name", .{ .string = "test" }); +try obj.put("count", .{ .integer = 42 }); + +var arr = std.json.Array.init(allocator); +try arr.append(.{ .integer = 1 }); +try arr.append(.{ .integer = 2 }); +try obj.put("items", .{ .array = arr }); + +const value = std.json.Value{ .object = obj }; +``` + +### Accessing Values + +```zig +// Object access +if (value.object.get("key")) |v| { + switch (v) { + .string => |s| std.debug.print("{s}\n", .{s}), + else => {}, + } +} + +// Array iteration +for (value.array.items) |item| { + if (item == .integer) { + std.debug.print("{d}\n", .{item.integer}); + } +} +``` + +## Custom Serialization + +### Custom jsonParse + +Define `jsonParse` for custom deserialization: + +```zig +const Point = struct { + x: i32, + y: i32, + + // Parse from "x,y" string format + pub fn jsonParse( + allocator: std.mem.Allocator, + source: anytype, + options: std.json.ParseOptions, + ) !@This() { + _ = allocator; + _ = options; + const token = try source.next(); + const str = switch (token) { + .string, .allocated_string => |s| s, + else => return error.UnexpectedToken, + }; + var it = std.mem.splitScalar(u8, str, ','); + return .{ + .x = try std.fmt.parseInt(i32, it.next() orelse return error.UnexpectedToken, 10), + .y = try std.fmt.parseInt(i32, it.next() orelse return error.UnexpectedToken, 10), + }; + } +}; + +// Parses: "10,20" → Point{ .x = 10, .y = 20 } +``` + +### Custom jsonStringify + +Define `jsonStringify` for custom serialization: + +```zig +const Point = struct { + x: i32, + y: i32, + + pub fn jsonStringify(self: @This(), jw: anytype) !void { + // Serialize as "x,y" string + try jw.print("\"{d},{d}\"", .{ self.x, self.y }); + } +}; + +// Serializes: Point{ .x = 10, .y = 20 } → "10,20" +``` + +## Streaming API + +### Stringify (Write Stream) + +Build JSON incrementally: + +```zig +var out: std.io.Writer.Allocating = .init(allocator); +defer out.deinit(); + +var jw: std.json.Stringify = .{ + .writer = &out.writer, + .options = .{ .whitespace = .indent_2 }, +}; + +try jw.beginObject(); +try jw.objectField("users"); +try jw.beginArray(); + +for (users) |user| { + try jw.beginObject(); + try jw.objectField("name"); + try jw.write(user.name); + try jw.objectField("age"); + try jw.write(user.age); + try jw.endObject(); +} + +try jw.endArray(); +try jw.endObject(); + +const json = out.written(); +``` + +### Scanner (Low-Level Parsing) + +Token-based parsing for streaming: + +```zig +var scanner = std.json.Scanner.initCompleteInput(allocator, json_str); +defer scanner.deinit(); + +while (true) { + const token = try scanner.next(); + switch (token) { + .object_begin => { ... }, + .object_end => { ... }, + .array_begin => { ... }, + .array_end => { ... }, + .string => |s| { ... }, + .number => |n| { ... }, + .true, .false, .null => { ... }, + .end_of_document => break, + else => {}, + } +} +``` + +## Common Patterns + +### Config File Loading + +```zig +const Config = struct { + host: []const u8 = "localhost", + port: u16 = 8080, + debug: bool = false, +}; + +fn loadConfig(allocator: std.mem.Allocator, path: []const u8) !Config { + const file = std.fs.cwd().openFile(path, .{}) catch |err| switch (err) { + error.FileNotFound => return Config{}, // defaults + else => return err, + }; + defer file.close(); + + const content = try file.readToEndAlloc(allocator, 1024 * 1024); + defer allocator.free(content); + + const parsed = try std.json.parseFromSlice(Config, allocator, content, .{ + .ignore_unknown_fields = true, + }); + defer parsed.deinit(); + + // Copy strings to owned memory since parsed will be freed + return Config{ + .host = try allocator.dupe(u8, parsed.value.host), + .port = parsed.value.port, + .debug = parsed.value.debug, + }; +} +``` + +### API Response Handling + +```zig +const ApiResponse = struct { + success: bool, + data: ?Data = null, + @"error": ?[]const u8 = null, // use @"error" for reserved words + + const Data = struct { + id: u64, + name: []const u8, + }; +}; + +fn handleResponse(json: []const u8, allocator: std.mem.Allocator) !void { + const parsed = try std.json.parseFromSlice(ApiResponse, allocator, json, .{ + .ignore_unknown_fields = true, + }); + defer parsed.deinit(); + + if (!parsed.value.success) { + std.debug.print("Error: {s}\n", .{parsed.value.@"error" orelse "unknown"}); + return error.ApiError; + } + + if (parsed.value.data) |data| { + std.debug.print("Got: {s} (id={})\n", .{ data.name, data.id }); + } +} +``` + +### Pretty Print JSON + +```zig +fn prettyPrint(allocator: std.mem.Allocator, json: []const u8) ![]u8 { + const parsed = try std.json.parseFromSlice(std.json.Value, allocator, json, .{}); + defer parsed.deinit(); + + return std.json.Stringify.valueAlloc(allocator, parsed.value, .{ + .whitespace = .indent_2, + }); +} +``` + +### Serialize with Filtering + +```zig +fn serializePublicFields(allocator: std.mem.Allocator, user: User) ![]u8 { + // Create anonymous struct with only public fields + const public = .{ + .id = user.id, + .name = user.name, + // Exclude: .password, .internal_state + }; + return std.json.Stringify.valueAlloc(allocator, public, .{}); +} +``` diff --git a/references/std-linked-list.md b/references/std-linked-list.md new file mode 100644 index 0000000..f24fc3d --- /dev/null +++ b/references/std-linked-list.md @@ -0,0 +1,155 @@ +# std.DoublyLinkedList / std.SinglyLinkedList + +Intrusive linked lists for O(1) insertion/removal. Nodes are embedded in user structs via `@fieldParentPtr`. + +## When to Use + +- O(1) insertion/removal anywhere in list +- Elements that need to be in multiple lists +- Preallocated/arena-allocated nodes +- No allocation on insert (nodes already exist) + +## DoublyLinkedList + +Bidirectional traversal, O(1) removal of any node. + +```zig +const std = @import("std"); + +const Item = struct { + data: u32, + node: std.DoublyLinkedList.Node = .{}, // embed node +}; + +var list: std.DoublyLinkedList = .{}; + +// Create items (you manage memory) +var a: Item = .{ .data = 1 }; +var b: Item = .{ .data = 2 }; +var c: Item = .{ .data = 3 }; + +// Insert +list.append(&a.node); // add to end +list.prepend(&b.node); // add to start +list.insertAfter(&a.node, &c.node); // insert c after a +list.insertBefore(&a.node, &c.node); // insert c before a + +// Remove +list.remove(&a.node); // O(1) remove specific node +const last = list.pop(); // remove and return last +const first = list.popFirst(); // remove and return first + +// Get data from node +if (list.first) |node| { + const item: *Item = @fieldParentPtr("node", node); + std.debug.print("data: {}\n", .{item.data}); +} + +// Traverse forward +var it = list.first; +while (it) |node| : (it = node.next) { + const item: *Item = @fieldParentPtr("node", node); + // use item.data +} + +// Traverse backward +var it = list.last; +while (it) |node| : (it = node.prev) { + const item: *Item = @fieldParentPtr("node", node); + // use item.data +} + +// Concatenate (moves all from list2 to end of list1) +list1.concatByMoving(&list2); + +// Length (O(n) - consider tracking separately) +const n = list.len(); +``` + +## SinglyLinkedList + +Forward-only, minimal memory (one pointer per node). + +```zig +const Item = struct { + data: u32, + node: std.SinglyLinkedList.Node = .{}, +}; + +var list: std.SinglyLinkedList = .{}; + +var a: Item = .{ .data = 1 }; +var b: Item = .{ .data = 2 }; + +// Insert (only at front or after existing node) +list.prepend(&a.node); // add to front +a.node.insertAfter(&b.node); // insert b after a + +// Remove +const first = list.popFirst(); // remove and return first +_ = a.node.removeNext(); // remove node after a +list.remove(&b.node); // O(n) - must find predecessor + +// Traverse (forward only) +var it = list.first; +while (it) |node| : (it = node.next) { + const item: *Item = @fieldParentPtr("node", node); + // use item.data +} + +// Find last (O(n)) +if (list.first) |first| { + const last = first.findLast(); +} + +// Reverse in place +std.SinglyLinkedList.Node.reverse(&list.first); + +// Length (O(n)) +const n = list.len(); +``` + +## Node Methods + +```zig +// DoublyLinkedList.Node +node.prev // ?*Node +node.next // ?*Node + +// SinglyLinkedList.Node +node.next // ?*Node +node.insertAfter(new_node) +node.removeNext() // ?*Node - removes and returns next +node.findLast() // *Node +node.countChildren() // usize +node.reverse(&optional_ptr) +``` + +## Common Pattern: LRU Cache + +```zig +const Entry = struct { + key: []const u8, + value: Value, + node: std.DoublyLinkedList.Node = .{}, +}; + +var lru_list: std.DoublyLinkedList = .{}; +var entries: std.StringHashMap(*Entry) = .init(allocator); + +fn access(key: []const u8) ?*Entry { + const entry = entries.get(key) orelse return null; + // Move to front (most recently used) + lru_list.remove(&entry.node); + lru_list.prepend(&entry.node); + return entry; +} + +fn evictOldest() void { + if (lru_list.pop()) |node| { + const entry: *Entry = @fieldParentPtr("node", node); + _ = entries.remove(entry.key); + // free entry + } +} +``` diff --git a/references/std-log.md b/references/std-log.md new file mode 100644 index 0000000..8ce82e8 --- /dev/null +++ b/references/std-log.md @@ -0,0 +1,259 @@ +# std.log (Zig 0.16.0) + +Standardized logging interface with configurable scopes, levels, and output. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +When writing custom log functions in Zig 0.16, use `@EnumLiteral()` instead of removed `@Type(.enum_literal)`, use `std.Io` writers for direct file/stdout/stderr output, and route timestamps through a shared `std.Io`-aware helper. + +## Quick Reference + +| Function | Purpose | +|----------|---------| +| `log.err(fmt, args)` | Log error (something went wrong) | +| `log.warn(fmt, args)` | Log warning (uncertain if wrong) | +| `log.info(fmt, args)` | Log info (general state) | +| `log.debug(fmt, args)` | Log debug (debugging only) | +| `log.scoped(.name)` | Create scoped logger | + +## Basic Usage + +```zig +const std = @import("std"); +const log = std.log; + +pub fn main() void { + log.info("Starting application", .{}); + log.debug("Debug value: {}", .{x}); // Hidden in release builds + log.warn("Config missing, using defaults", .{}); + log.err("Failed to connect: {s}", .{@errorName(e)}); +} +``` + +## Log Levels + +| Level | Build Mode Default | Purpose | +|-------|-------------------|---------| +| `.err` | Always shown | Something went wrong | +| `.warn` | Always shown | Uncertain if wrong, worth investigating | +| `.info` | Debug + Release | General program state | +| `.debug` | Debug only | Messages only useful for debugging | + +Default level by build mode: +- **Debug**: `.debug` (all messages) +- **ReleaseSafe/Fast/Small**: `.info` (no debug messages) + +## Scoped Logging + +Create loggers with custom scopes for filtering: + +```zig +const std = @import("std"); + +// Library logger with custom scope +const log = std.log.scoped(.my_library); + +pub fn doWork() void { + log.info("Processing...", .{}); // Prefixed with (my_library) + log.debug("Details: {}", .{x}); +} +``` + +Multiple scopes in one file: + +```zig +const network_log = std.log.scoped(.network); +const db_log = std.log.scoped(.database); + +fn fetchData() void { + network_log.info("Connecting...", .{}); + db_log.debug("Query: {s}", .{sql}); +} +``` + +## Configuration via std_options + +Configure logging in your root file: + +```zig +const std = @import("std"); + +pub const std_options: std.Options = .{ + // Global log level + .log_level = .warn, // Only show warn and err + + // Per-scope levels (override global) + .log_scope_levels = &.{ + .{ .scope = .my_library, .level = .debug }, // Full debug for this scope + .{ .scope = .noisy_lib, .level = .err }, // Errors only + }, + + // Custom log function + .logFn = myLogFn, +}; +``` + +## Custom Log Function + +Replace the default log output: + +```zig +const std = @import("std"); + +pub const std_options: std.Options = .{ + .logFn = myLogFn, +}; + +fn myLogFn( + comptime level: std.log.Level, + comptime scope: @EnumLiteral(), + comptime format: []const u8, + args: anytype, +) void { + // Filter: only errors from unknown scopes + const scope_prefix = switch (scope) { + .my_app, .default => @tagName(scope), + else => if (@intFromEnum(level) <= @intFromEnum(std.log.Level.err)) + @tagName(scope) + else + return, // Skip non-error from other scopes + }; + + const level_txt = comptime level.asText(); + const prefix = "[" ++ level_txt ++ "] (" ++ scope_prefix ++ "): "; + + std.debug.lockStdErr(); + defer std.debug.unlockStdErr(); + + const io = applicationIo(); // Application-owned accessor for std.Io. + var buf: [64]u8 = undefined; + var stderr = std.Io.File.stderr().writer(io, &buf); + stderr.interface.print(prefix ++ format ++ "\n", args) catch return; + stderr.interface.flush() catch return; +} +``` + +## Check if Logging Enabled + +Avoid expensive computations when logging is disabled: + +```zig +const log = std.log.scoped(.my_scope); + +fn process() void { + // Check before expensive operation + if (std.log.logEnabled(.debug, .my_scope)) { + const debug_info = computeExpensiveDebugInfo(); + log.debug("Info: {}", .{debug_info}); + } + + // For default scope + if (std.log.defaultLogEnabled(.debug)) { + std.log.debug("Debug message", .{}); + } +} +``` + +## Level Methods + +```zig +const level: std.log.Level = .warn; + +// Get text representation +const text = level.asText(); // "warning" + +// Compare levels (lower = more severe) +const is_error_or_worse = @intFromEnum(level) <= @intFromEnum(std.log.Level.err); +``` + +## Default Log Function + +Forward to the standard implementation: + +```zig +fn myLogFn( + comptime level: std.log.Level, + comptime scope: @EnumLiteral(), + comptime format: []const u8, + args: anytype, +) void { + // Add timestamp, then forward to default + std.debug.print("[{d}] ", .{applicationTimestampNow().toNanoseconds()}); + std.log.defaultLog(level, scope, format, args); +} +``` + +## Output Format + +Default output format: +``` +level: message # default scope +level(scope): message # named scope +``` + +Examples: +``` +info: Server started on port 8080 +warning(database): Connection pool exhausted +error(network): Failed to resolve hostname +debug: Variable x = 42 +``` + +## Common Patterns + +### Conditional Debug Logging + +```zig +fn processItem(item: Item) void { + if (comptime std.log.logEnabled(.debug, .default)) { + log.debug("Processing: {}", .{item}); + } + // ... process +} +``` + +### Error Context Logging + +```zig +fn loadConfig(io: std.Io, path: []const u8) !Config { + return std.Io.Dir.cwd().openFile(io, path, .{}) catch |err| { + log.err("Failed to open config '{s}': {s}", .{path, @errorName(err)}); + return err; + }; +} +``` + +### Library Logging Pattern + +```zig +// In library code +pub const log = std.log.scoped(.my_lib); + +// Users can filter with: +// .log_scope_levels = &.{ .{ .scope = .my_lib, .level = .warn } } +``` + +## Log to File + +```zig +fn fileLogFn( + comptime level: std.log.Level, + comptime scope: @EnumLiteral(), + comptime format: []const u8, + args: anytype, +) void { + const io = applicationIo(); + const file = std.Io.Dir.cwd().openFile(io, "app.log", .{ .mode = .write_only }) catch return; + defer file.close(io); + + var buf: [256]u8 = undefined; + var writer = file.writer(io, &buf); + const w = &writer.interface; + + const level_txt = comptime level.asText(); + const scope_txt = if (scope == .default) "" else "(" ++ @tagName(scope) ++ ")"; + + w.print("[{s}]{s} " ++ format ++ "\n", .{level_txt, scope_txt} ++ args) catch return; + w.flush() catch return; +} +``` diff --git a/references/std-math.md b/references/std-math.md new file mode 100644 index 0000000..ae2a49c --- /dev/null +++ b/references/std-math.md @@ -0,0 +1,411 @@ +# std.math + +Mathematical functions, constants, and utilities. Provides floating-point operations, trigonometry, integer arithmetic with overflow checking, and arbitrary-precision integers. + +## Quick Reference + +| Category | Functions | +|----------|-----------| +| Constants | `e`, `pi`, `phi`, `tau`, `sqrt2`, `ln2`, `ln10` | +| Trig | `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `atan2` | +| Hyperbolic | `sinh`, `cosh`, `tanh`, `asinh`, `acosh`, `atanh` | +| Exponential | `exp`, `exp2`, `expm1`, `log`, `log2`, `log10`, `log1p` | +| Powers/Roots | `pow`, `powi`, `sqrt`, `cbrt`, `hypot` | +| Rounding | `floor`, `ceil`, `round`, `trunc` | +| Float Tests | `isNan`, `isInf`, `isFinite`, `isNormal`, `signbit` | +| Integer Ops | `add`, `sub`, `mul`, `divTrunc`, `divFloor`, `divCeil` | +| Bit Ops | `shl`, `shr`, `rotl`, `rotr`, `log2_int`, `isPowerOfTwo` | +| Comparison | `order`, `compare`, `clamp`, `sign` | + +## Mathematical Constants + +```zig +const std = @import("std"); +const math = std.math; + +// Fundamental constants +const euler = math.e; // 2.71828... +const pi_val = math.pi; // 3.14159... +const golden = math.phi; // 1.61803... (golden ratio) +const tau_val = math.tau; // 2 * pi + +// Logarithmic constants +const log2_e = math.log2e; // log2(e) +const log10_e = math.log10e; // log10(e) +const ln_2 = math.ln2; // ln(2) +const ln_10 = math.ln10; // ln(10) + +// Square root constants +const sqrt_2 = math.sqrt2; // sqrt(2) +const inv_sqrt2 = math.sqrt1_2; // 1/sqrt(2) + +// Angle conversion +const rad_deg = math.rad_per_deg; // pi/180 +const deg_rad = math.deg_per_rad; // 180/pi +``` + +## Angle Conversion + +```zig +// Convert between radians and degrees +const radians = std.math.degreesToRadians(@as(f32, 90.0)); // pi/2 +const degrees = std.math.radiansToDegrees(@as(f32, std.math.pi)); // 180.0 + +// Works with vectors +const angles: @Vector(3, f32) = .{ 90.0, 180.0, 270.0 }; +const rads = std.math.degreesToRadians(angles); +``` + +## Trigonometric Functions + +```zig +const x: f32 = std.math.pi / 4.0; + +// Basic trig (use hardware instructions when available) +const sine = std.math.sin(x); // 0.7071... +const cosine = std.math.cos(x); // 0.7071... +const tangent = std.math.tan(x); // 1.0 + +// Inverse trig +const asin_val = std.math.asin(@as(f32, 0.5)); // pi/6 +const acos_val = std.math.acos(@as(f32, 0.5)); // pi/3 +const atan_val = std.math.atan(@as(f32, 1.0)); // pi/4 +const atan2_val = std.math.atan2(@as(f32, 1.0), @as(f32, 1.0)); // pi/4 + +// Hyperbolic functions +const sinh_val = std.math.sinh(x); +const cosh_val = std.math.cosh(x); +const tanh_val = std.math.tanh(x); +const asinh_val = std.math.asinh(x); +const acosh_val = std.math.acosh(@as(f32, 2.0)); +const atanh_val = std.math.atanh(@as(f32, 0.5)); +``` + +## Exponential and Logarithmic Functions + +```zig +const x: f64 = 2.0; + +// Exponential +const exp_val = std.math.exp(x); // e^x +const exp2_val = std.math.exp2(x); // 2^x +const expm1_val = std.math.expm1(x); // e^x - 1 (more precise near 0) + +// Logarithms +const log_val = std.math.log(f64, std.math.e, x); // natural log +const log2_val = std.math.log2(x); // log base 2 +const log10_val = std.math.log10(x); // log base 10 +const log1p_val = std.math.log1p(x); // ln(1 + x) (more precise near 0) + +// Integer logarithms (for integer types) +const log2_int_val = std.math.log2_int(u32, 8); // 3 (floor) +const log2_ceil = std.math.log2_int_ceil(u32, 9); // 4 (ceil) +const log10_int_val = std.math.log10_int(1000); // 3 +``` + +## Power and Root Functions + +```zig +// Powers +const pow_val = std.math.pow(f64, 2.0, 3.0); // 2^3 = 8.0 +const powi_val = std.math.powi(f64, 2.0, 3); // 2^3 (integer exponent) + +// Roots +const sqrt_val = std.math.sqrt(@as(f64, 16.0)); // 4.0 +const cbrt_val = std.math.cbrt(@as(f64, 27.0)); // 3.0 + +// Hypotenuse (sqrt(x^2 + y^2), avoids overflow) +const hyp = std.math.hypot(@as(f64, 3.0), @as(f64, 4.0)); // 5.0 +``` + +## Rounding Functions + +```zig +const x: f32 = 2.7; + +const floor_val = std.math.floor(x); // 2.0 (toward -inf) +const ceil_val = std.math.ceil(x); // 3.0 (toward +inf) +const trunc_val = std.math.trunc(x); // 2.0 (toward zero) +const round_val = std.math.round(x); // 3.0 (nearest, ties away from zero) +``` + +## Floating-Point Classification + +```zig +const x: f32 = 1.0; +const inf_val = std.math.inf(f32); +const nan_val = std.math.nan(f32); + +// Classification tests +const is_nan = std.math.isNan(nan_val); // true +const is_inf = std.math.isInf(inf_val); // true +const is_pos_inf = std.math.isPositiveInf(inf_val); // true +const is_neg_inf = std.math.isNegativeInf(-inf_val); // true +const is_finite = std.math.isFinite(x); // true +const is_normal = std.math.isNormal(x); // true + +// Sign operations +const has_neg_sign = std.math.signbit(-1.0); // true +const copied = std.math.copysign(@as(f32, 5.0), @as(f32, -1.0)); // -5.0 +``` + +## Float Properties + +```zig +// Get float type properties +const mantissa_bits = std.math.floatMantissaBits(f32); // 23 +const exponent_bits = std.math.floatExponentBits(f32); // 8 +const eps = std.math.floatEps(f32); // ~1.19e-7 +const min_val = std.math.floatMin(f32); // smallest positive normal +const max_val = std.math.floatMax(f32); // largest finite +const true_min = std.math.floatTrueMin(f32); // smallest positive (including subnormal) + +// Special values +const inf_val = std.math.inf(f32); // positive infinity +const nan_val = std.math.nan(f32); // quiet NaN +const snan_val = std.math.snan(f32); // signaling NaN +``` + +## Approximate Equality + +```zig +const x: f32 = 1.0; +const y: f32 = 1.0 + std.math.floatEps(f32); + +// Absolute tolerance (good for values near zero) +const abs_eq = std.math.approxEqAbs(f32, x, y, 1e-6); + +// Relative tolerance (good for larger values) +const rel_eq = std.math.approxEqRel(f32, x, y, std.math.sqrt(std.math.floatEps(f32))); +``` + +## Integer Arithmetic with Overflow Checking + +```zig +// These return errors on overflow instead of wrapping +const sum = std.math.add(i32, 2147483647, 1) catch |err| { + // err is error.Overflow + return err; +}; + +const product = std.math.mul(i32, 1000000, 1000000) catch |err| { + return err; // Overflow for i32 +}; + +const diff = std.math.sub(u32, 5, 10) catch |err| { + return err; // Overflow (underflow) for unsigned +}; + +// Negation with potential overflow +const negated = std.math.negate(@as(i8, -128)) catch |err| { + return err; // Can't represent 128 in i8 +}; + +// Shift with overflow check +const shifted = std.math.shlExact(u8, 1, 8) catch |err| { + return err; // Overflow: 1 << 8 doesn't fit in u8 +}; +``` + +## Division Functions + +```zig +// Division toward zero +const trunc_div = try std.math.divTrunc(i32, -7, 3); // -2 + +// Division toward negative infinity +const floor_div = try std.math.divFloor(i32, -7, 3); // -3 + +// Division toward positive infinity +const ceil_div = try std.math.divCeil(i32, 7, 3); // 3 + +// Exact division (error if remainder) +const exact = try std.math.divExact(i32, 10, 5); // 2 +// std.math.divExact(i32, 10, 3) returns error.UnexpectedRemainder + +// Modulo (always non-negative result) +const mod_val = try std.math.mod(i32, -5, 3); // 1 + +// Remainder (can be negative) +const rem_val = try std.math.rem(i32, -5, 3); // -2 +``` + +## Bit Operations + +```zig +// Shift with truncation (no overflow, large shifts -> 0) +const shl_val = std.math.shl(u8, 0b11111111, 3); // 0b11111000 +const shr_val = std.math.shr(u8, 0b11111111, 3); // 0b00011111 + +// Negative shift amounts reverse direction +const neg_shift = std.math.shl(u8, 0b11111111, -2); // 0b00111111 + +// Rotation (unsigned integers only) +const rotl_val = std.math.rotl(u8, 0b00000001, 4); // 0b00010000 +const rotr_val = std.math.rotr(u8, 0b00010000, 4); // 0b00000001 + +// Power of two checks +const is_pow2 = std.math.isPowerOfTwo(@as(u32, 8)); // true +const floor_pow2 = std.math.floorPowerOfTwo(u32, 65); // 64 +const ceil_pow2 = try std.math.ceilPowerOfTwo(u32, 65); // 128 +``` + +## Integer Type Utilities + +```zig +// Get min/max values of integer type +const max_i32 = std.math.maxInt(i32); // 2147483647 +const min_i32 = std.math.minInt(i32); // -2147483648 + +// Log2Int: type for bit indices +const Log2U32 = std.math.Log2Int(u32); // u5 (can hold 0-31) + +// Smallest type fitting a range +const T = std.math.IntFittingRange(0, 100); // u7 +const S = std.math.IntFittingRange(-50, 50); // i7 + +// Byte-aligned integer type +const ByteAligned = std.math.ByteAlignedInt(u5); // u8 +``` + +## Comparison and Ordering + +```zig +// Get ordering between values +const ord = std.math.order(@as(i32, 5), @as(i32, 3)); // .gt +// ord is std.math.Order: .lt, .eq, or .gt + +// Runtime comparison operator +const result = std.math.compare(@as(i32, 5), .gte, @as(i32, 3)); // true + +// Clamp to range +const clamped = std.math.clamp(@as(i32, 15), @as(i32, 0), @as(i32, 10)); // 10 + +// Wrap to half-open interval [-r, r) +const wrapped = std.math.wrap(@as(i32, 270), @as(i32, 180)); // -90 +``` + +## Sign and Interpolation + +```zig +// Get sign (-1, 0, or 1) +const s = std.math.sign(@as(i32, -42)); // -1 + +// Linear interpolation +const lerped = std.math.lerp(@as(f32, 0.0), @as(f32, 100.0), @as(f32, 0.25)); // 25.0 +``` + +## Type Casting + +```zig +// Safe cast (returns null if doesn't fit) +const maybe: ?u8 = std.math.cast(u8, @as(i32, 300)); // null + +// Lossy cast (clamps to representable range) +const clamped = std.math.lossyCast(u8, @as(i32, 300)); // 255 +const from_float = std.math.lossyCast(i16, @as(f32, 70000.0)); // 32767 + +// Negate and cast to signed +const negated = try std.math.negateCast(@as(u32, 100)); // -100 as i32 +``` + +## Wide Multiplication + +```zig +// Multiply without overflow (result is double width) +const wide = std.math.mulWide(u8, 200, 200); // 40000 as u16 +``` + +## Complex Numbers + +```zig +const Complex = std.math.Complex; + +const z1 = Complex(f32).init(3.0, 4.0); // 3 + 4i +const z2 = Complex(f32).init(1.0, 2.0); // 1 + 2i + +// Arithmetic +const sum = z1.add(z2); // 4 + 6i +const diff = z1.sub(z2); // 2 + 2i +const prod = z1.mul(z2); // -5 + 10i +const quot = z1.div(z2); + +// Operations +const conj = z1.conjugate(); // 3 - 4i +const neg = z1.neg(); // -3 - 4i +const recip = z1.reciprocal(); +const mag = z1.magnitude(); // 5.0 (|z|) + +// Multiply by i +const times_i = z1.mulbyi(); // -4 + 3i + +// Complex math functions +const z_exp = std.math.complex.exp(z1); +const z_log = std.math.complex.log(z1); +const z_sin = std.math.complex.sin(z1); +const z_sqrt = std.math.complex.sqrt(z1); +``` + +## Big Integers (Arbitrary Precision) + +```zig +const big = std.math.big; +const Managed = big.int.Managed; + +// Create big integers (requires allocator) +var a = try Managed.initSet(allocator, 12345678901234567890); +defer a.deinit(); + +var b = try Managed.initSet(allocator, 98765432109876543210); +defer b.deinit(); + +// Arithmetic +try a.add(&a, &b); +try a.mul(&a, &b); +try a.div(&q, &r, &a, &b); // quotient and remainder + +// Comparison +const ord = a.order(b); // .lt, .eq, or .gt + +// Convert to primitive (if fits) +const val = a.to(i128) catch |err| { + // Value doesn't fit in i128 + return err; +}; + +// Convert from string +var c = try Managed.init(allocator); +defer c.deinit(); +try c.setString(10, "123456789012345678901234567890"); +``` + +## GCD and LCM + +```zig +// Greatest common divisor +const gcd_val = std.math.gcd(@as(u32, 48), @as(u32, 18)); // 6 + +// Least common multiple +const lcm_val = try std.math.lcm(@as(u32, 4), @as(u32, 6)); // 12 +// Returns error.Overflow if result doesn't fit +``` + +## Gamma Functions + +```zig +// Gamma function +const g = std.math.gamma(f64, 5.0); // 24.0 (= 4!) + +// Log gamma (more numerically stable for large values) +const lg = std.math.lgamma(f64, 100.0); +``` + +## Notes + +- Most functions work with `f16`, `f32`, `f64`, `f80`, `f128` and `comptime_float` +- Many functions support SIMD vectors: `sin(@Vector(4, f32){...})` +- Integer overflow-checking functions return `error.Overflow` or `error.DivisionByZero` +- Hardware instructions used when available (`@sin`, `@cos`, `@sqrt`, etc.) +- `approxEqAbs` for values near zero, `approxEqRel` for larger values +- Complex number operations available in `std.math.complex` +- Big integers require allocator and are in `std.math.big.int` diff --git a/references/std-mem.md b/references/std-mem.md new file mode 100644 index 0000000..a99e824 --- /dev/null +++ b/references/std-mem.md @@ -0,0 +1,315 @@ +# std.mem (Zig 0.16.0) + +Memory manipulation utilities: slice operations, searching, splitting, alignment, endianness, and byte conversion. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +## Zig 0.16 Search/Cut Names + +The release notes rename "index of" style APIs toward `find` and add `cut` helpers. Use these names in new code: + +```zig +std.mem.find(u8, "hello world", "wor") +std.mem.findLast(u8, "ababa", "ab") +std.mem.findScalar(u8, "hello", 'l') +std.mem.findScalarLast(u8, "hello", 'l') +std.mem.findAny(u8, "hello", "aeiou") +std.mem.findNone(u8, " hello", " ") + +std.mem.cut(u8, "key=value", "=") +std.mem.cutScalar(u8, "key=value", '=') +std.mem.cutPrefix(u8, path, "assets/") +std.mem.cutSuffix(u8, file_name, ".zig") +``` + +Older examples below may use 0.15 `indexOf` names; translate them to the `find`/`cut` APIs in Zig 0.16. + +## Slice Comparison & Search + +```zig +// Equality +std.mem.eql(u8, "hello", "hello") // true +std.mem.order(u8, "abc", "abd") // .lt + +// Find substring/element +std.mem.find(u8, "hello world", "wor") // ?usize = 6 +std.mem.findLast(u8, "ababa", "ab") // ?usize = 2 +std.mem.findScalar(u8, "hello", 'l') // ?usize = 2 +std.mem.findScalarLast(u8, "hello", 'l') // ?usize = 3 + +// Find any/none of characters +std.mem.findAny(u8, "hello", "aeiou") // ?usize = 1 (first vowel) +std.mem.findNone(u8, " hello", " ") // ?usize = 3 (first non-space) + +// Check prefix/suffix +std.mem.startsWith(u8, "hello", "hel") // true +std.mem.endsWith(u8, "hello.txt", ".txt") // true + +// Count occurrences +std.mem.count(u8, "ababa", "ab") // 2 +std.mem.containsAtLeast(u8, "ababa", 2, "ab") // true +``` + +## Tokenize vs Split + +**Tokenize**: Skip empty tokens (like shell word splitting) +```zig +var it = std.mem.tokenizeAny(u8, " hello world ", " "); +while (it.next()) |token| { + // "hello", "world" +} + +// Other tokenize variants +std.mem.tokenizeScalar(u8, "a,b,c", ','); // single delimiter +std.mem.tokenizeSequence(u8, "a::b::c", "::"); // exact sequence +``` + +**Split**: Preserve empty tokens +```zig +var it = std.mem.splitScalar(u8, "a,,b", ','); +while (it.next()) |part| { + // "a", "", "b" +} + +// Other split variants +std.mem.splitAny(u8, "a,b;c", ",;"); // any of delimiters +std.mem.splitSequence(u8, "a::b::c", "::"); // exact sequence + +// Split backwards +var it = std.mem.splitBackwardsScalar(u8, "a/b/c", '/'); +// "c", "b", "a" +``` + +## Window Iterator + +Sliding window over slice: +```zig +var it = std.mem.window(u8, "hello", 3, 1); // size=3, advance=1 +while (it.next()) |w| { + // "hel", "ell", "llo" +} +``` + +## Join & Concat + +```zig +const allocator = std.heap.page_allocator; + +// Join with separator +const joined = try std.mem.join(allocator, ", ", &.{ "a", "b", "c" }); +defer allocator.free(joined); // "a, b, c" + +// Join with null terminator +const joinedZ = try std.mem.joinZ(allocator, "/", &.{ "path", "to", "file" }); +// [:0]u8 = "path/to/file" + +// Concatenate without separator +const concatted = try std.mem.concat(allocator, u8, &.{ "hello", " ", "world" }); +// "hello world" +``` + +## Trim + +```zig +std.mem.trim(u8, " hello ", " ") // "hello" +std.mem.trimStart(u8, " hello", " ") // "hello" (left only) +std.mem.trimEnd(u8, "hello ", " ") // "hello" (right only) + +// Trim multiple characters +std.mem.trim(u8, "\n\thello\n\t", " \t\n") +``` + +## Replace + +```zig +// In-place replace (returns count) +var buf: [100]u8 = undefined; +const count = std.mem.replace(u8, "hello", "l", "L", &buf); +// buf contains "heLLo", count = 2 + +// Allocate new slice +const result = try std.mem.replaceOwned(u8, allocator, "hello", "l", "L"); +defer allocator.free(result); // "heLLo" + +// Replace single scalar +var data = [_]u8{ 'a', 'b', 'a' }; +std.mem.replaceScalar(u8, &data, 'a', 'x'); // "xbx" + +// Calculate replacement size first +const size = std.mem.replacementSize(u8, "hello", "l", "LL"); // 7 +``` + +## Byte Conversion + +```zig +// Value to bytes +const val: u32 = 0xDEADBEEF; +const bytes = std.mem.asBytes(&val); // *const [4]u8 +const byte_copy = std.mem.toBytes(val); // [4]u8 (copy) + +// Bytes to value +const bytes = [_]u8{ 0xEF, 0xBE, 0xAD, 0xDE }; +const ptr = std.mem.bytesAsValue(u32, &bytes); // *const u32 +const val = std.mem.bytesToValue(u32, &bytes); // u32 (copy) + +// Slice conversions +const u16_slice = [_]u16{ 0x0102, 0x0304 }; +const u8_slice = std.mem.sliceAsBytes(&u16_slice); // []const u8 + +const u8_data = [_]u8{ 1, 0, 2, 0, 3, 0, 4, 0 }; +const u16_view = std.mem.bytesAsSlice(u16, &u8_data); // []const u16 +``` + +## Alignment + +```zig +// Align forward (round up) +std.mem.alignForward(usize, 7, 4) // 8 +std.mem.alignForward(usize, 8, 4) // 8 +std.mem.alignForward(usize, 9, 4) // 12 + +// Align backward (round down) +std.mem.alignBackward(usize, 7, 4) // 4 +std.mem.alignBackward(usize, 8, 4) // 8 + +// Check alignment +std.mem.isAligned(8, 4) // true +std.mem.isAligned(7, 4) // false +std.mem.isValidAlign(4) // true (power of 2) +std.mem.isValidAlign(3) // false + +// Align pointer +const ptr: [*]u8 = @ptrFromInt(0x123); +const aligned = std.mem.alignPointer(ptr, 0x100); // ?[*]u8 = 0x200 + +// Find aligned slice within bytes +const aligned_slice = std.mem.alignInBytes(bytes, 16); // ?[]align(16) u8 +``` + +## Alignment Type + +```zig +const align_val: std.mem.Alignment = .@"16"; // 16-byte alignment +const bytes = align_val.toByteUnits(); // 16 + +// From byte units +const a = std.mem.Alignment.fromByteUnits(8); // .@"8" + +// From type +const a = std.mem.Alignment.of(u64); // .@"8" + +// Forward/backward with Alignment +const addr = align_val.forward(0x123); // next aligned address +const addr = align_val.backward(0x123); // previous aligned address +const ok = align_val.check(0x100); // true if aligned +``` + +## Endianness Conversion + +```zig +// To/from native endianness +const native = std.mem.littleToNative(u32, 0x12345678); +const native = std.mem.bigToNative(u32, 0x12345678); +const little = std.mem.nativeToLittle(u32, native_val); +const big = std.mem.nativeToBig(u32, native_val); + +// General conversion +const val = std.mem.toNative(u32, x, .little); // from little to native +const val = std.mem.nativeTo(u32, x, .big); // from native to big + +// Byte swap all fields in struct +std.mem.byteSwapAllFields(MyStruct, &my_struct); + +// Byte swap all elements in slice +std.mem.byteSwapAllElements(u32, slice); +``` + +## Packed Integer Read/Write + +Read/write integers at bit offsets: +```zig +var bytes = [_]u8{ 0, 0, 0, 0 }; + +// Write u12 at bit offset 4 +std.mem.writePackedInt(u12, &bytes, 4, 0xABC, .little); + +// Read it back +const val = std.mem.readPackedInt(u12, &bytes, 4, .little); + +// Variable-width read/write +std.mem.writeVarPackedInt(&bytes, bit_offset, bit_count, value, .little); +const val = std.mem.readVarPackedInt(u32, &bytes, bit_offset, bit_count, .little, .unsigned); +``` + +## Zero Initialization + +```zig +// Zero-initialize a type +const zeroed: MyStruct = std.mem.zeroes(MyStruct); +// All numeric fields = 0, optionals = null, slices = empty + +// Partial initialization with zeros for rest +const partial = std.mem.zeroInit(MyStruct, .{ + .name = "foo", // explicit value + // other fields zeroed +}); +``` + +## Min/Max + +```zig +const slice = [_]i32{ 3, 1, 4, 1, 5 }; + +std.mem.min(i32, &slice) // 1 +std.mem.max(i32, &slice) // 5 +std.mem.minMax(i32, &slice) // .{ 1, 5 } + +std.mem.indexOfMin(i32, &slice) // 1 +std.mem.indexOfMax(i32, &slice) // 4 +std.mem.indexOfMinMax(i32, &slice) // .{ 1, 4 } +``` + +## Reverse & Rotate + +```zig +var arr = [_]u8{ 1, 2, 3, 4, 5 }; + +std.mem.reverse(u8, &arr); // [5, 4, 3, 2, 1] +std.mem.rotate(u8, &arr, 2); // rotate left by 2 + +// Swap two values +std.mem.swap(u32, &a, &b); + +// Reverse iterator (no mutation) +var it = std.mem.reverseIterator(&arr); +while (it.next()) |val| { + // 5, 4, 3, 2, 1 +} +``` + +## Other Utilities + +```zig +// All elements equal to value +std.mem.allEqual(u8, slice, 0) // true if all zeros + +// Sentinel-terminated length +const len = std.mem.len(c_string); // length of null-terminated string + +// Span from sentinel pointer (convert [*:0]T to []T) +const slice = std.mem.span(c_string); + +// Index of first difference +std.mem.indexOfDiff(u8, "hello", "helps") // ?usize = 3 + +// Collapse repeated elements +var data = "aabbcc".*; +const len = std.mem.collapseRepeatsLen(u8, &data, 'a'); // "abbcc", 5 +``` + +## Benchmark Utility + +```zig +// Prevent compiler from optimizing away a value +std.mem.doNotOptimizeAway(result); +``` diff --git a/references/std-meta.md b/references/std-meta.md new file mode 100644 index 0000000..227e22b --- /dev/null +++ b/references/std-meta.md @@ -0,0 +1,448 @@ +# std.meta + +Comptime type introspection and manipulation utilities. Essential for generic programming, serialization, and metaprogramming. + +## Quick Reference + +| Function | Purpose | +|----------|---------| +| `stringToEnum(T, str)` | Convert string to enum variant | +| `fields(T)` | Get struct/union/enum/error fields | +| `fieldNames(T)` | Get field names as string slice | +| `fieldInfo(T, field)` | Get info for specific field | +| `fieldIndex(T, name)` | Get field index by name | +| `tags(T)` | Get all enum/error set values | +| `Tag(T)` | Get tag type of enum/union | +| `activeTag(u)` | Get active variant of tagged union | +| `eql(a, b)` | Deep equality comparison | +| `Child(T)` | Get child type of pointer/array/optional | +| `Elem(T)` | Get element type of memory span | +| `sentinel(T)` | Get sentinel value if any | +| `FieldEnum(T)` | Generate enum from field names | +| `hasFn(T, name)` | Check if type has function | +| `hasMethod(T, name)` | Check if type has method | + +## String to Enum Conversion + +```zig +const std = @import("std"); + +const Color = enum { red, green, blue }; + +// Convert runtime string to enum +const color = std.meta.stringToEnum(Color, "green"); +if (color) |c| { + // c == Color.green +} + +// Returns null for invalid strings +const invalid = std.meta.stringToEnum(Color, "purple"); // null +``` + +Uses `StaticStringMap` for small enums (≤100 variants), inline iteration for larger ones. + +## Field Introspection + +### Get All Fields + +```zig +const Point = struct { + x: f32, + y: f32, + z: f32 = 0, +}; + +// Get struct fields +const point_fields = std.meta.fields(Point); +// point_fields.len == 3 +// point_fields[0].name == "x" +// point_fields[0].type == f32 + +// Works for unions +const Result = union { ok: u32, err: []const u8 }; +const union_fields = std.meta.fields(Result); + +// Works for enums +const Status = enum { pending, done }; +const enum_fields = std.meta.fields(Status); + +// Works for error sets +const MyError = error{ NotFound, Timeout }; +const error_fields = std.meta.fields(MyError); +``` + +### Get Field Names + +```zig +const names = std.meta.fieldNames(Point); +// names.* == .{ "x", "y", "z" } + +for (names) |name| { + std.debug.print("{s}\n", .{name}); +} +``` + +### Get Specific Field + +```zig +// By enum literal +const x_info = std.meta.fieldInfo(Point, .x); +// x_info.name == "x" +// x_info.type == f32 +// x_info.default_value_ptr == null + +const z_info = std.meta.fieldInfo(Point, .z); +// z_info.default_value_ptr != null (has default 0) + +// Get field index +const idx = std.meta.fieldIndex(Point, "y"); // 1 +const bad = std.meta.fieldIndex(Point, "w"); // null +``` + +## Enum/Union Tag Operations + +### Get Tag Type + +```zig +const Status = enum(u8) { active = 1, inactive = 2 }; +const TagInt = std.meta.Tag(Status); // u8 + +const Tagged = union(enum) { a: u32, b: f32 }; +const TagEnum = std.meta.Tag(Tagged); // enum { a, b } +``` + +### Get Active Tag + +```zig +const Value = union(enum) { int: i32, float: f32 }; + +var v = Value{ .int = 42 }; +const tag = std.meta.activeTag(v); // Value.int + +switch (tag) { + .int => std.debug.print("integer\n", .{}), + .float => std.debug.print("float\n", .{}), +} +``` + +### Get All Tags + +```zig +const Color = enum { red, green, blue }; +const colors = std.meta.tags(Color); +// colors.* == .{ Color.red, Color.green, Color.blue } + +const MyError = error{ A, B }; +const errors = std.meta.tags(MyError); +// errors.* == .{ MyError.A, MyError.B } +``` + +## Type Construction + +### FieldEnum - Generate Enum from Fields + +```zig +const Point = struct { x: f32, y: f32 }; +const PointField = std.meta.FieldEnum(Point); +// Equivalent to: enum { x, y } + +// Iterate fields generically +inline for (std.meta.tags(PointField)) |field| { + const info = std.meta.fieldInfo(Point, field); + std.debug.print("{s}: {s}\n", .{ info.name, @typeName(info.type) }); +} +``` + +For tagged unions, returns the existing tag type if compatible. + +### DeclEnum - Generate Enum from Declarations + +```zig +const Api = struct { + pub fn getUser() void {} + pub fn deleteUser() void {} +}; + +const ApiMethod = std.meta.DeclEnum(Api); +// Equivalent to: enum { getUser, deleteUser } +``` + +### Int/Float Type Construction + +```zig +const U24 = std.meta.Int(.unsigned, 24); // u24 +const I7 = std.meta.Int(.signed, 7); // i7 +const F32 = std.meta.Float(32); // f32 +const F16 = std.meta.Float(16); // f16 +``` + +### Tuple Construction + +```zig +// From type array +const T1 = std.meta.Tuple(&.{ u32, f32, bool }); +// Equivalent to: struct { u32, f32, bool } + +// From function signature +const T2 = std.meta.ArgsTuple(fn (u32, f32) void); +// Equivalent to: struct { u32, f32 } +``` + +## Child/Element Types + +### Child - Direct Child Type + +```zig +std.meta.Child(*u8) // u8 +std.meta.Child([]u8) // u8 +std.meta.Child([5]u8) // u8 +std.meta.Child(?u8) // u8 +std.meta.Child(@Vector(4, f32)) // f32 +``` + +### Elem - Element Type of Memory Spans + +```zig +std.meta.Elem([5]u8) // u8 +std.meta.Elem([]u8) // u8 +std.meta.Elem([*]u8) // u8 +std.meta.Elem(*[10]u8) // u8 (through pointer to array) +std.meta.Elem(?[*]u8) // u8 (through optional) +``` + +### Sentinel + +```zig +std.meta.sentinel([:0]u8) // @as(u8, 0) +std.meta.sentinel([*:0]u8) // @as(u8, 0) +std.meta.sentinel([5:0]u8) // @as(u8, 0) +std.meta.sentinel([]u8) // null +std.meta.sentinel([5]u8) // null +``` + +### Sentinel Type Construction + +```zig +// Add sentinel to type +const S1 = std.meta.Sentinel([]u8, 0); // [:0]u8 +const S2 = std.meta.Sentinel([*]u8, 0); // [*:0]u8 +``` + +## Deep Equality + +```zig +const std = @import("std"); + +const Point = struct { x: i32, y: i32 }; + +const p1 = Point{ .x = 1, .y = 2 }; +const p2 = Point{ .x = 1, .y = 2 }; +const p3 = Point{ .x = 1, .y = 3 }; + +std.meta.eql(p1, p2) // true +std.meta.eql(p1, p3) // false + +// Works with nested structs, arrays, optionals, error unions +const Complex = struct { + data: [3]u8, + opt: ?i32, +}; + +const a = Complex{ .data = .{ 1, 2, 3 }, .opt = 42 }; +const b = Complex{ .data = .{ 1, 2, 3 }, .opt = 42 }; +std.meta.eql(a, b) // true + +// Pointers compared by address, not content +std.meta.eql(&p1, &p2) // false (different addresses) +std.meta.eql(&p1, &p1) // true +``` + +**Supported types:** structs, arrays, vectors, optionals, error unions, tagged unions, primitives. + +**Not supported:** untagged unions (compile error). + +## Type Queries + +### Check for Function/Method + +```zig +const S = struct { + value: u32, + pub fn method(self: *@This()) void { _ = self; } +}; + +std.meta.hasFn(S, "method") // true +std.meta.hasFn(S, "value") // false (field, not fn) +std.meta.hasFn(S, "missing") // false + +std.meta.hasMethod(S, "method") // true +std.meta.hasMethod(*S, "method") // true (through pointer) +std.meta.hasMethod([]S, "method") // false (slice, not single pointer) +``` + +### Check Unique Representation + +```zig +// True if type has no padding/unused bits +std.meta.hasUniqueRepresentation(u8) // true +std.meta.hasUniqueRepresentation(u32) // true +std.meta.hasUniqueRepresentation(i24) // false (padded to 4 bytes) + +// Struct with no padding +const Packed = struct { a: u32, b: u32 }; +std.meta.hasUniqueRepresentation(Packed) // true + +// Struct with padding +const Padded = struct { a: u32, b: u16 }; +std.meta.hasUniqueRepresentation(Padded) // false +``` + +### Container Layout + +```zig +const Auto = struct {}; +const Packed = packed struct {}; +const Extern = extern struct {}; + +std.meta.containerLayout(Auto) // .auto +std.meta.containerLayout(Packed) // .@"packed" +std.meta.containerLayout(Extern) // .@"extern" +``` + +### Alignment + +```zig +// For pointers, returns the pointed-to alignment (not pointer alignment) +std.meta.alignment(*align(16) u8) // 16 +std.meta.alignment([]align(8) u8) // 8 +std.meta.alignment(u8) // 1 +``` + +### Declarations + +```zig +const S = struct { + pub const value = 42; + pub fn method() void {} +}; + +const decls = std.meta.declarations(S); +// decls[0].name == "method" or "value" +``` + +## Error Handling + +```zig +// Check if value is error (deprecated: use std.enums.fromInt) +const result = std.math.divTrunc(u8, 5, 0); +std.meta.isError(result) // true + +// Enum from int (deprecated: use std.enums.fromInt) +const Color = enum { red, green, blue }; +const c = std.meta.intToEnum(Color, 1) catch unreachable; // Color.green +``` + +## TrailerFlags + +Memory-efficient optional field storage using bit flags: + +```zig +const std = @import("std"); + +const Flags = std.meta.TrailerFlags(struct { + name: []const u8, + age: u32, + email: []const u8, +}); + +// Initialize with some fields active +var flags = Flags.init(.{ + .name = true, + .age = true, + .email = false, +}); + +// Allocate only needed space +const size = flags.sizeInBytes(); +const data = try allocator.alignedAlloc(u8, @alignOf(@TypeOf(flags).Fields), size); +defer allocator.free(data); + +// Set values +flags.set(data.ptr, .name, "Alice"); +flags.set(data.ptr, .age, 30); + +// Get values (returns optional) +const name = flags.get(data.ptr, .name); // ?"Alice" +const email = flags.get(data.ptr, .email); // null + +// Set multiple at once +flags.setMany(data.ptr, .{ + .name = "Bob", + .age = 25, +}); +``` + +Use case: Allocating objects with many optional components where memory matters. + +## Generic Programming Patterns + +### Iterate All Fields + +```zig +fn printStruct(value: anytype) void { + const T = @TypeOf(value); + inline for (std.meta.fields(T)) |field| { + const v = @field(value, field.name); + std.debug.print("{s}: {any}\n", .{ field.name, v }); + } +} +``` + +### Create Default Instance + +```zig +fn createDefault(comptime T: type) T { + var result: T = undefined; + inline for (std.meta.fields(T)) |field| { + if (field.default_value_ptr) |ptr| { + @field(result, field.name) = @as(*const field.type, @ptrCast(@alignCast(ptr))).*; + } else { + @field(result, field.name) = std.mem.zeroes(field.type); + } + } + return result; +} +``` + +### Dynamic Field Access + +```zig +fn getField(comptime T: type, value: T, comptime name: []const u8) ?std.meta.fieldInfo(T, std.meta.stringToEnum(std.meta.FieldEnum(T), name) orelse return null).type { + const field = std.meta.stringToEnum(std.meta.FieldEnum(T), name) orelse return null; + return @field(value, @tagName(field)); +} +``` + +### Serialize to JSON-like + +```zig +fn toJson(value: anytype, writer: anytype) !void { + const T = @TypeOf(value); + switch (@typeInfo(T)) { + .@"struct" => |info| { + try writer.writeAll("{"); + inline for (info.fields, 0..) |field, i| { + if (i > 0) try writer.writeAll(","); + try writer.print("\"{s}\":", .{field.name}); + try toJson(@field(value, field.name), writer); + } + try writer.writeAll("}"); + }, + .int, .float => try writer.print("{}", .{value}), + .pointer => |ptr| if (ptr.size == .slice and ptr.child == u8) { + try writer.print("\"{s}\"", .{value}); + }, + else => try writer.writeAll("null"), + } +} +``` diff --git a/references/std-multi-array-list.md b/references/std-multi-array-list.md new file mode 100644 index 0000000..1572501 --- /dev/null +++ b/references/std-multi-array-list.md @@ -0,0 +1,137 @@ +# std.MultiArrayList + +Struct-of-Arrays container for cache-efficient struct storage. Stores each field in a separate contiguous array, reducing padding overhead and improving cache locality when accessing individual fields. + +## When to Use + +- Storing many structs where you often access only some fields +- Performance-critical code benefiting from cache-friendly access patterns +- Tagged unions (stores tags and data separately) + +## Initialization + +```zig +const Item = struct { + id: u32, + name: []const u8, + score: f32, +}; + +var list: std.MultiArrayList(Item) = .{}; +defer list.deinit(allocator); + +// Pre-allocate capacity +try list.ensureTotalCapacity(allocator, 100); +``` + +## Basic Operations + +```zig +// Append +try list.append(allocator, .{ .id = 1, .name = "foo", .score = 0.5 }); +list.appendAssumeCapacity(.{ .id = 2, .name = "bar", .score = 0.8 }); + +// Get/set individual elements +const item = list.get(0); // returns full struct +list.set(0, new_item); // set full struct + +// Access individual field arrays (MAIN BENEFIT) +const ids = list.items(.id); // []u32 slice +const scores = list.items(.score); // []f32 slice + +// Modify field directly +list.items(.score)[0] = 1.0; + +// Pop last element +const last = list.pop(); // returns ?Item + +// Length +const n = list.len; +``` + +## Slice API (More Efficient) + +When accessing multiple fields, use `slice()` to compute pointers once: + +```zig +const slices = list.slice(); + +// Now access fields without recomputing offsets +for (slices.items(.id), slices.items(.score)) |id, score| { + // process id and score together +} + +// Get/set via slice +const item = slices.get(index); +slices.set(index, new_item); +``` + +## Removal + +```zig +// O(1) but doesn't preserve order +list.swapRemove(index); + +// O(n) but preserves order +list.orderedRemove(index); + +// Remove multiple indices (must be sorted ascending) +list.orderedRemoveMany(&.{ 1, 5, 7, 9 }); +``` + +## Tagged Union Support + +MultiArrayList works with tagged unions, storing tags separately: + +```zig +const Value = union(enum) { + int: i64, + float: f64, + string: []const u8, +}; + +var values: std.MultiArrayList(Value) = .{}; +try values.append(allocator, .{ .int = 42 }); +try values.append(allocator, .{ .float = 3.14 }); + +// Access tags and data separately +const tags = values.items(.tags); // []meta.Tag(Value) +const data = values.items(.data); // []Value.Bare (untagged union) + +// Reconstruct full union +const full = values.get(0); // Value{ .int = 42 } +``` + +## Sorting + +```zig +// Sort with custom comparator (index-based) +list.sort(struct { + scores: []const f32, + + pub fn lessThan(ctx: @This(), a: usize, b: usize) bool { + return ctx.scores[a] < ctx.scores[b]; + } +}{ .scores = list.items(.score) }); + +// Also: sortUnstable, sortSpan, sortSpanUnstable +``` + +## Capacity Management + +```zig +try list.ensureTotalCapacity(allocator, 100); +try list.ensureUnusedCapacity(allocator, 10); +try list.resize(allocator, new_len); // doesn't initialize +list.shrinkAndFree(allocator, new_len); +list.shrinkRetainingCapacity(new_len); +list.clearRetainingCapacity(); +list.clearAndFree(allocator); +``` + +## Clone and Transfer + +```zig +const copy = try list.clone(allocator); +const owned_slice = list.toOwnedSlice(); // empties list, caller owns +``` diff --git a/references/std-net.md b/references/std-net.md new file mode 100644 index 0000000..1e1b45c --- /dev/null +++ b/references/std-net.md @@ -0,0 +1,642 @@ +# std.Io.net Reference (Zig 0.16.0) + +Cross-platform networking abstractions for TCP/IP connections, address handling, and DNS resolution. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 moves networking under `std.Io.net`. New networking code should accept/use `std.Io`, and stream/socket close/read/write operations should use the `io` parameter. Older `std.net` examples below are historical; translate them to `std.Io.net` before using them in new code. + +High-level HTTP clients also store `io`: + +```zig +var client: std.http.Client = .{ + .allocator = allocator, + .io = io, +}; +defer client.deinit(); +``` + +## Table of Contents +- [TCP Client](#tcp-client) +- [TCP Server](#tcp-server) +- [Address Types](#address-types) +- [Stream I/O](#stream-io) +- [DNS Resolution](#dns-resolution) +- [Unix Sockets](#unix-sockets) +- [Common Patterns](#common-patterns) + +## TCP Client + +### Connect by Hostname + +```zig +const std = @import("std"); +const net = std.net; + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + const allocator = gpa.allocator(); + + // Connect to host:port (handles DNS resolution) + const stream = try net.tcpConnectToHost(allocator, "example.com", 80); + defer stream.close(); + + // Create buffered reader/writer + var read_buf: [4096]u8 = undefined; + var write_buf: [1024]u8 = undefined; + + var reader = stream.reader(&read_buf); + var writer = stream.writer(&write_buf); + + // Write request + try writer.interface.writeAll("GET / HTTP/1.1\r\nHost: example.com\r\n\r\n"); + try writer.interface.flush(); + + // Read response + while (reader.interface().take(4096)) |chunk| { + std.debug.print("{s}", .{chunk}); + } else |err| switch (err) { + error.EndOfStream => {}, + else => return err, + } +} +``` + +### Connect by Address + +```zig +// Parse and connect to IP address directly (no DNS) +const address = try net.Address.parseIp4("192.168.1.1", 8080); +const stream = try net.tcpConnectToAddress(address); +defer stream.close(); +``` + +### Connect with IPv6 + +```zig +// IPv6 address +const addr6 = try net.Address.parseIp6("::1", 8080); +const stream = try net.tcpConnectToAddress(addr6); +defer stream.close(); + +// IPv6 with scope ID (link-local) +const link_local = try net.Address.resolveIp6("fe80::1%eth0", 8080); +``` + +## TCP Server + +### Basic Server + +```zig +const std = @import("std"); +const net = std.net; + +pub fn main() !void { + // Create address to listen on + const address = net.Address.initIp4(.{ 0, 0, 0, 0 }, 8080); + + // Start listening + var server = try address.listen(.{ + .reuse_address = true, + }); + defer server.deinit(); + + std.debug.print("Listening on port {d}\n", .{server.listen_address.getPort()}); + + // Accept loop + while (true) { + const conn = try server.accept(); + defer conn.stream.close(); + + // Handle connection + try handleClient(conn.stream, conn.address); + } +} + +fn handleClient(stream: net.Stream, client_addr: net.Address) !void { + var read_buf: [4096]u8 = undefined; + var write_buf: [1024]u8 = undefined; + + var reader = stream.reader(&read_buf); + var writer = stream.writer(&write_buf); + + // Read request + const request = reader.interface().takeDelimiter('\n') catch |err| switch (err) { + error.EndOfStream => return, + else => return err, + } orelse return; + + std.debug.print("Request from client: {s}\n", .{request}); + + // Send response + try writer.interface.writeAll("HTTP/1.1 200 OK\r\nContent-Length: 5\r\n\r\nHello"); + try writer.interface.flush(); +} +``` + +### Listen Options + +```zig +const server = try address.listen(.{ + // Allow address reuse (SO_REUSEADDR + SO_REUSEPORT on POSIX) + .reuse_address = true, + + // Connection backlog (default 128) + .kernel_backlog = 256, + + // Non-blocking accept (O_NONBLOCK) + .force_nonblocking = false, +}); +``` + +### Server on Any Available Port + +```zig +// Listen on port 0 to let OS assign an available port +const address = net.Address.initIp4(.{ 127, 0, 0, 1 }, 0); +var server = try address.listen(.{}); +defer server.deinit(); + +// Get the assigned port +const port = server.listen_address.getPort(); +std.debug.print("Listening on port {d}\n", .{port}); +``` + +## Address Types + +### Address Union + +```zig +pub const Address = extern union { + any: posix.sockaddr, + in: Ip4Address, + in6: Ip6Address, + un: posix.sockaddr.un, // Unix socket (if supported) +}; +``` + +### Creating Addresses + +```zig +// IPv4 from bytes +const addr4 = net.Address.initIp4(.{ 127, 0, 0, 1 }, 8080); + +// IPv6 from bytes +const addr6 = net.Address.initIp6( + .{ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1 }, // ::1 + 8080, // port + 0, // flowinfo + 0, // scope_id +); + +// Unix socket +const unix = try net.Address.initUnix("/tmp/my.sock"); +``` + +### Parsing Addresses + +```zig +// Parse IPv4 +const addr4 = try net.Address.parseIp4("192.168.1.1", 8080); + +// Parse IPv6 +const addr6 = try net.Address.parseIp6("2001:db8::1", 8080); + +// Parse either (tries IPv4 first, then IPv6) +const addr = try net.Address.parseIp("::1", 8080); + +// Parse IP:port format +// IPv4: "192.168.1.1:8080" +// IPv6: "[::1]:8080" (brackets required) +const addr_port = try net.Address.parseIpAndPort("[::1]:8080"); + +// Resolve with interface lookup (for link-local IPv6) +const resolved = try net.Address.resolveIp6("fe80::1%eth0", 8080); +``` + +### Address Methods + +```zig +var addr = net.Address.initIp4(.{ 127, 0, 0, 1 }, 8080); + +// Get/set port (native endian) +const port = addr.getPort(); // 8080 +addr.setPort(9090); + +// Get socket length for syscalls +const socklen = addr.getOsSockLen(); + +// Compare addresses +if (addr.eql(other_addr)) { + // addresses match +} + +// Format for printing +var buf: [64]u8 = undefined; +var writer: std.Io.Writer = .fixed(&buf); +try addr.format(&writer); +const formatted = writer.buffered(); // "127.0.0.1:8080" +``` + +### Ip4Address + +```zig +const Ip4Address = extern struct { + sa: posix.sockaddr.in, + + pub fn parse(buf: []const u8, port: u16) !Ip4Address; + pub fn init(addr: [4]u8, port: u16) Ip4Address; + pub fn getPort(self: Ip4Address) u16; + pub fn setPort(self: *Ip4Address, port: u16) void; + pub fn format(self: Ip4Address, w: *std.Io.Writer) !void; +}; +``` + +### Ip6Address + +```zig +const Ip6Address = extern struct { + sa: posix.sockaddr.in6, + + pub fn parse(buf: []const u8, port: u16) !Ip6Address; + pub fn resolve(buf: []const u8, port: u16) !Ip6Address; // handles %interface + pub fn init(addr: [16]u8, port: u16, flowinfo: u32, scope_id: u32) Ip6Address; + pub fn getPort(self: Ip6Address) u16; + pub fn setPort(self: *Ip6Address, port: u16) void; + pub fn format(self: Ip6Address, w: *std.Io.Writer) !void; +}; +``` + +## Stream I/O + +### Stream Type + +```zig +pub const Stream = struct { + handle: Handle, // fd on POSIX, SOCKET on Windows + + pub fn close(s: Stream) void; + pub fn reader(stream: Stream, buffer: []u8) Reader; + pub fn writer(stream: Stream, buffer: []u8) Writer; +}; +``` + +### Reading from Stream + +```zig +const stream = try net.tcpConnectToHost(allocator, "example.com", 80); +defer stream.close(); + +var buf: [4096]u8 = undefined; +var reader = stream.reader(&buf); +const r = reader.interface(); + +// Read bytes +const data = r.take(100) catch |err| switch (err) { + error.EndOfStream => &.{}, + error.ReadFailed => return reader.getError().?, +}; + +// Read until delimiter +const line = r.takeDelimiter('\n') catch |err| switch (err) { + error.EndOfStream => null, + error.StreamTooLong => return error.LineTooLong, + error.ReadFailed => return reader.getError().?, +} orelse return; + +// Discard bytes +_ = try r.discard(.limited(100)); + +// Stream to writer +_ = try r.streamRemaining(&output_writer); +``` + +### Writing to Stream + +```zig +var buf: [1024]u8 = undefined; +var writer = stream.writer(&buf); +const w = &writer.interface; + +// Write bytes +try w.writeAll("Hello, World!"); + +// Formatted output +try w.print("Count: {d}\n", .{42}); + +// MUST flush before close +try w.flush(); +``` + +### Error Handling + +```zig +var reader = stream.reader(&buf); +const r = reader.interface(); + +const data = r.take(100) catch |err| switch (err) { + error.EndOfStream => { + // Connection closed normally + return; + }, + error.ReadFailed => { + // Get underlying error + const read_err = reader.getError().?; + switch (read_err) { + error.ConnectionResetByPeer => return error.Disconnected, + error.SocketNotConnected => return error.Disconnected, + else => return read_err, + } + }, +}; +``` + +## DNS Resolution + +### Get Address List + +```zig +const std = @import("std"); +const net = std.net; + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + const allocator = gpa.allocator(); + + // Resolve hostname to addresses + const list = try net.getAddressList(allocator, "example.com", 80); + defer list.deinit(); + + // Canonical name (if available) + if (list.canon_name) |name| { + std.debug.print("Canonical name: {s}\n", .{name}); + } + + // Iterate addresses + for (list.addrs) |addr| { + var buf: [64]u8 = undefined; + var w: std.Io.Writer = .fixed(&buf); + try addr.format(&w); + std.debug.print("Address: {s}\n", .{w.buffered()}); + } +} +``` + +### Connect with Fallback + +`tcpConnectToHost` automatically tries all resolved addresses: + +```zig +// Tries each resolved address until one connects +const stream = net.tcpConnectToHost(allocator, "example.com", 80) catch |err| switch (err) { + error.ConnectionRefused => return error.ServerDown, + error.UnknownHostName => return error.DnsError, + error.TemporaryNameServerFailure => return error.DnsError, + else => return err, +}; +``` + +## Unix Sockets + +### Check Platform Support + +```zig +if (net.has_unix_sockets) { + // Unix sockets available +} +``` + +### Connect to Unix Socket + +```zig +const stream = try net.connectUnixSocket("/var/run/app.sock"); +defer stream.close(); + +var buf: [4096]u8 = undefined; +var reader = stream.reader(&buf); +var writer = stream.writer(&buf); +// ... use like TCP +``` + +### Unix Socket Server + +```zig +const address = try net.Address.initUnix("/tmp/my.sock"); +var server = try address.listen(.{ .reuse_address = true }); +defer server.deinit(); + +// Remove socket file on cleanup +defer std.fs.deleteFileAbsolute("/tmp/my.sock") catch {}; + +while (true) { + const conn = try server.accept(); + defer conn.stream.close(); + // handle connection... +} +``` + +## Common Patterns + +### Echo Server + +```zig +const std = @import("std"); +const net = std.net; + +pub fn main() !void { + const address = net.Address.initIp4(.{ 0, 0, 0, 0 }, 7); // echo port + var server = try address.listen(.{ .reuse_address = true }); + defer server.deinit(); + + while (true) { + const conn = try server.accept(); + defer conn.stream.close(); + + var buf: [4096]u8 = undefined; + var reader = conn.stream.reader(&buf); + var writer = conn.stream.writer(&buf); + + // Echo back everything received + _ = reader.interface().streamRemaining(&writer.interface) catch {}; + writer.interface.flush() catch {}; + } +} +``` + +### Simple HTTP GET + +```zig +fn httpGet(allocator: Allocator, host: []const u8, path: []const u8) ![]u8 { + const stream = try net.tcpConnectToHost(allocator, host, 80); + defer stream.close(); + + var write_buf: [1024]u8 = undefined; + var writer = stream.writer(&write_buf); + const w = &writer.interface; + + try w.print("GET {s} HTTP/1.1\r\n", .{path}); + try w.print("Host: {s}\r\n", .{host}); + try w.writeAll("Connection: close\r\n\r\n"); + try w.flush(); + + var read_buf: [4096]u8 = undefined; + var reader = stream.reader(&read_buf); + + var response: std.ArrayList(u8) = .empty; + defer response.deinit(allocator); + + while (true) { + const chunk = reader.interface().take(4096) catch |err| switch (err) { + error.EndOfStream => break, + error.ReadFailed => return reader.getError().?, + }; + try response.appendSlice(allocator, chunk); + } + + return response.toOwnedSlice(allocator); +} +``` + +### Non-blocking Accept with Timeout + +```zig +const std = @import("std"); +const net = std.net; +const posix = std.posix; + +fn acceptWithTimeout(server: *net.Server, timeout_ms: i32) !?net.Server.Connection { + var pfd = [1]posix.pollfd{.{ + .fd = server.stream.handle, + .events = posix.POLL.IN, + .revents = undefined, + }}; + + const ready = try posix.poll(&pfd, timeout_ms); + if (ready == 0) return null; // timeout + + return try server.accept(); +} +``` + +### Address Validation + +```zig +fn isValidIpAddress(str: []const u8) bool { + _ = net.Address.parseIp(str, 0) catch return false; + return true; +} + +fn isValidHostname(hostname: []const u8) bool { + return net.isValidHostName(hostname); +} +``` + +### Dual-Stack Server (IPv4 + IPv6) + +```zig +// Listen on IPv6 with dual-stack (accepts both IPv4 and IPv6) +const address = net.Address.initIp6( + .{ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }, // :: + 8080, + 0, + 0, +); + +var server = try address.listen(.{ .reuse_address = true }); +defer server.deinit(); + +// IPv4 clients appear as IPv4-mapped IPv6 addresses (::ffff:x.x.x.x) +``` + +### Connection Pool Pattern + +```zig +const Connection = struct { + stream: net.Stream, + in_use: bool, +}; + +const Pool = struct { + connections: std.ArrayList(Connection), + allocator: Allocator, + + pub fn acquire(self: *Pool, address: net.Address) !net.Stream { + // Find free connection + for (self.connections.items) |*conn| { + if (!conn.in_use) { + conn.in_use = true; + return conn.stream; + } + } + // Create new connection + const stream = try net.tcpConnectToAddress(address); + try self.connections.append(self.allocator, .{ + .stream = stream, + .in_use = true, + }); + return stream; + } + + pub fn release(self: *Pool, stream: net.Stream) void { + for (self.connections.items) |*conn| { + if (conn.stream.handle == stream.handle) { + conn.in_use = false; + return; + } + } + } + + pub fn deinit(self: *Pool) void { + for (self.connections.items) |conn| { + conn.stream.close(); + } + self.connections.deinit(self.allocator); + } +}; +``` + +## Error Types + +### Connection Errors + +```zig +pub const TcpConnectToHostError = GetAddressListError || TcpConnectToAddressError; + +pub const TcpConnectToAddressError = posix.SocketError || posix.ConnectError; +// Includes: ConnectionRefused, NetworkUnreachable, ConnectionTimedOut, etc. +``` + +### DNS Errors + +```zig +pub const GetAddressListError = error{ + TemporaryNameServerFailure, + NameServerFailure, + AddressFamilyNotSupported, + UnknownHostName, + HostLacksNetworkAddresses, + // ... and others +}; +``` + +### Address Parse Errors + +```zig +pub const IPv4ParseError = error{ + Overflow, + InvalidEnd, + InvalidCharacter, + Incomplete, + NonCanonical, // e.g., leading zeros like "01.02.03.04" +}; + +pub const IPv6ParseError = error{ + Overflow, + InvalidEnd, + InvalidCharacter, + Incomplete, + InvalidIpv4Mapping, +}; +``` diff --git a/references/std-os.md b/references/std-os.md new file mode 100644 index 0000000..25bd896 --- /dev/null +++ b/references/std-os.md @@ -0,0 +1,628 @@ +# std.os - OS-Specific APIs Reference (Zig 0.16.0) + +Thin wrappers around OS-specific APIs. Zig 0.16 moves many blocking/nondeterministic operations behind `std.Io`; prefer `std.Io` abstractions for portable code and drop down to `std.posix` / `std.os.windows` only for explicitly platform-specific code. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +## Table of Contents +- [Module Structure](#module-structure) +- [Platform Submodules](#platform-submodules) +- [Linux-Specific APIs](#linux-specific-apis) +- [Windows-Specific APIs](#windows-specific-apis) +- [WASI-Specific APIs](#wasi-specific-apis) +- [io_uring (Linux)](#io_uring-linux) +- [Common Functions](#common-functions) +- [Common Patterns](#common-patterns) + +## Module Structure + +```zig +std.os.linux // Linux syscalls and constants +std.os.windows // Windows NT APIs +std.os.wasi // WebAssembly System Interface +std.os.plan9 // Plan 9 system calls +std.os.uefi // UEFI firmware interface +std.os.emscripten // Emscripten runtime +std.os.freebsd // FreeBSD-specific definitions + +std.os.environ // Environment variables (populated at startup) +std.os.argv // Command line arguments (POSIX only) +``` + +**Note**: For most use cases, prefer `std.posix` (cross-platform POSIX-like APIs) or `std.fs`/`std.process` (high-level abstractions). Use `std.os` when you need direct OS-specific functionality. + +## Platform Submodules + +### When to Use Each Level + +```zig +// High-level (recommended for most code) +const file = try std.fs.cwd().openFile("data.txt", .{}); + +// POSIX-level (cross-platform low-level) +const fd = try std.posix.open("data.txt", .{}, 0); + +// OS-specific (platform-specific features) +const result = std.os.linux.syscall3(.read, fd, buf.ptr, buf.len); +``` + +## Linux-Specific APIs + +### Direct Syscalls + +```zig +const linux = std.os.linux; + +// Raw syscall interface +const result = linux.syscall3(.write, fd, @intFromPtr(buf.ptr), buf.len); +if (linux.E.init(result) != .SUCCESS) { + // handle error +} + +// Common syscalls with typed wrappers +_ = linux.dup(old_fd); +_ = linux.dup2(old_fd, new_fd); +_ = linux.fork(); +_ = linux.execve(path, argv, envp); +_ = linux.chdir(path); +_ = linux.chroot(path); +``` + +### Memory Mapping + +```zig +const linux = std.os.linux; + +// mmap with typed flags +const addr = linux.mmap( + null, + length, + linux.PROT.READ | linux.PROT.WRITE, + .{ .TYPE = .PRIVATE, .ANONYMOUS = true }, + -1, + 0, +); +if (addr == linux.MAP_FAILED) { + // handle error +} + +// Remap +_ = linux.mremap(old_addr, old_size, new_size, .{ .MAYMOVE = true }, null); + +// Unmap +_ = linux.munmap(addr, length); +``` + +### File Operations + +```zig +const linux = std.os.linux; + +// Open flags (architecture-specific packed struct) +const flags: linux.O = .{ + .ACCMODE = .RDWR, + .CREAT = true, + .TRUNC = true, + .CLOEXEC = true, +}; + +// fallocate - preallocate file space +_ = linux.fallocate(fd, 0, 0, size); + +// utimensat - set file timestamps +_ = linux.utimensat(dirfd, path, ×, 0); +``` + +### Futex (Fast Userspace Mutex) + +```zig +const linux = std.os.linux; + +// Wait on futex +_ = linux.futex( + &futex_word, + .{ .op = .WAIT, .PRIVATE = true }, + expected_value, + .{ .timeout = &timeout }, + null, + 0, +); + +// Wake waiters +_ = linux.futex( + &futex_word, + .{ .op = .WAKE, .PRIVATE = true }, + num_to_wake, + .{ .val2 = 0 }, + null, + 0, +); +``` + +### Signals + +```zig +const linux = std.os.linux; + +// Signal handling +var act: linux.Sigaction = .{ + .handler = .{ .handler = signal_handler }, + .mask = linux.empty_sigset, + .flags = .{}, +}; +_ = linux.sigaction(linux.SIG.INT, &act, null); + +// Kill process +_ = linux.kill(pid, linux.SIG.TERM); +``` + +### Epoll + +```zig +const linux = std.os.linux; + +// Create epoll instance +const epfd = linux.epoll_create1(.{ .CLOEXEC = true }); + +// Add file descriptor +var event: linux.epoll_event = .{ + .events = linux.EPOLL.IN | linux.EPOLL.ET, + .data = .{ .fd = client_fd }, +}; +_ = linux.epoll_ctl(epfd, .ADD, client_fd, &event); + +// Wait for events +var events: [64]linux.epoll_event = undefined; +const n = linux.epoll_wait(epfd, &events, -1); +for (events[0..n]) |ev| { + // handle event +} +``` + +### getauxval + +```zig +const linux = std.os.linux; + +// Get auxiliary vector values (set by kernel at process start) +const page_size = linux.getauxval(std.elf.AT_PAGESZ); +const entry_point = linux.getauxval(std.elf.AT_ENTRY); +const platform = linux.getauxval(std.elf.AT_PLATFORM); +``` + +## Windows-Specific APIs + +### File Operations + +```zig +const windows = std.os.windows; + +// Open file with NT API +const handle = try windows.OpenFile(path_utf16, .{ + .access_mask = windows.GENERIC_READ | windows.GENERIC_WRITE, + .creation = windows.FILE_OPEN, + .share_access = windows.FILE_SHARE_READ, + .filter = .file_only, + .follow_symlinks = true, +}); +defer windows.CloseHandle(handle); +``` + +### Process Information + +```zig +const windows = std.os.windows; + +// Current process/thread +const process = windows.GetCurrentProcess(); +const pid = windows.GetCurrentProcessId(); +const thread = windows.GetCurrentThread(); +const tid = windows.GetCurrentThreadId(); + +// Last error +const err = windows.GetLastError(); +``` + +### Pipes + +```zig +const windows = std.os.windows; + +var read_handle: windows.HANDLE = undefined; +var write_handle: windows.HANDLE = undefined; +var sa: windows.SECURITY_ATTRIBUTES = .{ + .nLength = @sizeOf(windows.SECURITY_ATTRIBUTES), + .lpSecurityDescriptor = null, + .bInheritHandle = windows.TRUE, +}; + +try windows.CreatePipe(&read_handle, &write_handle, &sa); +``` + +### Submodules + +```zig +windows.kernel32 // kernel32.dll functions +windows.ntdll // ntdll.dll functions (NT native API) +windows.advapi32 // advapi32.dll (security, registry) +windows.ws2_32 // Winsock 2 networking +windows.crypt32 // Cryptographic functions +windows.nls // National Language Support +``` + +## WASI-Specific APIs + +### File Descriptors + +```zig +const wasi = std.os.wasi; + +// Read/write +var nread: usize = undefined; +switch (wasi.fd_read(fd, &iovs, iovs.len, &nread)) { + .SUCCESS => {}, + .BADF => return error.BadFileDescriptor, + else => |e| return unexpectedErrno(e), +} + +// Seek +var new_offset: wasi.filesize_t = undefined; +_ = wasi.fd_seek(fd, offset, .SET, &new_offset); + +// Sync +_ = wasi.fd_sync(fd); +_ = wasi.fd_datasync(fd); +``` + +### Path Operations + +```zig +const wasi = std.os.wasi; + +// Create directory +_ = wasi.path_create_directory(dirfd, path.ptr, path.len); + +// Open file +var result_fd: wasi.fd_t = undefined; +_ = wasi.path_open( + dirfd, + .{ .SYMLINK_FOLLOW = true }, + path.ptr, + path.len, + .{ .CREAT = true }, + rights_base, + rights_inheriting, + .{}, + &result_fd, +); + +// Symlinks +_ = wasi.path_symlink(old_path.ptr, old_path.len, dirfd, new_path.ptr, new_path.len); +_ = wasi.path_readlink(dirfd, path.ptr, path.len, buf.ptr, buf.len, &bufused); +``` + +### Clock + +```zig +const wasi = std.os.wasi; + +var timestamp: wasi.timestamp_t = undefined; +switch (wasi.clock_time_get(.MONOTONIC, 1, ×tamp)) { + .SUCCESS => {}, + else => |e| return error.ClockGetFailed, +} +``` + +### Environment and Arguments + +```zig +const wasi = std.os.wasi; + +// Arguments +var argc: usize = undefined; +var argv_buf_size: usize = undefined; +_ = wasi.args_sizes_get(&argc, &argv_buf_size); + +// Environment +var environ_count: usize = undefined; +var environ_buf_size: usize = undefined; +_ = wasi.environ_sizes_get(&environ_count, &environ_buf_size); +``` + +### Random + +```zig +const wasi = std.os.wasi; + +var buf: [32]u8 = undefined; +switch (wasi.random_get(&buf, buf.len)) { + .SUCCESS => {}, + else => return error.RandomFailed, +} +``` + +## io_uring (Linux) + +High-performance async I/O for Linux 5.4+. + +### Basic Setup + +```zig +const IoUring = std.os.linux.IoUring; + +// Initialize with 256 entries +var ring = try IoUring.init(256, 0); +defer ring.deinit(); + +// With custom parameters +var params = std.mem.zeroInit(std.os.linux.io_uring_params, .{ + .flags = std.os.linux.IORING_SETUP_SQPOLL, // kernel-side submission + .sq_thread_idle = 2000, // ms before SQ thread sleeps +}); +var ring = try IoUring.init_params(256, ¶ms); +``` + +### Submitting Operations + +```zig +// Get submission queue entry +const sqe = try ring.get_sqe(); + +// Prepare read operation +sqe.prep_read(fd, buffer, offset); +sqe.user_data = my_context; // identify this request in completion + +// Or write +sqe.prep_write(fd, data, offset); + +// Submit to kernel +const submitted = try ring.submit(); +``` + +### Waiting for Completions + +```zig +// Submit and wait for at least 1 completion +_ = try ring.submit_and_wait(1); + +// Process completions +while (ring.cq_ready() > 0) { + const cqe = ring.peek_cqe() orelse break; + + const user_data = cqe.user_data; + const result = cqe.res; // bytes transferred or -errno + + if (result < 0) { + const err = std.os.linux.E.init(@intCast(-result)); + // handle error + } + + ring.cq_advance(1); // mark CQE as consumed +} +``` + +### Common Operations + +```zig +// File I/O +sqe.prep_read(fd, buf, offset); +sqe.prep_write(fd, data, offset); +sqe.prep_readv(fd, iovecs, offset); +sqe.prep_writev(fd, iovecs, offset); + +// Fixed buffers (pre-registered, zero-copy) +sqe.prep_read_fixed(fd, buf, offset, buf_index); +sqe.prep_write_fixed(fd, data, offset, buf_index); + +// Network +sqe.prep_accept(listen_fd, &client_addr, &addr_len, 0); +sqe.prep_connect(fd, &addr, addr_len); +sqe.prep_recv(fd, buf, 0); +sqe.prep_send(fd, data, 0); + +// Timeouts +sqe.prep_timeout(×pec, 0, 0); +sqe.prep_link_timeout(×pec, 0); // timeout linked op + +// File operations +sqe.prep_openat(dirfd, path, flags, mode); +sqe.prep_close(fd); +sqe.prep_statx(dirfd, path, flags, mask, &statx); + +// Misc +sqe.prep_nop(); // no-op (for benchmarking) +sqe.prep_cancel(user_data, 0); // cancel pending request +``` + +### Linked Operations + +```zig +// Chain operations: second runs only if first succeeds +const sqe1 = try ring.get_sqe(); +sqe1.prep_write(fd, header, 0); +sqe1.flags |= std.os.linux.IOSQE_IO_LINK; + +const sqe2 = try ring.get_sqe(); +sqe2.prep_write(fd, body, header.len); + +_ = try ring.submit(); +``` + +### Buffer Registration + +```zig +// Register buffers for zero-copy I/O +var buffers: [16][4096]u8 = undefined; +var iovecs: [16]std.posix.iovec = undefined; +for (&iovecs, &buffers) |*iov, *buf| { + iov.* = .{ .base = buf, .len = buf.len }; +} + +try ring.register_buffers(&iovecs); +defer ring.unregister_buffers() catch {}; + +// Use registered buffer +const sqe = try ring.get_sqe(); +sqe.prep_read_fixed(fd, &buffers[0], 0, 0); // buf_index = 0 +``` + +### File Descriptor Registration + +```zig +// Register FDs for faster access +var fds = [_]std.posix.fd_t{ fd1, fd2, fd3 }; +try ring.register_files(&fds); +defer ring.unregister_files() catch {}; + +// Use with IOSQE_FIXED_FILE flag +const sqe = try ring.get_sqe(); +sqe.prep_read(0, buf, 0); // fd index, not actual fd +sqe.flags |= std.os.linux.IOSQE_FIXED_FILE; +``` + +## Common Functions + +### getFdPath + +Get canonical path from file descriptor (not all platforms). + +```zig +var buf: [std.fs.max_path_bytes]u8 = undefined; +const path = try std.os.getFdPath(fd, &buf); +std.debug.print("Path: {s}\n", .{path}); + +// Check if supported at comptime +if (comptime std.os.isGetFdPathSupportedOnTarget(builtin.os)) { + // safe to call +} +``` + +**Supported**: Linux, macOS, FreeBSD, Windows, Solaris/illumos, DragonFly (6.0+), NetBSD (10.0+) + +### accessW (Windows) + +Check file accessibility with WTF-16LE path. + +```zig +const path_w = std.unicode.utf8ToUtf16LeStringLiteral("C:\\file.txt"); +std.os.accessW(path_w) catch |err| switch (err) { + error.FileNotFound => {}, + error.AccessDenied => {}, + else => return err, +}; +``` + +### WASI stat functions + +```zig +// stat by path +const stat = try std.os.fstatat_wasi(dirfd, path, .{ .SYMLINK_FOLLOW = true }); + +// stat by fd +const stat = try std.os.fstat_wasi(fd); + +stat.size; // file size +stat.filetype; // .REGULAR_FILE, .DIRECTORY, .SYMBOLIC_LINK, etc. +stat.atim; // access time (nanoseconds) +stat.mtim; // modification time +stat.ctim; // status change time +``` + +## Common Patterns + +### Platform-Specific Code + +```zig +const builtin = @import("builtin"); + +fn platformSpecific() !void { + switch (builtin.os.tag) { + .linux => { + const linux = std.os.linux; + // Linux-specific code + }, + .windows => { + const windows = std.os.windows; + // Windows-specific code + }, + .wasi => { + const wasi = std.os.wasi; + // WASI-specific code + }, + else => @compileError("Unsupported OS"), + } +} +``` + +### io_uring Event Loop + +```zig +fn eventLoop(ring: *std.os.linux.IoUring) !void { + while (running) { + // Submit pending and wait for completions + _ = try ring.submit_and_wait(1); + + // Process all available completions + while (ring.cq_ready() > 0) { + const cqe = ring.peek_cqe() orelse break; + defer ring.cq_advance(1); + + const ctx = @as(*Context, @ptrFromInt(cqe.user_data)); + try ctx.handle_completion(cqe.res); + } + } +} +``` + +### Handling Syscall Errors + +```zig +const linux = std.os.linux; + +fn readSyscall(fd: i32, buf: []u8) !usize { + const result = linux.syscall3(.read, @intCast(fd), @intFromPtr(buf.ptr), buf.len); + + switch (linux.E.init(result)) { + .SUCCESS => return result, + .INTR => return error.Interrupted, + .AGAIN => return error.WouldBlock, + .BADF => return error.BadFileDescriptor, + .FAULT => return error.BadAddress, + .INVAL => return error.InvalidArgument, + .IO => return error.InputOutput, + .ISDIR => return error.IsDir, + else => |e| return std.posix.unexpectedErrno(e), + } +} +``` + +### Windows Error Handling + +```zig +const windows = std.os.windows; + +fn windowsOperation() !void { + const result = windows.kernel32.SomeFunction(...); + if (result == windows.FALSE) { + switch (windows.GetLastError()) { + .ERROR_FILE_NOT_FOUND => return error.FileNotFound, + .ERROR_ACCESS_DENIED => return error.AccessDenied, + else => |e| return windows.unexpectedError(e), + } + } +} +``` + +### Cross-Platform File Descriptor Path + +```zig +fn getFilePath(fd: std.posix.fd_t, allocator: Allocator) ![]u8 { + if (comptime !std.os.isGetFdPathSupportedOnTarget(builtin.os)) { + return error.Unsupported; + } + + var buf: [std.fs.max_path_bytes]u8 = undefined; + const path = try std.os.getFdPath(fd, &buf); + return try allocator.dupe(u8, path); +} +``` diff --git a/references/std-priority-dequeue.md b/references/std-priority-dequeue.md new file mode 100644 index 0000000..8fd3c4d --- /dev/null +++ b/references/std-priority-dequeue.md @@ -0,0 +1,183 @@ +# std.PriorityDequeue (Zig 0.16.0) + +Primary release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 changed priority dequeues to align with unmanaged containers: + +- Initialize with `.empty`. +- `add` -> `push`. +- `addSlice` -> `pushSlice`. +- `addUnchecked` -> `pushUnchecked`. +- `removeMin` / `removeMinOrNull` -> `popMin`. +- `removeMax` / `removeMaxOrNull` -> `popMax`. +- `removeIndex` -> `popIndex`. + +Old examples below may use removed 0.15 names; translate them before using in 0.16 code. + +A min-max heap that efficiently supports both min and max extraction. Unlike `PriorityQueue`, you can pop from either end. + +## When to Use + +- Need both min and max extraction +- Double-ended priority queue +- Sliding window min/max +- Median maintenance (with two heaps) + +## Initialization + +```zig +const std = @import("std"); + +fn compare(context: void, a: u32, b: u32) std.math.Order { + _ = context; + return std.math.order(a, b); +} + +const PDQ = std.PriorityDequeue(u32, void, compare); + +var dequeue = PDQ.init(allocator, {}); +defer dequeue.deinit(); +``` + +## Basic Operations + +```zig +// Add elements +try dequeue.add(54); +try dequeue.add(12); +try dequeue.add(7); + +// Add multiple +try dequeue.addSlice(&[_]u32{ 1, 2, 3 }); + +// Peek at min/max (doesn't remove) +if (dequeue.peekMin()) |min| { + std.debug.print("min: {}\n", .{min}); +} +if (dequeue.peekMax()) |max| { + std.debug.print("max: {}\n", .{max}); +} + +// Remove min/max +const min = dequeue.removeMin(); // asserts non-empty +const max = dequeue.removeMax(); // asserts non-empty + +// Safe removal (returns null if empty) +const maybe_min = dequeue.removeMinOrNull(); +const maybe_max = dequeue.removeMaxOrNull(); + +// Size +const n = dequeue.count(); +const cap = dequeue.capacity(); +``` + +## From Existing Slice + +```zig +// Take ownership of slice, heapify in place +var items = try allocator.dupe(u32, &[_]u32{ 5, 3, 8, 1, 2 }); +var dequeue = PDQ.fromOwnedSlice(allocator, items, {}); +defer dequeue.deinit(); +``` + +## Update Priority + +```zig +try dequeue.update(old_value, new_value); +// Error if old_value not found +``` + +## Remove by Index + +```zig +const removed = dequeue.removeIndex(index); +``` + +## Iteration + +```zig +// Iterate (order is NOT priority order) +var it = dequeue.iterator(); +while (it.next()) |elem| { + // process elem +} +it.reset(); +``` + +## Capacity Management + +```zig +try dequeue.ensureTotalCapacity(100); +try dequeue.ensureUnusedCapacity(10); +dequeue.shrinkAndFree(new_capacity); +``` + +## Context-Based Comparator + +```zig +fn compareByScore(scores: []const u32, a: usize, b: usize) std.math.Order { + return std.math.order(scores[a], scores[b]); +} + +const IndexPDQ = std.PriorityDequeue(usize, []const u32, compareByScore); + +const scores = [_]u32{ 50, 30, 80, 20 }; +var dequeue = IndexPDQ.init(allocator, &scores); +``` + +## Complete Example: Bounded Range Tracker + +```zig +const std = @import("std"); + +fn order(_: void, a: i32, b: i32) std.math.Order { + return std.math.order(a, b); +} + +const RangePDQ = std.PriorityDequeue(i32, void, order); + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + + var tracker = RangePDQ.init(gpa.allocator(), {}); + defer tracker.deinit(); + + // Add values + try tracker.add(10); + try tracker.add(5); + try tracker.add(20); + try tracker.add(3); + try tracker.add(15); + + // Get range without removing + const min = tracker.peekMin().?; // 3 + const max = tracker.peekMax().?; // 20 + const range = max - min; // 17 + + std.debug.print("Range: {} to {} = {}\n", .{ min, max, range }); + + // Pop from both ends + _ = tracker.removeMin(); // removes 3 + _ = tracker.removeMax(); // removes 20 + + // New range is 5 to 15 +} +``` + +## Difference from PriorityQueue + +| Feature | PriorityQueue | PriorityDequeue | +|---------|--------------|-----------------| +| Pop min | Yes | Yes | +| Pop max | No (unless you reverse comparator) | Yes | +| Peek min | Yes | Yes | +| Peek max | No | Yes | +| Structure | Binary heap | Min-max heap | + +## Notes + +- Both `removeMin()` and `removeMax()` are O(log n) +- `peekMin()` is O(1), `peekMax()` is O(1) after first 2 elements +- Iterator order is heap array order, not priority order +- Use when you need efficient access to both extremes diff --git a/references/std-priority-queue.md b/references/std-priority-queue.md new file mode 100644 index 0000000..8d86de0 --- /dev/null +++ b/references/std-priority-queue.md @@ -0,0 +1,192 @@ +# std.PriorityQueue (Zig 0.16.0) + +Primary release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 changed priority queues to align with unmanaged containers: + +- The queue no longer stores an allocator. +- Empty queues can use `.empty`. +- `init` -> `initContext` when context is needed. +- `add` -> `push`. +- `addUnchecked` -> `pushUnchecked`. +- `addSlice` -> `pushSlice`. +- `remove` / `removeOrNull` -> `pop`. +- `removeIndex` -> `popIndex`. + +Old examples below may use removed 0.15 names; translate them before using in 0.16 code. + +A binary heap-based priority queue. Efficiently retrieves elements by priority order. + +## When to Use + +- Need to repeatedly extract min or max element +- Task scheduling by priority +- Dijkstra's algorithm, A* pathfinding +- Event-driven simulation (process earliest event first) + +## Initialization + +```zig +const std = @import("std"); + +// Min-heap comparator (smallest first) +fn lessThan(context: void, a: u32, b: u32) std.math.Order { + _ = context; + return std.math.order(a, b); +} + +const PQ = std.PriorityQueue(u32, void, lessThan); + +var queue = PQ.init(allocator, {}); +defer queue.deinit(); +``` + +## Max-Heap + +```zig +fn greaterThan(context: void, a: u32, b: u32) std.math.Order { + _ = context; + return std.math.order(a, b).invert(); +} + +const MaxPQ = std.PriorityQueue(u32, void, greaterThan); +``` + +## Basic Operations + +```zig +// Add elements +try queue.add(54); +try queue.add(12); +try queue.add(7); + +// Add multiple +try queue.addSlice(&[_]u32{ 1, 2, 3 }); + +// Peek at highest priority (doesn't remove) +if (queue.peek()) |top| { + std.debug.print("top: {}\n", .{top}); // 7 for min-heap +} + +// Remove highest priority +const top = queue.remove(); // asserts non-empty +const maybe = queue.removeOrNull(); // returns ?T + +// Size +const n = queue.count(); +const cap = queue.capacity(); +``` + +## From Existing Slice + +```zig +// Take ownership of slice, heapify in place +var items = try allocator.dupe(u32, &[_]u32{ 5, 3, 8, 1, 2 }); +var queue = PQ.fromOwnedSlice(allocator, items, {}); +defer queue.deinit(); +// Now queue is a valid heap +``` + +## Update Priority + +```zig +// Change priority of existing element +try queue.update(old_value, new_value); +// Error if old_value not found +``` + +## Remove by Index + +```zig +// Remove element at specific position (not priority order) +const removed = queue.removeIndex(index); +``` + +## Iteration (Non-Priority Order) + +```zig +// Iterate without removing (order is NOT priority order!) +var it = queue.iterator(); +while (it.next()) |elem| { + // process elem +} +it.reset(); // restart iteration +``` + +## Capacity Management + +```zig +try queue.ensureTotalCapacity(100); +try queue.ensureUnusedCapacity(10); +queue.shrinkAndFree(new_capacity); +queue.clearRetainingCapacity(); +queue.clearAndFree(); +``` + +## Context-Based Comparator + +For comparing by external data (e.g., indices into an array): + +```zig +fn compareByScore(scores: []const u32, a: usize, b: usize) std.math.Order { + return std.math.order(scores[a], scores[b]); +} + +const IndexPQ = std.PriorityQueue(usize, []const u32, compareByScore); + +const scores = [_]u32{ 50, 30, 80, 20 }; +var queue = IndexPQ.init(allocator, &scores); +defer queue.deinit(); + +try queue.add(0); // score 50 +try queue.add(1); // score 30 +try queue.add(2); // score 80 +try queue.add(3); // score 20 + +// Removes index 3 (score 20 is smallest) +const best = queue.remove(); // 3 +``` + +## Complete Example: Task Scheduler + +```zig +const std = @import("std"); + +const Task = struct { + name: []const u8, + priority: u32, // lower = more urgent +}; + +fn taskCompare(_: void, a: Task, b: Task) std.math.Order { + return std.math.order(a.priority, b.priority); +} + +const TaskQueue = std.PriorityQueue(Task, void, taskCompare); + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + + var tasks = TaskQueue.init(gpa.allocator(), {}); + defer tasks.deinit(); + + try tasks.add(.{ .name = "low priority", .priority = 100 }); + try tasks.add(.{ .name = "urgent", .priority = 1 }); + try tasks.add(.{ .name = "medium", .priority = 50 }); + + while (tasks.removeOrNull()) |task| { + std.debug.print("Processing: {s}\n", .{task.name}); + } + // Output: + // Processing: urgent + // Processing: medium + // Processing: low priority +} +``` + +## Notes + +- Heap property: parent has higher priority than children +- `remove()` is O(log n), `peek()` is O(1) +- Iterator order is NOT priority order (it's heap array order) +- Use `removeOrNull()` for safe extraction from potentially empty queue diff --git a/references/std-process.md b/references/std-process.md new file mode 100644 index 0000000..454e813 --- /dev/null +++ b/references/std-process.md @@ -0,0 +1,241 @@ +# std.process - Process API Reference (Zig 0.16.0) + +Primary release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 moves process I/O, args, environment, current directory, and child process management behind explicit `std.Io` and Juicy Main. + +## Juicy Main + +Preferred application entry: + +```zig +const std = @import("std"); + +pub fn main(init: std.process.Init) !void { + const gpa = init.gpa; + const io = init.io; + const arena = init.arena.allocator(); + + const args = try init.minimal.args.toSlice(arena); + const env = init.environ_map; + const preopens = init.preopens; + + _ = .{ gpa, io, args, env, preopens }; +} +``` + +`std.process.Init` provides: + +- `minimal.args` +- `minimal.environ` +- `arena` +- `gpa` +- `io` +- `environ_map` +- `preopens` + +Use `std.process.Init.Minimal` only when a program deliberately wants less setup. + +## Args + +Prefer args from `std.process.Init`: + +```zig +const args = try init.minimal.args.toSlice(init.arena.allocator()); +``` + +Avoid older global argument APIs in new 0.16 code unless you are inside compatibility code. + +## Environment + +Prefer `init.environ_map` at application boundaries. + +Important error rename: + +- `error.EnvironmentVariableNotFound` -> `error.EnvironmentVariableMissing` + +When spawning, pass an environment map through process options: + +```zig +const result = try std.process.run(gpa, io, .{ + .argv = &.{ "tool" }, + .environ_map = init.environ_map, +}); +defer gpa.free(result.stdout); +defer gpa.free(result.stderr); +``` + +## Current Directory + +```zig +var buffer: [std.Io.Dir.max_path_bytes]u8 = undefined; +const n = try std.process.currentPath(io, &buffer); +const cwd = buffer[0..n]; +``` + +```zig +const cwd = try std.process.currentPathAlloc(io, gpa); +defer gpa.free(cwd); +``` + +Do not add new `std.process.getCwd*` callsites. + +## Run and Capture Output + +Use `std.process.run(gpa, io, options)`. + +```zig +const result = try std.process.run(gpa, io, .{ + .argv = &.{ "git", "status", "--short" }, + .stdout_limit = .limited(64 * 1024), + .stderr_limit = .limited(64 * 1024), + .cwd = .inherit, +}); +defer gpa.free(result.stdout); +defer gpa.free(result.stderr); + +switch (result.term) { + .exited => |code| if (code != 0) return error.CommandFailed, + else => return error.CommandFailed, +} +``` + +Important options: + +- `argv` +- `stdout_limit` / `stderr_limit` as `std.Io.Limit` +- `reserve_amount` +- `cwd` +- `environ_map` +- `expand_arg0` +- `progress_node` +- `create_no_window` +- `disable_aslr` +- `timeout` + +## Spawn Child Process + +Use `std.process.spawn(io, options)`. `std.process.Child.init` is not the 0.16 pattern. + +```zig +var child = try std.process.spawn(io, .{ + .argv = &.{ "tool", "--flag" }, + .stdin = .ignore, + .stdout = .pipe, + .stderr = .pipe, + .cwd = .inherit, +}); +defer child.kill(io); + +const term = try child.wait(io); +_ = term; +``` + +`child.wait(io)` blocks until termination and cleans resources. `child.kill(io)` is uncancelable and idempotent after wait/kill. + +## Pipes + +Pipe fields are `std.Io.File` values when requested. + +```zig +var child = try std.process.spawn(io, .{ + .argv = &.{ "cat" }, + .stdin = .pipe, + .stdout = .pipe, + .stderr = .pipe, +}); +defer child.kill(io); + +var stdin_buf: [4096]u8 = undefined; +var stdin_writer = child.stdin.?.writer(io, &stdin_buf); +try stdin_writer.interface.writeAll("hello\n"); +try stdin_writer.interface.flush(); +child.stdin.?.close(io); +child.stdin = null; + +var stdout_buf: [4096]u8 = undefined; +var stdout_reader = child.stdout.?.reader(io, &stdout_buf); +const stdout = try stdout_reader.interface.allocRemaining(gpa, .limited(64 * 1024)); +defer gpa.free(stdout); + +const term = try child.wait(io); +_ = term; +``` + +For simultaneous stdout/stderr capture, prefer `std.process.run` or `std.Io.File.MultiReader` to avoid pipe deadlocks. + +## Working Directory + +Child cwd uses `std.process.Child.Cwd`: + +```zig +.cwd = .inherit +.cwd = .{ .path = "projects" } +.cwd = .{ .dir = some_io_dir } +``` + +Use `std.process.spawnPath(io, dir, options)` when `argv[0]` should be resolved relative to a directory as a file path. + +## Standard I/O Options + +`SpawnOptions.StdIo` values: + +- `.inherit` +- `.file` +- `.ignore` +- `.pipe` +- `.close` + +Example: + +```zig +var child = try std.process.spawn(io, .{ + .argv = &.{ "tool" }, + .stdin = .ignore, + .stdout = .pipe, + .stderr = .pipe, +}); +``` + +## Preopens + +WASI preopens moved to `std.process.Preopens` and are exposed by `std.process.Init`. + +```zig +const preopens = init.preopens; +_ = preopens; +``` + +## Memory Locking + +Memory locking/protection APIs moved under `std.process`: + +- `std.process.lockMemory` +- `std.process.unlockMemory` +- `std.process.lockMemoryAll` +- `std.process.unlockMemoryAll` + +Use them only for explicit platform/security needs. + +## Migration Map + +| Old pattern | Zig 0.16 pattern | +|-------------|------------------| +| `pub fn main() !void` plus global args/env | `pub fn main(init: std.process.Init) !void` | +| `std.process.Child.run(.{ ... })` | `std.process.run(gpa, io, .{ ... })` | +| `std.process.Child.init(argv, allocator)` | `std.process.spawn(io, .{ .argv = argv, ... })` | +| `child.spawn()` | spawn returns the child | +| `child.wait()` | `child.wait(io)` | +| `child.kill()` | `child.kill(io)` | +| `std.process.getCwd(...)` | `std.process.currentPath(io, ...)` | +| `std.process.getCwdAlloc(...)` | `std.process.currentPathAlloc(io, allocator)` | +| `error.EnvironmentVariableNotFound` | `error.EnvironmentVariableMissing` | + +## Review Checklist + +- Does the function already have `std.process.Init` or an `io` parameter? +- Are args/env taken from `init` instead of globals? +- Are child processes bounded with output limits or timeouts where appropriate? +- Are pipe reads/writes using `std.Io.File` readers/writers with `io`? +- Is `child.kill(io)` used in `defer` when early exits could leave a process alive? +- Is stdout/stderr capture safe from deadlock? diff --git a/references/std-random.md b/references/std-random.md new file mode 100644 index 0000000..9591286 --- /dev/null +++ b/references/std-random.md @@ -0,0 +1,414 @@ +# std.Random - Random Number Generation (Zig 0.16.0) + +Pseudo-random number generators (PRNGs), cryptographically secure random number generators (CSPRNGs), and utilities for generating random values of various types. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +## Zig 0.16 Entropy Rule + +System entropy moved under `std.Io`. + +```zig +var seed: [32]u8 = undefined; +io.random(&seed); + +const source: std.Random.IoSource = .{ .io = io }; +const rng = source.interface(); +``` + +Use `io.randomSecure(&bytes)` when fresh OS-backed secure entropy is required and errors must be surfaced. + +Old examples below may mention `std.crypto.random`; use the `std.Io` patterns above in new 0.16 code. + +## Quick Reference + +| Category | Types/Functions | +|----------|-----------------| +| Default PRNGs | `DefaultPrng` (Xoshiro256), `DefaultCsprng` (ChaCha) | +| Fast PRNGs | `Xoshiro256`, `Xoroshiro128`, `Pcg`, `Sfc64`, `RomuTrio`, `Isaac64` | +| CSPRNGs | `ChaCha`, `Ascon` | +| Utilities | `SplitMix64` (seeding helper) | +| Integer | `int`, `uintLessThan`, `uintAtMost`, `intRangeLessThan`, `intRangeAtMost` | +| Float | `float`, `floatNorm`, `floatExp` | +| Collections | `boolean`, `enumValue`, `shuffle`, `weightedIndex` | +| Bytes | `bytes` | + +## Choosing a PRNG + +``` +Need crypto security? +├─ Yes → ChaCha (DefaultCsprng) or Ascon +└─ No → Need speed? + ├─ Yes → Xoshiro256 (DefaultPrng), Sfc64, or RomuTrio + └─ No → Pcg (smaller state), Xoroshiro128 +``` + +| PRNG | State | Output | Use Case | +|------|-------|--------|----------| +| `Xoshiro256` | 256-bit | 64-bit | Default, fast, good quality | +| `Xoroshiro128` | 128-bit | 64-bit | Smaller state than Xoshiro256 | +| `Pcg` | 128-bit | 32-bit | Compact, statistically excellent | +| `Sfc64` | 256-bit | 64-bit | Very fast | +| `RomuTrio` | 192-bit | 64-bit | Fast, small code size | +| `Isaac64` | 8KB | 64-bit | Cryptographic-ish (prefer ChaCha) | +| `ChaCha` | 512-bit | stream | CSPRNG, forward secure | +| `Ascon` | 320-bit | stream | CSPRNG, lightweight | + +## Basic Usage + +### Quick Start with DefaultPrng + +```zig +const std = @import("std"); + +pub fn main() void { + // Initialize with a seed + var prng = std.Random.DefaultPrng.init(12345); + const random = prng.random(); + + // Generate random values + const n = random.int(u32); // 0 to maxInt(u32) + const dice = random.intRangeLessThan(u8, 1, 7); // 1-6 + const coin = random.boolean(); + const prob = random.float(f32); // [0, 1) +} +``` + +### Cryptographically Secure Random + +```zig +const std = @import("std"); + +pub fn main() void { + // Use std.crypto.random for system entropy + const secure = std.crypto.random; + + var key: [32]u8 = undefined; + secure.bytes(&key); // fill with cryptographically secure random bytes + + const token = secure.int(u64); +} +``` + +### Seeding from System Entropy + +```zig +var seed: u64 = undefined; +std.crypto.random.bytes(std.mem.asBytes(&seed)); +var prng = std.Random.DefaultPrng.init(seed); +``` + +## PRNG Initialization + +### Xoshiro256 (Default) + +```zig +var prng = std.Random.Xoshiro256.init(seed); +const random = prng.random(); + +// Jump ahead 2^128 steps (for parallel streams) +prng.jump(); +``` + +### ChaCha (CSPRNG) + +```zig +// Requires 32-byte secret seed +var secret_seed: [std.Random.ChaCha.secret_seed_length]u8 = undefined; +std.crypto.random.bytes(&secret_seed); + +var csprng = std.Random.ChaCha.init(secret_seed); +const random = csprng.random(); + +// Add entropy to refresh internal state +csprng.addEntropy(&additional_entropy); +``` + +### Pcg + +```zig +var prng = std.Random.Pcg.init(seed); +const random = prng.random(); +``` + +### Other PRNGs + +```zig +// All follow the same pattern +var xoro = std.Random.Xoroshiro128.init(seed); +var sfc = std.Random.Sfc64.init(seed); +var romu = std.Random.RomuTrio.init(seed); +var isaac = std.Random.Isaac64.init(seed); +var ascon = std.Random.Ascon.init(secret_seed); +``` + +## Generating Random Values + +### Integers + +```zig +const random = prng.random(); + +// Full range of type +const u8_val = random.int(u8); // 0 to 255 +const i32_val = random.int(i32); // minInt to maxInt + +// Less than upper bound: [0, less_than) +const index = random.uintLessThan(usize, array.len); +const digit = random.uintLessThan(u8, 10); // 0-9 + +// At most (inclusive): [0, at_most] +const die = random.uintAtMost(u8, 5); // 0-5 + +// Range (exclusive upper): [at_least, less_than) +const temp = random.intRangeLessThan(i16, -40, 50); + +// Range (inclusive): [at_least, at_most] +const year = random.intRangeAtMost(u16, 2000, 2024); +``` + +### Biased Variants (Constant Time) + +For timing-sensitive code where bias is acceptable: + +```zig +// Slightly biased but constant-time +const n = random.uintLessThanBiased(u32, 100); +const m = random.uintAtMostBiased(u32, 99); +const r = random.intRangeLessThanBiased(i32, -50, 50); +const s = random.intRangeAtMostBiased(i32, -50, 50); +``` + +### Floating Point + +```zig +// Uniform in [0, 1) +const uniform: f32 = random.float(f32); +const uniform64: f64 = random.float(f64); + +// Scale to range [a, b) +const scaled = a + (b - a) * random.float(f64); + +// Normal distribution (mean=0, stddev=1) +const normal: f64 = random.floatNorm(f64); +// Custom mean/stddev: value * stddev + mean +const custom_normal = random.floatNorm(f64) * 10.0 + 50.0; + +// Exponential distribution (rate=1) +const exponential: f64 = random.floatExp(f64); +// Custom rate: value / rate +const custom_exp = random.floatExp(f64) / 0.5; +``` + +### Boolean + +```zig +const coin_flip = random.boolean(); + +if (random.boolean()) { + // 50% chance +} +``` + +### Enum Values + +```zig +const Direction = enum { north, south, east, west }; + +// Random enum value (evenly distributed) +const dir = random.enumValue(Direction); + +// With explicit index type for cross-platform consistency +const dir2 = random.enumValueWithIndex(Direction, u32); +``` + +### Bytes + +```zig +var buffer: [32]u8 = undefined; +random.bytes(&buffer); + +// Generate a random string +var id: [16]u8 = undefined; +random.bytes(&id); +const hex = std.fmt.fmtSliceHexLower(&id); +``` + +## Collections + +### Shuffle + +```zig +var items = [_]u32{ 1, 2, 3, 4, 5 }; +random.shuffle(u32, &items); + +// With explicit index type for reproducibility +random.shuffleWithIndex(u32, &items, u32); +``` + +### Weighted Selection + +```zig +const weights = [_]f32{ 0.5, 0.3, 0.2 }; // 50%, 30%, 20% +const choice = random.weightedIndex(f32, &weights); + +// With integer weights +const int_weights = [_]u32{ 5, 3, 2 }; +const int_choice = random.weightedIndex(u32, &int_weights); +``` + +### Random Element from Slice + +```zig +fn randomElement(comptime T: type, random: std.Random, slice: []const T) T { + const index = random.uintLessThan(usize, slice.len); + return slice[index]; +} + +const colors = [_][]const u8{ "red", "green", "blue" }; +const color = randomElement([]const u8, random, &colors); +``` + +### Random Sample (Without Replacement) + +```zig +fn sample(comptime T: type, random: std.Random, source: []const T, dest: []T) void { + // Fisher-Yates partial shuffle + var indices: [source.len]usize = undefined; + for (&indices, 0..) |*idx, i| idx.* = i; + + for (dest, 0..) |*d, i| { + const j = random.intRangeLessThan(usize, i, source.len); + std.mem.swap(usize, &indices[i], &indices[j]); + d.* = source[indices[i]]; + } +} +``` + +## Common Patterns + +### Reproducible Sequences + +```zig +// Same seed = same sequence +const seed: u64 = 42; + +var prng1 = std.Random.DefaultPrng.init(seed); +var prng2 = std.Random.DefaultPrng.init(seed); + +std.debug.assert(prng1.random().int(u64) == prng2.random().int(u64)); +``` + +### Thread-Local PRNGs + +```zig +threadlocal var tls_prng: ?std.Random.DefaultPrng = null; + +fn getThreadRandom() std.Random { + if (tls_prng == null) { + var seed: u64 = undefined; + std.crypto.random.bytes(std.mem.asBytes(&seed)); + tls_prng = std.Random.DefaultPrng.init(seed); + } + return tls_prng.?.random(); +} +``` + +### Parallel Streams with Jump + +```zig +fn createParallelStreams(base_seed: u64, n: usize, allocator: std.mem.Allocator) ![]std.Random.Xoshiro256 { + const prngs = try allocator.alloc(std.Random.Xoshiro256, n); + + prngs[0] = std.Random.Xoshiro256.init(base_seed); + for (prngs[1..], 1..) |*prng, i| { + prng.* = prngs[i - 1]; + prng.jump(); // advance 2^128 steps + } + + return prngs; +} +``` + +### Monte Carlo Simulation + +```zig +fn estimatePi(random: std.Random, samples: usize) f64 { + var inside: usize = 0; + for (0..samples) |_| { + const x = random.float(f64); + const y = random.float(f64); + if (x * x + y * y <= 1.0) inside += 1; + } + return 4.0 * @as(f64, @floatFromInt(inside)) / @as(f64, @floatFromInt(samples)); +} +``` + +### Random Password Generator + +```zig +const charset = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*"; + +fn generatePassword(random: std.Random, buf: []u8) void { + for (buf) |*c| { + c.* = charset[random.uintLessThan(usize, charset.len)]; + } +} + +// Usage +var password: [16]u8 = undefined; +generatePassword(std.crypto.random, &password); +``` + +### Gaussian Random with Box-Muller + +The built-in `floatNorm` uses ziggurat algorithm. For explicit Box-Muller: + +```zig +fn boxMullerPair(random: std.Random) struct { f64, f64 } { + const u1 = 1.0 - random.float(f64); // (0, 1] + const u2 = random.float(f64); // [0, 1) + + const r = @sqrt(-2.0 * @log(u1)); + const theta = 2.0 * std.math.pi * u2; + + return .{ r * @cos(theta), r * @sin(theta) }; +} +``` + +## Custom PRNG Implementation + +Implement a custom PRNG by providing a `fill` function: + +```zig +const MyPrng = struct { + state: u64, + + pub fn init(seed: u64) MyPrng { + return .{ .state = seed }; + } + + pub fn random(self: *MyPrng) std.Random { + return std.Random.init(self, fill); + } + + fn fill(self: *MyPrng, buf: []u8) void { + for (buf) |*b| { + // Simple LCG (not for production!) + self.state = self.state *% 6364136223846793005 +% 1; + b.* = @truncate(self.state >> 56); + } + } +}; +``` + +## Notes + +- `DefaultPrng` is `Xoshiro256` - fast, high quality, not cryptographic +- `DefaultCsprng` is `ChaCha` - cryptographically secure with forward secrecy +- For crypto: use `std.crypto.random` which provides system entropy +- `uintLessThan`/`intRangeLessThan` may reject values (not constant-time) +- Use biased variants (`*Biased`) for timing-sensitive applications +- `jump()` on Xoshiro256 advances 2^128 steps for parallel streams +- `float()` returns values in [0, 1) covering all representable values +- `floatNorm()` and `floatExp()` use efficient ziggurat algorithm diff --git a/references/std-segmented-list.md b/references/std-segmented-list.md new file mode 100644 index 0000000..d70e4cb --- /dev/null +++ b/references/std-segmented-list.md @@ -0,0 +1,142 @@ +# std.SegmentedList - removed in Zig 0.16.0 + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +`std.SegmentedList` was removed in Zig 0.16. Do not use the API below in new code. Choose a project-owned segmented container, an arena-backed structure, an index-based `std.ArrayList`, or another stable-handle design depending on the ownership/lifetime requirement. + +A dynamic list where element pointers remain stable across growth. Unlike ArrayList, appending never invalidates existing pointers. Elements are stored in exponentially-sized segments. + +## When to Use + +- Need stable pointers to elements (pointers survive append) +- Arena allocator backing (avoids wasted memory on reallocation) +- Non-copyable element types +- Stack-like access patterns (append/pop) + +## Trade-offs + +- Elements not contiguous (most are, but not guaranteed) +- O(log n) random access (vs O(1) for ArrayList) +- Higher per-element overhead + +## Initialization + +```zig +// Without preallocation +var list = std.SegmentedList(i32, 0){}; +defer list.deinit(allocator); + +// With preallocation (must be power of 2) +// First N elements stored inline, no allocation needed +var list = std.SegmentedList(i32, 16){}; +defer list.deinit(allocator); +``` + +## Basic Operations + +```zig +// Append (pointer remains valid forever) +try list.append(allocator, 42); +try list.appendSlice(allocator, &[_]i32{ 1, 2, 3 }); + +// Get pointer to element (STABLE across appends) +const ptr = list.at(0); // *i32 +ptr.* = 100; // modify in place + +// Add and get pointer in one operation +const new_ptr = try list.addOne(allocator); +new_ptr.* = 42; + +// Pop +const last = list.pop(); // ?i32 + +// Length +const n = list.count(); +// Or: list.len +``` + +## Iteration + +```zig +// Forward iteration with mutable access +var it = list.iterator(0); // start at index 0 +while (it.next()) |ptr| { + ptr.* += 1; // modify in place +} + +// Const iteration +var it = list.constIterator(0); +while (it.next()) |ptr| { + std.debug.print("{}\n", .{ptr.*}); +} + +// Bidirectional +while (it.prev()) |ptr| { + // ... +} + +// Peek without advancing +if (it.peek()) |ptr| { + // ... +} + +// Jump to index +it.set(50); +``` + +## Capacity Management + +```zig +// Grow capacity +try list.growCapacity(allocator, 100); +try list.setCapacity(allocator, 100); // grow or shrink + +// Shrink +list.shrinkCapacity(allocator, 50); // may fail silently +list.shrinkRetainingCapacity(new_len); + +// Clear +list.clearRetainingCapacity(); +list.clearAndFree(allocator); +``` + +## Copy to Contiguous Slice + +```zig +var dest: [100]i32 = undefined; +list.writeToSlice(&dest, 0); // copy from index 0 + +// Copy subset +list.writeToSlice(dest[50..], 50); // copy starting at index 50 +``` + +## Memory Layout + +Segments grow exponentially: +``` +prealloc=0: shelf 0: 1 element + shelf 1: 2 elements + shelf 2: 4 elements + ... + +prealloc=4: prealloc: 4 elements (inline) + shelf 0: 8 elements + shelf 1: 16 elements + ... +``` + +## Common Pattern: Object Pool with Stable References + +```zig +const Object = struct { + data: [1024]u8, + next: ?*Object, +}; + +var pool = std.SegmentedList(Object, 64){}; + +// Create objects - pointers remain valid +const obj1 = try pool.addOne(allocator); +const obj2 = try pool.addOne(allocator); +obj1.next = obj2; // safe: obj2 pointer won't change +``` diff --git a/references/std-simd.md b/references/std-simd.md new file mode 100644 index 0000000..2c53f9f --- /dev/null +++ b/references/std-simd.md @@ -0,0 +1,586 @@ +# std.simd + +SIMD (Single Instruction, Multiple Data) utilities for parallel processing of multiple elements at once. Provides convenience functions for vector manipulation, pattern generation, searching, and parallel computation. + +## Quick Reference + +| Category | Functions | +|----------|-----------| +| Vector Length | `suggestVectorLength`, `suggestVectorLengthForCpu`, `VectorIndex`, `VectorCount` | +| Pattern Generation | `iota`, `repeat`, `join`, `interlace`, `deinterlace` | +| Extraction/Shifting | `extract`, `mergeShift`, `shiftElementsLeft`, `shiftElementsRight` | +| Rotation/Reversal | `rotateElementsLeft`, `rotateElementsRight`, `reverseOrder` | +| Searching | `firstTrue`, `lastTrue`, `countTrues`, `firstIndexOfValue`, `lastIndexOfValue`, `countElementsWithValue` | +| Parallel Scans | `prefixScan`, `prefixScanWithFunc` | + +## Core Concepts + +### Vector Types + +Zig vectors are first-class types declared with `@Vector(len, T)`. Element types can be booleans, integers, floats, or pointers: + +```zig +const Vec4 = @Vector(4, f64); // 4 f64 values +const Vec8i = @Vector(8, i32); // 8 i32 values +const Vec16b = @Vector(16, bool); // 16 booleans +const Vec4p = @Vector(4, *u8); // 4 pointers +``` + +**Vector length limits:** Zig supports lengths up to 2^32-1, but powers of two from 2-64 are typical. Excessively long vectors (e.g., 2^20) may crash the compiler. + +**Compilation behavior:** Vectors shorter than the native SIMD size compile to single instructions. Longer vectors compile to multiple SIMD instructions. Without SIMD support, operations fall back to element-by-element execution. + +### Built-in Operations + +Vectors support arithmetic, comparisons, and builtins directly (all element-wise): + +```zig +const a: @Vector(4, f32) = .{ 1.0, 2.0, 3.0, 4.0 }; +const b: @Vector(4, f32) = .{ 5.0, 6.0, 7.0, 8.0 }; + +// Arithmetic (element-wise) +const sum = a + b; // { 6.0, 8.0, 10.0, 12.0 } +const prod = a * b; // { 5.0, 12.0, 21.0, 32.0 } + +// Comparison (returns bool vector) +const mask = a < b; // { true, true, true, true } + +// Broadcast scalar to all lanes +const twos: @Vector(4, f32) = @splat(2.0); // { 2.0, 2.0, 2.0, 2.0 } + +// Math builtins (hardware-accelerated when available) +const sines = @sin(a); +const sqrts = @sqrt(a); + +// Horizontal reduction +const total = @reduce(.Add, a); // 10.0 +const max_val = @reduce(.Max, a); // 4.0 +``` + +**Important:** `and` and `or` keywords do NOT work on bool vectors (they affect control flow). Use `&` and `|` bitwise operators, or `@select` instead. + +### Vector-Compatible Builtins + +These builtins work element-wise on vectors: + +| Category | Builtins | +|----------|----------| +| Math | `@sqrt`, `@sin`, `@cos`, `@exp`, `@exp2`, `@log`, `@log2`, `@log10` | +| Rounding | `@floor`, `@ceil`, `@trunc`, `@round` | +| Arithmetic | `@abs`, `@min`, `@max`, `@mulAdd`, `@divFloor`, `@divTrunc`, `@mod`, `@rem` | +| Bit ops | `@clz`, `@ctz`, `@popCount`, `@byteSwap`, `@bitReverse` | +| Overflow | `@addWithOverflow`, `@subWithOverflow`, `@mulWithOverflow`, `@shlWithOverflow` | + +### Array/Slice Conversion + +```zig +// Array to vector (automatic) +const arr: [4]f32 = .{ 1.1, 3.2, 4.5, 5.6 }; +const vec: @Vector(4, f32) = arr; + +// Vector to array (automatic) +const arr2: [4]f32 = vec; + +// Slice with comptime-known length to vector +const vec2: @Vector(2, f32) = arr[1..3].*; + +// Runtime offset with comptime length +const slice: []const f32 = &arr; +var offset: usize = 1; +const vec3: @Vector(2, f32) = slice[offset..][0..2].*; +``` + +### Vector Destructuring + +Vectors can be destructured like tuples: + +```zig +const vec: @Vector(4, f32) = .{ 1.0, 2.0, 3.0, 4.0 }; +const a, const b, _, _ = vec; // a=1.0, b=2.0, ignore rest + +// Useful for SIMD unpacking (emulating punpckldq) +pub fn unpack(x: @Vector(4, f32), y: @Vector(4, f32)) @Vector(4, f32) { + const a, const c, _, _ = x; + const b, const d, _, _ = y; + return .{ a, b, c, d }; +} +``` + +### @shuffle - Rearrange Elements + +Rearrange elements from one or two vectors using an index mask: + +```zig +const a: @Vector(7, u8) = .{ 'o', 'l', 'h', 'e', 'r', 'z', 'w' }; +const b: @Vector(4, u8) = .{ 'w', 'd', '!', 'x' }; + +// Shuffle within single vector (pass undefined as second) +const mask1: @Vector(5, i32) = .{ 2, 3, 1, 1, 0 }; +const hello: @Vector(5, u8) = @shuffle(u8, a, undefined, mask1); +// "hello" + +// Combine two vectors (negative indices select from b: -1=b[0], -2=b[1], etc.) +const mask2: @Vector(6, i32) = .{ -1, 0, 4, 1, -2, -3 }; +const world: @Vector(6, u8) = @shuffle(u8, a, b, mask2); +// "world!" +``` + +### @select - Conditional Selection + +Select elements from two vectors based on a bool mask: + +```zig +const a: @Vector(4, f32) = .{ 1.0, 2.0, 3.0, 4.0 }; +const b: @Vector(4, f32) = .{ 5.0, 6.0, 7.0, 8.0 }; +const mask: @Vector(4, bool) = .{ true, false, true, false }; + +const result = @select(f32, mask, a, b); +// { 1.0, 6.0, 3.0, 8.0 } (a where true, b where false) +``` + +### @reduce - Horizontal Reduction + +Reduce a vector to a scalar using an operation: + +```zig +const vec: @Vector(4, i32) = .{ 1, 2, 3, 4 }; + +// Arithmetic reductions +const sum = @reduce(.Add, vec); // 10 +const prod = @reduce(.Mul, vec); // 24 +const min_val = @reduce(.Min, vec); // 1 +const max_val = @reduce(.Max, vec); // 4 + +// Bitwise reductions +const and_val = @reduce(.And, vec); // 0 +const or_val = @reduce(.Or, vec); // 7 +const xor_val = @reduce(.Xor, vec); // 4 + +// Boolean reductions (for bool vectors) +const mask: @Vector(4, bool) = .{ true, true, false, true }; +const all_true = @reduce(.And, mask); // false +const any_true = @reduce(.Or, mask); // true +``` + +**Available operations by type:** +- **Integers:** All operations (Add, Mul, Min, Max, And, Or, Xor) +- **Floats:** Add, Mul, Min, Max +- **Booleans:** And, Or, Xor + +## Optimal Vector Length + +### suggestVectorLength + +Query the optimal vector length for the current CPU: + +```zig +const std = @import("std"); + +// Get optimal lane count for this type on current hardware +const len = std.simd.suggestVectorLength(f32) orelse 4; + +// Use comptime to create vector type +const Vec = @Vector(len, f32); +``` + +Returns `null` if scalars are recommended (no SIMD benefit). + +### suggestVectorLengthForCpu + +Query optimal length for a specific CPU target: + +```zig +const len = std.simd.suggestVectorLengthForCpu(f64, target_cpu) orelse 2; +``` + +**Architecture support:** +- **x86**: SSE (128-bit), AVX2 (256-bit), AVX-512 (512-bit) +- **ARM**: NEON (128-bit) +- **AArch64**: NEON (128-bit), SVE (128-bit default) +- **RISC-V**: V extension (32-bit to 65536-bit via zvl* features) +- **WebAssembly**: simd128 (128-bit) +- **PowerPC**: AltiVec (128-bit) + +### Vector Index/Count Types + +Get the smallest integer type for indexing or counting: + +```zig +const Vec8 = @Vector(8, u32); + +// Type that can index any element (0-7) +const Idx = std.simd.VectorIndex(Vec8); // u3 + +// Type that can hold the count (0-8) +const Cnt = std.simd.VectorCount(Vec8); // u4 +``` + +## Pattern Generation + +### iota - Sequential Values + +Generate a vector of sequential values starting from 0: + +```zig +const indices = std.simd.iota(i32, 8); +// { 0, 1, 2, 3, 4, 5, 6, 7 } + +const floats = std.simd.iota(f32, 4); +// { 0.0, 1.0, 2.0, 3.0 } +``` + +### repeat - Repeating Pattern + +Repeat a smaller vector/array to fill a larger one: + +```zig +const pattern = [_]u32{ 1, 2, 3 }; +const repeated = std.simd.repeat(8, pattern); +// { 1, 2, 3, 1, 2, 3, 1, 2 } + +const vec: @Vector(2, f32) = .{ 10.0, 20.0 }; +const tiled = std.simd.repeat(6, vec); +// { 10.0, 20.0, 10.0, 20.0, 10.0, 20.0 } +``` + +### join - Concatenate Vectors + +Concatenate two vectors end-to-end: + +```zig +const a: @Vector(4, u32) = .{ 10, 20, 30, 40 }; +const b: @Vector(4, u32) = .{ 55, 66, 77, 88 }; +const joined = std.simd.join(a, b); +// { 10, 20, 30, 40, 55, 66, 77, 88 } +``` + +### interlace - Interleave Multiple Vectors + +Alternate elements from multiple vectors: + +```zig +const a: @Vector(4, u32) = .{ 10, 20, 30, 40 }; +const b: @Vector(4, u32) = .{ 55, 66, 77, 88 }; +const interleaved = std.simd.interlace(.{ a, b }); +// { 10, 55, 20, 66, 30, 77, 40, 88 } + +// Works with more than 2 vectors +const v1: @Vector(2, u8) = .{ 0, 1 }; +const v2: @Vector(2, u8) = .{ 2, 3 }; +const v3: @Vector(2, u8) = .{ 4, 5 }; +const result = std.simd.interlace(.{ v1, v2, v3 }); +// { 0, 2, 4, 1, 3, 5 } +``` + +**Note:** Does not work on MIPS (compile error). + +### deinterlace - Split Interleaved Data + +Reverse of interlace - split into separate vectors: + +```zig +const interleaved: @Vector(8, u32) = .{ 10, 55, 20, 66, 30, 77, 40, 88 }; +const result = std.simd.deinterlace(2, interleaved); +// result[0] = { 10, 20, 30, 40 } +// result[1] = { 55, 66, 77, 88 } +``` + +## Element Extraction and Shifting + +### extract - Get Subvector + +Extract a contiguous slice of elements: + +```zig +const vec: @Vector(8, u32) = .{ 0, 1, 2, 3, 4, 5, 6, 7 }; +const slice = std.simd.extract(vec, 2, 3); +// { 2, 3, 4 } +``` + +### shiftElementsLeft / shiftElementsRight + +Shift elements, filling with a value: + +```zig +const vec: @Vector(4, u32) = .{ 10, 20, 30, 40 }; + +// Shift left (toward lower indices), fill from right +const left = std.simd.shiftElementsLeft(vec, 2, 999); +// { 30, 40, 999, 999 } + +// Shift right (toward higher indices), fill from left +const right = std.simd.shiftElementsRight(vec, 2, 999); +// { 999, 999, 10, 20 } +``` + +### rotateElementsLeft / rotateElementsRight + +Circular rotation (elements wrap around): + +```zig +const vec: @Vector(4, u32) = .{ 10, 20, 30, 40 }; + +const rotl = std.simd.rotateElementsLeft(vec, 1); +// { 20, 30, 40, 10 } + +const rotr = std.simd.rotateElementsRight(vec, 1); +// { 40, 10, 20, 30 } +``` + +### reverseOrder + +Reverse element order: + +```zig +const vec: @Vector(4, u32) = .{ 10, 20, 30, 40 }; +const reversed = std.simd.reverseOrder(vec); +// { 40, 30, 20, 10 } +``` + +### mergeShift + +Combine two vectors and extract a shifted window: + +```zig +const a: @Vector(4, u32) = .{ 1, 2, 3, 4 }; +const b: @Vector(4, u32) = .{ 5, 6, 7, 8 }; +const merged = std.simd.mergeShift(a, b, 2); +// Joins to { 1, 2, 3, 4, 5, 6, 7, 8 }, extracts starting at index 2 +// { 3, 4, 5, 6 } +``` + +## Searching + +### firstTrue / lastTrue + +Find first/last true element in a boolean vector: + +```zig +const mask: @Vector(8, bool) = .{ false, false, true, false, true, false, false, false }; + +const first = std.simd.firstTrue(mask); // 2 +const last = std.simd.lastTrue(mask); // 4 + +// Returns null if no true values +const all_false: @Vector(4, bool) = .{ false, false, false, false }; +const none = std.simd.firstTrue(all_false); // null +``` + +### countTrues + +Count true elements: + +```zig +const mask: @Vector(8, bool) = .{ true, false, true, false, true, false, true, false }; +const count = std.simd.countTrues(mask); // 4 +``` + +### firstIndexOfValue / lastIndexOfValue + +Find first/last occurrence of a value: + +```zig +const vec: @Vector(8, u32) = .{ 6, 4, 7, 4, 4, 2, 3, 7 }; + +const first_4 = std.simd.firstIndexOfValue(vec, 4); // 1 +const last_4 = std.simd.lastIndexOfValue(vec, 4); // 4 +const not_found = std.simd.lastIndexOfValue(vec, 99); // null +``` + +### countElementsWithValue + +Count occurrences of a value: + +```zig +const vec: @Vector(8, u32) = .{ 6, 4, 7, 4, 4, 2, 3, 7 }; +const count = std.simd.countElementsWithValue(vec, 4); // 3 +``` + +## Parallel Prefix Scans + +### prefixScan + +Compute cumulative operations across vector lanes: + +```zig +const vec: @Vector(4, i32) = .{ 11, 23, 9, -21 }; + +// Running sum +const sums = std.simd.prefixScan(.Add, 1, vec); +// { 11, 34, 43, 22 } + +// Running product +const prods = std.simd.prefixScan(.Mul, 1, vec); +// { 11, 253, 2277, -47817 } + +// Running min +const mins = std.simd.prefixScan(.Min, 1, vec); +// { 11, 11, 9, -21 } + +// Running max +const maxs = std.simd.prefixScan(.Max, 1, vec); +// { 11, 23, 23, 23 } + +// Bitwise operations +const ands = std.simd.prefixScan(.And, 1, vec); +const ors = std.simd.prefixScan(.Or, 1, vec); +const xors = std.simd.prefixScan(.Xor, 1, vec); +``` + +**Hop parameter:** Controls which elements combine. `hop=2` combines every other element. + +```zig +const vec: @Vector(4, i32) = .{ 11, 23, 9, -21 }; +const skip = std.simd.prefixScan(.Add, 2, vec); +// { 11, 23, 20, 2 } (11+9=20, 23+(-21)=2) + +// Negative hop scans in reverse +const rev = std.simd.prefixScan(.Add, -1, vec); +// { 22, 11, -12, -21 } +``` + +**Note:** Does not work on MIPS (compile error). + +### prefixScanWithFunc + +Use a custom associative function: + +```zig +fn myMax(a: @Vector(4, f32), b: @Vector(4, f32)) @Vector(4, f32) { + return @max(a, b); +} + +const vec: @Vector(4, f32) = .{ 1.0, 5.0, 2.0, 8.0 }; +const result = std.simd.prefixScanWithFunc(1, vec, void, myMax, -std.math.inf(f32)); +// { 1.0, 5.0, 5.0, 8.0 } +``` + +The identity value must satisfy: `func(x, identity) == x`. + +## Practical Patterns + +### Branchless Selection + +Replace `if` statements with vector selection: + +```zig +// Scalar (branching) +fn clampScalar(x: f32, lo: f32, hi: f32) f32 { + if (x < lo) return lo; + if (x > hi) return hi; + return x; +} + +// Vector (branchless) +fn clampSimd(x: @Vector(4, f32), lo: f32, hi: f32) @Vector(4, f32) { + const lo_vec: @Vector(4, f32) = @splat(lo); + const hi_vec: @Vector(4, f32) = @splat(hi); + return @min(@max(x, lo_vec), hi_vec); +} +``` + +### Convergence Loops + +Process lanes that converge at different rates: + +```zig +fn iterateUntilConverged(vec: @Vector(4, f64), tolerance: f64) @Vector(4, f64) { + const tol_vec: @Vector(4, f64) = @splat(tolerance); + var current = vec; + var converged: @Vector(4, bool) = @splat(false); + + while (!@reduce(.And, converged)) { + const next = computeNext(current); + const delta = @abs(next - current); + converged = delta <= tol_vec; + current = next; + } + return current; +} +``` + +### Time-Batched Processing + +Process multiple time points for one object: + +```zig +const Vec4 = @Vector(4, f64); + +fn propagateV4(state: *const State, times: [4]f64) [4]Result { + const time_vec: Vec4 = times; + // Process all 4 times simultaneously + const positions = computePositions(state, time_vec); + const velocities = computeVelocities(state, time_vec); + // ... +} +``` + +### Object-Batched Processing (Struct of Arrays) + +Process multiple objects at the same time point: + +```zig +// Struct of Arrays layout for 4 objects +const ObjectsV4 = struct { + x: @Vector(4, f64), + y: @Vector(4, f64), + vx: @Vector(4, f64), + vy: @Vector(4, f64), +}; + +fn updatePositions(objs: *ObjectsV4, dt: f64) void { + const dt_vec: @Vector(4, f64) = @splat(dt); + objs.x += objs.vx * dt_vec; + objs.y += objs.vy * dt_vec; +} +``` + +### Custom atan2 Approximation + +LLVM lacks vectorized `atan2`. Implement polynomial approximation: + +```zig +fn atan2Simd(y: @Vector(4, f64), x: @Vector(4, f64)) @Vector(4, f64) { + const abs_x = @abs(x); + const abs_y = @abs(y); + const max_xy = @max(abs_x, abs_y); + const min_xy = @min(abs_x, abs_y); + const epsilon: @Vector(4, f64) = @splat(1.0e-30); + const t = min_xy / @max(max_xy, epsilon); + + // Polynomial approximation (Horner's method) + const t2 = t * t; + var atan_t = @as(@Vector(4, f64), @splat(0.0028662257)); + atan_t = atan_t * t2 + @as(@Vector(4, f64), @splat(-0.0161657367)); + atan_t = atan_t * t2 + @as(@Vector(4, f64), @splat(0.0429096138)); + atan_t = atan_t * t2 + @as(@Vector(4, f64), @splat(-0.0752896400)); + atan_t = atan_t * t2 + @as(@Vector(4, f64), @splat(0.1065626393)); + atan_t = atan_t * t2 + @as(@Vector(4, f64), @splat(-0.1420889944)); + atan_t = atan_t * t2 + @as(@Vector(4, f64), @splat(0.1999355085)); + atan_t = atan_t * t2 + @as(@Vector(4, f64), @splat(-0.3333314528)); + atan_t = atan_t * t2 + @as(@Vector(4, f64), @splat(1.0)); + atan_t = atan_t * t; + + // Quadrant correction + const half_pi: @Vector(4, f64) = @splat(std.math.pi / 2.0); + const pi: @Vector(4, f64) = @splat(std.math.pi); + const swap_mask = abs_y > abs_x; + atan_t = @select(f64, swap_mask, half_pi - atan_t, atan_t); + const x_neg = x < @as(@Vector(4, f64), @splat(0.0)); + atan_t = @select(f64, x_neg, pi - atan_t, atan_t); + const y_neg = y < @as(@Vector(4, f64), @splat(0.0)); + return @select(f64, y_neg, -atan_t, atan_t); +} +``` + +## Performance Notes + +- **Optimal vector size:** Use `suggestVectorLength` for portable code; don't hardcode lane counts. Powers of two (2-64) are most efficient +- **Compilation:** Short vectors → single SIMD instruction; long vectors → multiple instructions; no SIMD → scalar fallback +- **Alignment:** Vectors are automatically aligned; use `@alignCast` when loading from byte pointers +- **Branching:** Replace scalar branches with `@select` for branchless SIMD code. `and`/`or` keywords don't work on bool vectors +- **Reductions:** `@reduce` operations break SIMD parallelism; minimize their use in hot paths +- **Memory layout:** Prefer Struct-of-Arrays over Array-of-Structs for better vectorization +- **Cache tiling:** For large datasets, process in cache-sized chunks (e.g., 64 elements) to maintain data locality +- **Fused operations:** Use `@mulAdd(a, b, c)` for `(a * b) + c` - rounds once, more accurate +- **MIPS limitation:** `interlace` and `prefixScan` don't work on MIPS architecture diff --git a/references/std-sort.md b/references/std-sort.md new file mode 100644 index 0000000..718578a --- /dev/null +++ b/references/std-sort.md @@ -0,0 +1,234 @@ +# std.sort + +Sorting algorithms and binary search utilities. All sorts are in-place and require no allocator. + +## Quick Reference + +| Function | Stable | Complexity | When to Use | +|----------|--------|------------|-------------| +| `block` | Yes | O(n log n) | Default choice when stability matters | +| `pdq` | No | O(n log n) | Default choice when stability doesn't matter | +| `insertion` | Yes | O(n²) | Small arrays (<20), nearly sorted data | +| `heap` | No | O(n log n) | Guaranteed worst-case, no recursion | + +## Comparator Functions + +All sort functions take a `lessThan` comparator: + +```zig +const std = @import("std"); + +// Simple comparator (ascending) +fn lessThan(_: void, a: i32, b: i32) bool { + return a < b; +} + +// Use built-in generators for common cases +const asc_i32 = std.sort.asc(i32); // ascending +const desc_i32 = std.sort.desc(i32); // descending +``` + +## Basic Sorting + +```zig +var items = [_]i32{ 5, 2, 8, 1, 9 }; + +// Unstable sort (fastest general-purpose) +std.sort.pdq(i32, &items, {}, std.sort.asc(i32)); +// items = [1, 2, 5, 8, 9] + +// Stable sort (preserves order of equal elements) +std.sort.block(i32, &items, {}, std.sort.asc(i32)); + +// Descending order +std.sort.pdq(i32, &items, {}, std.sort.desc(i32)); +// items = [9, 8, 5, 2, 1] +``` + +## Sorting with Context + +Pass external data to the comparator: + +```zig +const scores = [_]u32{ 50, 30, 80, 20 }; +var indices = [_]usize{ 0, 1, 2, 3 }; + +fn compareByScore(scores_ctx: []const u32, a: usize, b: usize) bool { + return scores_ctx[a] < scores_ctx[b]; +} + +std.sort.pdq(usize, &indices, &scores, compareByScore); +// indices = [3, 1, 0, 2] (sorted by score: 20, 30, 50, 80) +``` + +## Sorting Structs + +```zig +const Person = struct { + name: []const u8, + age: u32, +}; + +fn byAge(_: void, a: Person, b: Person) bool { + return a.age < b.age; +} + +var people = [_]Person{ + .{ .name = "Alice", .age = 30 }, + .{ .name = "Bob", .age = 25 }, + .{ .name = "Carol", .age = 35 }, +}; + +std.sort.pdq(Person, &people, {}, byAge); +// Sorted: Bob (25), Alice (30), Carol (35) +``` + +## Check if Sorted + +```zig +const items = [_]i32{ 1, 2, 3, 4, 5 }; +const sorted = std.sort.isSorted(i32, &items, {}, std.sort.asc(i32)); +// true +``` + +## Binary Search + +Find element in sorted array. Comparator returns `Order` (.lt, .eq, .gt): + +```zig +fn order(target: i32, item: i32) std.math.Order { + return std.math.order(target, item); +} + +const items = [_]i32{ 1, 3, 5, 7, 9 }; + +// Find exact match +const idx = std.sort.binarySearch(i32, &items, @as(i32, 5), order); +// ?usize = 2 + +// Not found +const missing = std.sort.binarySearch(i32, &items, @as(i32, 4), order); +// null +``` + +## Lower/Upper Bound + +Find insertion points for sorted arrays: + +```zig +fn order(target: i32, item: i32) std.math.Order { + return std.math.order(target, item); +} + +const items = [_]i32{ 1, 3, 5, 5, 5, 7, 9 }; + +// First position where target could be inserted (first >= target) +const lower = std.sort.lowerBound(i32, &items, @as(i32, 5), order); +// 2 (first 5) + +// First position after all equal elements (first > target) +const upper = std.sort.upperBound(i32, &items, @as(i32, 5), order); +// 5 (after last 5) + +// Both bounds at once +const range = std.sort.equalRange(i32, &items, @as(i32, 5), order); +// .{ 2, 5 } (indices of all 5s) +``` + +## Partition Point + +Find where predicate changes from true to false: + +```zig +fn lessThan5(_: void, item: i32) bool { + return item < 5; +} + +const items = [_]i32{ 1, 2, 3, 4, 5, 6, 7 }; +const point = std.sort.partitionPoint(i32, &items, {}, lessThan5); +// 4 (first index where predicate is false) +``` + +## Min/Max + +```zig +const items = [_]i32{ 5, 2, 8, 1, 9 }; + +// Get min/max value +const minimum = std.sort.min(i32, &items, {}, std.sort.asc(i32)); // ?i32 = 1 +const maximum = std.sort.max(i32, &items, {}, std.sort.asc(i32)); // ?i32 = 9 + +// Get index of min/max +const min_idx = std.sort.argMin(i32, &items, {}, std.sort.asc(i32)); // ?usize = 3 +const max_idx = std.sort.argMax(i32, &items, {}, std.sort.asc(i32)); // ?usize = 4 + +// Empty slice returns null +const empty: []const i32 = &.{}; +const none = std.sort.min(i32, empty, {}, std.sort.asc(i32)); // null +``` + +## Context-Based Sorting (Advanced) + +For sorting indices into external data using index-based context: + +```zig +const Context = struct { + items: []i32, + + pub fn lessThan(ctx: @This(), a: usize, b: usize) bool { + return ctx.items[a] < ctx.items[b]; + } + + pub fn swap(ctx: @This(), a: usize, b: usize) void { + std.mem.swap(i32, &ctx.items[a], &ctx.items[b]); + } +}; + +var items = [_]i32{ 5, 2, 8, 1 }; +const ctx = Context{ .items = &items }; + +// Sort a subrange using indices +std.sort.pdqContext(1, 4, ctx); // sort indices 1..4 +// items = [5, 1, 2, 8] +``` + +## Stable Sort Example + +When sorting by one field but preserving order of equal elements: + +```zig +const Item = struct { + id: usize, + priority: u32, +}; + +fn byPriority(_: void, a: Item, b: Item) bool { + return a.priority < b.priority; +} + +var items = [_]Item{ + .{ .id = 0, .priority = 1 }, + .{ .id = 1, .priority = 1 }, // same priority as id=0 + .{ .id = 2, .priority = 0 }, +}; + +// Stable sort preserves id order for equal priorities +std.sort.block(Item, &items, {}, byPriority); +// Result: {id=2, p=0}, {id=0, p=1}, {id=1, p=1} +// id=0 still comes before id=1 +``` + +## Algorithm Selection + +- **`pdq`** (Pattern-Defeating Quicksort): Best general-purpose unstable sort. Adapts to input patterns, falls back to heapsort for worst cases. +- **`block`**: Best general-purpose stable sort. Preserves relative order of equal elements. +- **`insertion`**: O(n) on nearly sorted data. Use for small arrays or as final pass. +- **`heap`**: Guaranteed O(n log n) with O(1) memory. No recursion, predictable performance. + +## Notes + +- All sorts are **in-place** with O(1) or O(log n) auxiliary memory +- Comparators must define strict weak ordering (if `a < b` then not `b < a`) +- `asc`/`desc` helpers work with any type supporting `<` operator +- Binary search functions require the array to already be sorted +- `equalRange` is more efficient than calling `lowerBound` + `upperBound` separately diff --git a/references/std-static-string-map.md b/references/std-static-string-map.md new file mode 100644 index 0000000..cb69161 --- /dev/null +++ b/references/std-static-string-map.md @@ -0,0 +1,223 @@ +# std.StaticStringMap + +Compile-time optimized string lookup. Perfect hash for small, fixed sets of string keys. + +## When to Use + +- Keywords/reserved words lookup +- Command/option parsing +- Static configuration keys +- When string set is known at compile time +- Very fast O(1) lookups by string length + +## Basic Usage + +```zig +const std = @import("std"); + +const keywords = std.StaticStringMap(enum { @"if", @"else", @"while", @"for" }).initComptime(.{ + .{ "if", .@"if" }, + .{ "else", .@"else" }, + .{ "while", .@"while" }, + .{ "for", .@"for" }, +}); + +// Lookup +if (keywords.get("while")) |kw| { + switch (kw) { + .@"while" => std.debug.print("found while\n", .{}), + else => {}, + } +} + +// Check existence +if (keywords.has("if")) { + // it's a keyword +} +``` + +## Void Value (Set) + +```zig +// When you only need presence check, use void +const reserved = std.StaticStringMap(void).initComptime(.{ + .{"break"}, + .{"continue"}, + .{"return"}, + .{"defer"}, +}); + +if (reserved.has("break")) { + std.debug.print("'break' is reserved\n", .{}); +} +``` + +## Case-Insensitive Lookup + +```zig +const commands = std.StaticStringMapWithEql( + i32, + std.static_string_map.eqlAsciiIgnoreCase, +).initComptime(.{ + .{ "help", 1 }, + .{ "quit", 2 }, + .{ "list", 3 }, +}); + +// All find the same entry +_ = commands.get("HELP"); // 1 +_ = commands.get("Help"); // 1 +_ = commands.get("help"); // 1 +``` + +## Runtime Initialization + +```zig +// When data isn't known at comptime +const pairs = [_]struct { []const u8, i32 }{ + .{ "one", 1 }, + .{ "two", 2 }, + .{ "three", 3 }, +}; + +const map = try std.StaticStringMap(i32).init(&pairs, allocator); +defer map.deinit(allocator); + +_ = map.get("two"); // 2 +``` + +## Get Index + +```zig +// Get the index in the internal array +if (keywords.getIndex("if")) |idx| { + // idx is position in sorted-by-length array +} +``` + +## Longest Prefix Match + +```zig +const prefixes = std.StaticStringMap(u32).initComptime(.{ + .{ "/api", 1 }, + .{ "/api/v1", 2 }, + .{ "/api/v2", 3 }, +}); + +// Find longest matching prefix +if (prefixes.getLongestPrefix("/api/v2/users")) |kv| { + std.debug.print("matched: {s} -> {}\n", .{ kv.key, kv.value }); + // matched: /api/v2 -> 3 +} +``` + +## Access All Keys/Values + +```zig +const all_keys = keywords.keys(); // []const []const u8 +const all_values = keywords.values(); // []const EnumType +``` + +## Complete Example: HTTP Method Parser + +```zig +const std = @import("std"); + +const Method = enum { + GET, + POST, + PUT, + DELETE, + PATCH, + HEAD, + OPTIONS, +}; + +const methods = std.StaticStringMap(Method).initComptime(.{ + .{ "GET", .GET }, + .{ "POST", .POST }, + .{ "PUT", .PUT }, + .{ "DELETE", .DELETE }, + .{ "PATCH", .PATCH }, + .{ "HEAD", .HEAD }, + .{ "OPTIONS", .OPTIONS }, +}); + +fn parseMethod(s: []const u8) ?Method { + return methods.get(s); +} + +pub fn main() void { + if (parseMethod("POST")) |m| { + std.debug.print("Method: {}\n", .{m}); // Method: POST + } + + if (parseMethod("INVALID")) |_| { + unreachable; + } else { + std.debug.print("Invalid method\n", .{}); + } +} +``` + +## Complete Example: Config Parser + +```zig +const std = @import("std"); + +const ConfigKey = enum { + host, + port, + debug, + timeout, +}; + +const config_keys = std.StaticStringMapWithEql( + ConfigKey, + std.static_string_map.eqlAsciiIgnoreCase, +).initComptime(.{ + .{ "host", .host }, + .{ "port", .port }, + .{ "debug", .debug }, + .{ "timeout", .timeout }, +}); + +fn parseConfig(line: []const u8) ?struct { key: ConfigKey, value: []const u8 } { + const eq_idx = std.mem.indexOf(u8, line, "=") orelse return null; + const key_str = std.mem.trim(u8, line[0..eq_idx], " "); + const value = std.mem.trim(u8, line[eq_idx + 1 ..], " "); + + const key = config_keys.get(key_str) orelse return null; + return .{ .key = key, .value = value }; +} + +pub fn main() void { + const lines = [_][]const u8{ + "HOST = localhost", + "PORT = 8080", + "Debug = true", + }; + + for (lines) |line| { + if (parseConfig(line)) |cfg| { + std.debug.print("{}: {s}\n", .{ cfg.key, cfg.value }); + } + } +} +``` + +## How It Works + +1. Strings are sorted by length at compile time +2. Lookup first checks length to narrow candidates +3. Only strings of matching length are compared +4. Very efficient for disparate key lengths + +## Notes + +- Comptime version has zero runtime allocation +- Keys are grouped by length for fast rejection +- Use `eqlAsciiIgnoreCase` for case-insensitive matching +- Runtime `init()` requires `deinit()` to free memory +- Best for small to medium-sized static string sets +- Not suitable for dynamic key sets (use HashMap instead) diff --git a/references/std-tar.md b/references/std-tar.md new file mode 100644 index 0000000..ddc1926 --- /dev/null +++ b/references/std-tar.md @@ -0,0 +1,436 @@ +# std.tar - Tar Archive API Reference (Zig 0.16.0) + +Tar archive reading and writing. Zig 0.16 file and stream APIs use `std.Io.Dir`, `std.Io.File`, `std.Io.Reader`, and `std.Io.Writer`. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Old examples below may contain 0.15 `std.fs` or `std.io` patterns; translate them to `std.Io` before using in new code. The 0.16 release notes also call out path traversal sanitization in `tar.extract`. + +## Table of Contents +- [Module Structure](#module-structure) +- [Reading Tar Archives](#reading-tar-archives) +- [Extracting to Filesystem](#extracting-to-filesystem) +- [Writing Tar Archives](#writing-tar-archives) +- [Diagnostics and Error Handling](#diagnostics-and-error-handling) +- [Common Patterns](#common-patterns) + +## Module Structure + +```zig +std.tar.Iterator // Iterate over entries in tar archive +std.tar.Writer // Create tar archives +std.tar.Diagnostics // Collect errors during extraction +std.tar.FileKind // .file, .directory, .sym_link +std.tar.PipeOptions // Options for pipeToFileSystem +std.tar.pipeToFileSystem() // Extract archive to directory +``` + +## Reading Tar Archives + +### Iterator API + +Iterate over files, directories, and symlinks in a tar archive: + +```zig +const data = @embedFile("archive.tar"); +var reader: std.Io.Reader = .fixed(data); + +// Buffers must be provided by caller +var file_name_buffer: [std.fs.max_path_bytes]u8 = undefined; +var link_name_buffer: [std.fs.max_path_bytes]u8 = undefined; + +var it: std.tar.Iterator = .init(&reader, .{ + .file_name_buffer = &file_name_buffer, + .link_name_buffer = &link_name_buffer, +}); + +while (try it.next()) |file| { + switch (file.kind) { + .directory => std.debug.print("Dir: {s}\n", .{file.name}), + .file => { + std.debug.print("File: {s} ({d} bytes)\n", .{ file.name, file.size }); + // Read file content - see below + }, + .sym_link => std.debug.print("Link: {s} -> {s}\n", .{ file.name, file.link_name }), + } +} +``` + +### Iterator.File Structure + +```zig +pub const File = struct { + name: []const u8, // file/dir/symlink path + link_name: []const u8, // symlink target (empty for files/dirs) + size: u64, // file size in bytes + mode: u32, // POSIX permission mode + kind: FileKind, // .file, .directory, .sym_link +}; +``` + +### Reading File Contents + +File content must be read before calling `next()` again: + +```zig +while (try it.next()) |file| { + if (file.kind == .file) { + // Option 1: Stream to writer + var buf: [1024]u8 = undefined; + var output_file = try dir.createFile(file.name, .{}); + defer output_file.close(); + var file_writer = output_file.writer(&buf); + try it.streamRemaining(file, &file_writer.interface); + try file_writer.interface.flush(); + + // Option 2: Stream to allocated buffer + var content: std.Io.Writer.Allocating = .init(allocator); + defer content.deinit(); + try it.streamRemaining(file, &content.writer); + const bytes = content.written(); // []const u8 + } +} +``` + +### Iterator Options + +```zig +pub const Options = struct { + file_name_buffer: []u8, // buffer for file paths (use max_path_bytes) + link_name_buffer: []u8, // buffer for symlink targets + diagnostics: ?*Diagnostics, // optional error collection +}; +``` + +## Extracting to Filesystem + +### pipeToFileSystem + +Extract entire archive to a directory: + +```zig +const data = @embedFile("archive.tar"); +var reader: std.Io.Reader = .fixed(data); + +try std.tar.pipeToFileSystem(std.fs.cwd(), &reader, .{ + .strip_components = 1, // remove leading path component + .mode_mode = .executable_bit_only, + .exclude_empty_directories = false, +}); +``` + +### From File + +```zig +const file = try std.fs.cwd().openFile("archive.tar", .{}); +defer file.close(); + +var buf: [4096]u8 = undefined; +var file_reader = file.reader(&buf); + +try std.tar.pipeToFileSystem(output_dir, &file_reader.interface, .{}); +``` + +### PipeOptions + +```zig +pub const PipeOptions = struct { + strip_components: u32 = 0, // directories to strip from paths + mode_mode: ModeMode = .executable_bit_only, + exclude_empty_directories: bool = false, + diagnostics: ?*Diagnostics = null, + + pub const ModeMode = enum { + ignore, // ignore tar mode, use system defaults + executable_bit_only, // copy only executable bit to group/other + }; +}; +``` + +**strip_components**: Removes leading path components. `strip_components = 1` converts `archive/src/main.zig` to `src/main.zig`. + +**mode_mode**: +- `.ignore`: All files created with default permissions +- `.executable_bit_only`: If owner has execute bit, set it for group and other too + +## Writing Tar Archives + +### Writer API + +```zig +var output: std.Io.Writer.Allocating = .init(allocator); +defer output.deinit(); + +var w: std.tar.Writer = .{ .underlying_writer = &output.writer }; + +// Optional: set root directory prefix +try w.setRoot("myproject"); + +// Write files +try w.writeFileBytes("README.md", "# My Project\n", .{}); +try w.writeFileBytes("src/main.zig", source_code, .{ .mode = 0o644 }); + +// Write directory +try w.writeDir("data", .{}); + +// Write symlink +try w.writeLink("latest", "v1.0", .{}); + +// Get tar data +const tar_bytes = output.written(); +``` + +### Writing from File + +```zig +var output_file = try std.fs.cwd().createFile("archive.tar", .{}); +defer output_file.close(); +var buf: [4096]u8 = undefined; +var file_writer = output_file.writer(&buf); + +var w: std.tar.Writer = .{ .underlying_writer = &file_writer.interface }; + +// Write file from disk +var src_file = try std.fs.cwd().openFile("data.txt", .{}); +defer src_file.close(); +var src_buf: [4096]u8 = undefined; +var src_reader = src_file.reader(&src_buf); +const stat = try src_file.stat(); + +try w.writeFile("data.txt", &src_reader, stat.mtime); + +try file_writer.interface.flush(); +``` + +### Writing from Stream + +```zig +// When you know the size upfront +var content_reader: std.Io.Reader = .fixed(content_bytes); +try w.writeFileStream("file.txt", content_bytes.len, &content_reader, .{}); +``` + +### Writer Methods + +```zig +// Set prefix for all subsequent paths +pub fn setRoot(w: *Writer, root: []const u8) Error!void + +// Write directory entry +pub fn writeDir(w: *Writer, sub_path: []const u8, options: Options) Error!void + +// Write file from bytes +pub fn writeFileBytes(w: *Writer, sub_path: []const u8, content: []const u8, options: Options) Error!void + +// Write file from reader with known size +pub fn writeFileStream(w: *Writer, sub_path: []const u8, size: u64, reader: *std.Io.Reader, options: Options) WriteFileStreamError!void + +// Write file from file reader +pub fn writeFile(w: *Writer, sub_path: []const u8, file_reader: *std.fs.File.Reader, stat_mtime: i128) WriteFileError!void + +// Write symbolic link +pub fn writeLink(w: *Writer, sub_path: []const u8, link_name: []const u8, options: Options) Error!void + +// Write two zero blocks (optional, per spec) +pub fn finishPedantically(w: *Writer) std.Io.Writer.Error!void +``` + +### Writer Options + +```zig +pub const Options = struct { + mode: u32 = 0, // POSIX mode (0 = default: 0o664 for files) + mtime: u64 = 0, // modification time (0 = current time) +}; +``` + +## Diagnostics and Error Handling + +### Using Diagnostics + +Collect errors instead of failing immediately: + +```zig +var diagnostics: std.tar.Diagnostics = .{ .allocator = allocator }; +defer diagnostics.deinit(); + +std.tar.pipeToFileSystem(dir, &reader, .{ + .diagnostics = &diagnostics, +}) catch |err| { + // Some errors are still fatal + return err; +}; + +// Check collected errors +for (diagnostics.errors.items) |item| { + switch (item) { + .unable_to_create_file => |info| { + std.debug.print("Failed to create {s}: {}\n", .{ info.file_name, info.code }); + }, + .unable_to_create_sym_link => |info| { + std.debug.print("Failed to link {s} -> {s}\n", .{ info.file_name, info.link_name }); + }, + .unsupported_file_type => |info| { + std.debug.print("Unsupported: {s} (type {})\n", .{ info.file_name, info.file_type }); + }, + .components_outside_stripped_prefix => |info| { + std.debug.print("Stripped: {s}\n", .{info.file_name}); + }, + } +} + +// Diagnostics also tracks root directory discovery +std.debug.print("Root dir: {s}, entries: {d}\n", .{ diagnostics.root_dir, diagnostics.entries }); +``` + +### Diagnostics.Error Types + +```zig +pub const Error = union(enum) { + unable_to_create_sym_link: struct { + code: anyerror, + file_name: []const u8, + link_name: []const u8, + }, + unable_to_create_file: struct { + code: anyerror, + file_name: []const u8, + }, + unsupported_file_type: struct { + file_name: []const u8, + file_type: Header.Kind, + }, + components_outside_stripped_prefix: struct { + file_name: []const u8, + }, +}; +``` + +## Common Patterns + +### Extract and Process Archive + +```zig +fn extractTar(allocator: Allocator, tar_data: []const u8, dest: std.fs.Dir) !void { + var reader: std.Io.Reader = .fixed(tar_data); + + var diagnostics: std.tar.Diagnostics = .{ .allocator = allocator }; + defer diagnostics.deinit(); + + try std.tar.pipeToFileSystem(dest, &reader, .{ + .strip_components = 1, + .diagnostics = &diagnostics, + }); + + if (diagnostics.errors.items.len > 0) { + for (diagnostics.errors.items) |err| { + std.log.warn("tar extraction issue: {}", .{err}); + } + } +} +``` + +### List Archive Contents + +```zig +fn listTar(allocator: Allocator, tar_data: []const u8) !void { + _ = allocator; + var reader: std.Io.Reader = .fixed(tar_data); + + var file_name_buffer: [std.fs.max_path_bytes]u8 = undefined; + var link_name_buffer: [std.fs.max_path_bytes]u8 = undefined; + + var it: std.tar.Iterator = .init(&reader, .{ + .file_name_buffer = &file_name_buffer, + .link_name_buffer = &link_name_buffer, + }); + + while (try it.next()) |file| { + const kind_char: u8 = switch (file.kind) { + .directory => 'd', + .file => '-', + .sym_link => 'l', + }; + std.debug.print("{c} {o:0>4} {d:>10} {s}", .{ + kind_char, file.mode, file.size, file.name, + }); + if (file.kind == .sym_link) { + std.debug.print(" -> {s}", .{file.link_name}); + } + std.debug.print("\n", .{}); + } +} +``` + +### Create Archive from Directory + +```zig +fn createTarFromDir(allocator: Allocator, source_dir: std.fs.Dir, root_name: []const u8) ![]u8 { + var output: std.Io.Writer.Allocating = .init(allocator); + errdefer output.deinit(); + + var w: std.tar.Writer = .{ .underlying_writer = &output.writer }; + try w.setRoot(root_name); + + var walker = try source_dir.walk(allocator); + defer walker.deinit(); + + while (try walker.next()) |entry| { + switch (entry.kind) { + .directory => try w.writeDir(entry.path, .{}), + .file => { + var file = try entry.dir.openFile(entry.basename, .{}); + defer file.close(); + var buf: [4096]u8 = undefined; + var file_reader = file.reader(&buf); + const stat = try file.stat(); + try w.writeFile(entry.path, &file_reader, stat.mtime); + }, + .sym_link => { + var link_buf: [std.fs.max_path_bytes]u8 = undefined; + const target = try entry.dir.readLink(entry.basename, &link_buf); + try w.writeLink(entry.path, target, .{}); + }, + else => {}, // skip special files + } + } + + return output.toOwnedSlice(); +} +``` + +### Extract Single File + +```zig +fn extractFile(tar_data: []const u8, target_name: []const u8, allocator: Allocator) !?[]u8 { + var reader: std.Io.Reader = .fixed(tar_data); + + var file_name_buffer: [std.fs.max_path_bytes]u8 = undefined; + var link_name_buffer: [std.fs.max_path_bytes]u8 = undefined; + + var it: std.tar.Iterator = .init(&reader, .{ + .file_name_buffer = &file_name_buffer, + .link_name_buffer = &link_name_buffer, + }); + + while (try it.next()) |file| { + if (file.kind == .file and std.mem.eql(u8, file.name, target_name)) { + var content: std.Io.Writer.Allocating = .init(allocator); + errdefer content.deinit(); + try it.streamRemaining(file, &content.writer); + return content.toOwnedSlice(); + } + } + return null; +} +``` + +## Supported Features + +**Formats**: POSIX ustar, GNU long name/link extensions, pax extended headers + +**Entry types**: Regular files, directories, symbolic links + +**Not supported**: Hard links, device nodes, FIFOs, sparse files (logged via diagnostics) + +**Path handling**: Automatic prefix/name splitting, GNU extended headers for paths > 256 bytes diff --git a/references/std-testing.md b/references/std-testing.md new file mode 100644 index 0000000..67c8319 --- /dev/null +++ b/references/std-testing.md @@ -0,0 +1,381 @@ +# std.testing (Zig 0.16.0) + +Unit testing utilities and assertions for Zig tests. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Use `std.testing.io` for tests that call file, process, networking, time, entropy, or I/O synchronization APIs: + +```zig +const io = std.testing.io; +``` + +Fuzz tests now use `*std.testing.Smith` rather than raw `[]const u8` input. + +## Quick Reference + +| Function | Purpose | +|----------|---------| +| `expect(bool)` | Assert condition is true | +| `expectEqual(expected, actual)` | Shallow equality (peer type resolution) | +| `expectEqualDeep(expected, actual)` | Deep equality (follows pointers, compares contents) | +| `expectEqualStrings(expected, actual)` | String equality with diff output | +| `expectEqualSlices(T, expected, actual)` | Slice equality with diff output | +| `expectError(error, result)` | Assert specific error returned | +| `expectApproxEqAbs/Rel(expected, actual, tolerance)` | Float comparison | +| `expectFmt(expected, template, args)` | Format string output | +| `expectStringStartsWith(actual, prefix)` | String prefix check | +| `expectStringEndsWith(actual, suffix)` | String suffix check | + +## Basic Assertions + +```zig +const testing = std.testing; + +// Boolean condition +try testing.expect(value > 0); + +// Equality (uses peer type resolution) +try testing.expectEqual(expected, actual); +try testing.expectEqual(@as(u32, 42), some_u32); + +// String equality (with visual diff on failure) +try testing.expectEqualStrings("hello", slice); + +// String prefix/suffix +try testing.expectStringStartsWith(path, "/home/"); +try testing.expectStringEndsWith(filename, ".zig"); + +// Slice equality (with visual diff, works with any element type) +try testing.expectEqualSlices(u8, expected_bytes, actual_bytes); +try testing.expectEqualSlices(u32, &[_]u32{1, 2, 3}, result_slice); + +// Sentinel-terminated slice equality +try testing.expectEqualSentinel(u8, 0, expected_cstr, actual_cstr); + +// Deep equality (recursively compares structs, arrays, pointers) +try testing.expectEqualDeep(expected_struct, actual_struct); + +// Float comparison (absolute tolerance) +try testing.expectApproxEqAbs(@as(f32, 1.0), result, 0.001); + +// Float comparison (relative tolerance) +try testing.expectApproxEqRel(@as(f64, 100.0), result, 0.01); +``` + +### expectEqual vs expectEqualDeep + +```zig +const Point = struct { x: i32, y: i32 }; + +// expectEqual - compares by value for primitives, by identity for pointers +const p1 = Point{ .x = 1, .y = 2 }; +const p2 = Point{ .x = 1, .y = 2 }; +try testing.expectEqual(p1, p2); // OK - structs compared field-by-field + +// For slices, expectEqual compares ptr and len (identity) +const a = [_]u8{ 1, 2, 3 }; +const b = [_]u8{ 1, 2, 3 }; +// testing.expectEqual(&a, &b); // FAILS - different pointers + +// expectEqualDeep - follows pointers, compares contents +try testing.expectEqualDeep(&a, &b); // OK - compares contents +try testing.expectEqualDeep("abc", "abc"); // OK +``` + +## Error Assertions + +```zig +// Expect specific error +try testing.expectError(error.OutOfMemory, fallible_function()); + +// Unwrap or fail test (using try directly) +const value = try fallible_function(); // fails test on any error +``` + +## Format Testing + +```zig +// Test format string output +try testing.expectFmt("42", "{}", .{@as(u32, 42)}); +try testing.expectFmt("hello world", "{s} {s}", .{"hello", "world"}); +``` + +## Testing Allocator + +`std.testing.allocator` is a `DebugAllocator` (formerly `GeneralPurposeAllocator`) that detects memory leaks and use-after-free. **Only available in test builds.** + +```zig +test "with allocator" { + // Detects leaks and use-after-free + var list: std.ArrayList(u32) = .empty; + defer list.deinit(testing.allocator); + + try list.append(testing.allocator, 42); + try testing.expectEqual(@as(usize, 1), list.items.len); +} +// If defer is missing, test fails with leak report +``` + +## Failing Allocator + +`std.testing.failing_allocator` always returns `error.OutOfMemory`. Use for testing error paths: + +```zig +test "handle allocation failure" { + try testing.expectError( + error.OutOfMemory, + testing.failing_allocator.alloc(u8, 100) + ); +} +``` + +### Configurable FailingAllocator + +For controlled failure testing, use `FailingAllocator` to fail after N allocations: + +```zig +test "fail on third allocation" { + var failing = std.testing.FailingAllocator.init(std.testing.allocator, .{ + .fail_index = 2, // First 2 allocations succeed, third fails + }); + const allocator = failing.allocator(); + + const a = try allocator.create(i32); // succeeds (index 0) + defer allocator.destroy(a); + const b = try allocator.create(i32); // succeeds (index 1) + defer allocator.destroy(b); + + try testing.expectError(error.OutOfMemory, allocator.create(i32)); // fails (index 2) +} + +// Configuration options +var failing = std.testing.FailingAllocator.init(backing_allocator, .{ + .fail_index = 5, // Fail on 6th allocation (default: never) + .resize_fail_index = 3, // Fail on 4th resize (default: never) +}); + +// Inspect state after use +std.debug.print("Allocated: {} bytes\n", .{failing.allocated_bytes}); +std.debug.print("Freed: {} bytes\n", .{failing.freed_bytes}); +std.debug.print("Allocations: {}\n", .{failing.allocations}); +std.debug.print("Deallocations: {}\n", .{failing.deallocations}); +``` + +## Exhaustive Allocation Failure Testing + +`checkAllAllocationFailures` tests that your code handles `OutOfMemory` at every allocation point without leaking: + +```zig +fn myFunction(allocator: std.mem.Allocator, size: usize) !void { + var foo = try allocator.alloc(u8, size); + defer allocator.free(foo); + var bar = try allocator.alloc(u8, size); + defer allocator.free(bar); + // ... use foo and bar +} + +test "no leaks on allocation failure" { + // Runs myFunction multiple times, failing each allocation in turn + try std.testing.checkAllAllocationFailures( + std.testing.allocator, + myFunction, + .{@as(usize, 10)}, // extra args tuple + ); +} +``` + +**How it works:** +1. Runs function once to count total allocations +2. Runs N more times, failing allocation 0, then 1, then 2... +3. Verifies `OutOfMemory` is returned and no memory leaked + +**Errors returned:** +- `error.MemoryLeakDetected` - allocation failed but memory wasn't freed +- `error.SwallowedOutOfMemoryError` - `OutOfMemory` was caught but not propagated +- `error.NondeterministicMemoryUsage` - allocation count varies between runs + +## Temporary Directory + +Create an isolated temp directory for file system tests: + +```zig +test "file operations" { + var tmp = std.testing.tmpDir(.{}); // creates .zig-cache/tmp// + defer tmp.cleanup(); + + // Write and read files + var file = try tmp.dir.createFile("test.txt", .{}); + defer file.close(); + try file.writeAll("hello"); + + // Use tmp.dir for all operations + const content = try tmp.dir.readFileAlloc(std.testing.allocator, "test.txt", 1024); + defer std.testing.allocator.free(content); + try testing.expectEqualStrings("hello", content); +} +``` + +## Test Organization + +```zig +test "descriptive test name" { + // test body +} + +test { + // Anonymous test, runs with others +} + +// Reference other tests (pulls in tests from imported module) +test { + _ = @import("other_module.zig"); +} + +// Force semantic analysis of all declarations (catches unused code errors) +comptime { + std.testing.refAllDecls(@This()); +} + +// Recursive version for nested types +comptime { + std.testing.refAllDeclsRecursive(@This()); +} +``` + +## Skip Tests + +```zig +test "skip this" { + return error.SkipZigTest; +} + +test "conditional skip" { + if (builtin.os.tag == .windows) return error.SkipZigTest; + // ... +} + +test "skip if feature unavailable" { + if (!@hasDecl(std.os, "linux")) return error.SkipZigTest; + // Linux-specific test... +} +``` + +## Test Logging + +```zig +test "with logging" { + // Only shown when test fails or with --verbose + std.debug.print("Debug info: {}\n", .{value}); +} + +// Configurable log level for tests +// std.testing.log_level = .debug; // default is .warn +``` + +## Deterministic Randomness + +Tests have access to a deterministic random seed for reproducible "random" tests: + +```zig +test "deterministic random" { + var prng = std.Random.DefaultPrng.init(std.testing.random_seed); + const random = prng.random(); + + const value = random.int(u32); + // Same seed = same value on every run +} +``` + +## Fuzz Testing + +```zig +test "fuzz parser" { + try std.testing.fuzz( + {}, // context (passed to test function) + struct { + fn testOne(_: void, input: []const u8) !void { + // This runs with many different inputs + _ = myParser.parse(input) catch |err| switch (err) { + error.InvalidInput => return, // expected + else => return err, + }; + } + }.testOne, + .{ + .corpus = &.{ // seed inputs + "valid input 1", + "valid input 2", + }, + }, + ); +} +``` + +## Common Patterns + +### Table-Driven Tests + +```zig +test "parameterized" { + const cases = [_]struct { input: i32, expected: i32 }{ + .{ .input = 0, .expected = 0 }, + .{ .input = 1, .expected = 1 }, + .{ .input = -1, .expected = 1 }, + }; + + for (cases) |case| { + try testing.expectEqual(case.expected, abs(case.input)); + } +} +``` + +### Test Context/Fixture + +```zig +const TestContext = struct { + allocator: std.mem.Allocator, + data: *Data, + + fn init(ally: std.mem.Allocator) !TestContext { + const data = try ally.create(Data); + return .{ .allocator = ally, .data = data }; + } + + fn deinit(self: *TestContext) void { + self.allocator.destroy(self.data); + } +}; + +test "with context" { + var ctx = try TestContext.init(testing.allocator); + defer ctx.deinit(); + // use ctx.data... +} +``` + +### Testing with ArenaAllocator + +```zig +test "arena for test allocations" { + var arena = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena.deinit(); + const ally = arena.allocator(); + + // No need for individual frees - arena handles cleanup + const a = try ally.alloc(u8, 100); + const b = try ally.alloc(u8, 200); + _ = a; _ = b; + // arena.deinit() frees everything +} +``` + +## Running Tests + +```bash +zig build test # Run all tests +zig test src/lib.zig # Test single file +zig test --test-filter "name" # Filter by name substring +zig test -fsummary # Show test summary +zig test --verbose # Show debug output +``` diff --git a/references/std-thread.md b/references/std-thread.md new file mode 100644 index 0000000..6f11a36 --- /dev/null +++ b/references/std-thread.md @@ -0,0 +1,199 @@ +# std.Thread and std.Io Synchronization (Zig 0.16.0) + +Primary release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 keeps `std.Thread` for OS thread spawning and thread utilities, but blocking synchronization and task concurrency should move to `std.Io` when it may block inside code that participates in the I/O runtime. + +## What Remains in std.Thread + +Use `std.Thread` for explicit OS threads: + +```zig +const std = @import("std"); + +fn worker(id: usize) void { + std.debug.print("worker {d}\n", .{id}); +} + +pub fn main() !void { + const thread = try std.Thread.spawn(.{}, worker, .{42}); + thread.join(); +} +``` + +Useful thread utilities: + +```zig +const id = std.Thread.getCurrentId(); +const cpu_count = std.Thread.getCpuCount() catch 1; +std.Thread.sleep(10 * std.time.ns_per_ms); +std.Thread.yield() catch {}; +``` + +## Removed or Avoided std.Thread APIs + +- `std.Thread.Pool` was removed. +- `std.Thread.Mutex.Recursive` was removed. +- `std.once` was removed. +- Blocking synchronization in I/O-aware code should use `std.Io` primitives. + +## std.Io Sync Migration + +Release-note migration map: + +| Old | New | +|-----|-----| +| `std.Thread.Mutex` | `std.Io.Mutex` | +| `std.Thread.Condition` | `std.Io.Condition` | +| `std.Thread.Semaphore` | `std.Io.Semaphore` | +| `std.Thread.RwLock` | `std.Io.RwLock` | +| `std.Thread.ResetEvent` | `std.Io.Event` | +| `std.Thread.WaitGroup` | `std.Io.Group` | +| `std.Thread.Futex` | `std.Io.Futex` | + +Lock-free atomics do not need `std.Io`. + +## Mutex + +Use cancelable locking when cancelation should be honored: + +```zig +var mutex: std.Io.Mutex = .init; +var value: u64 = 0; + +fn increment(io: std.Io) !void { + try mutex.lock(io); + defer mutex.unlock(io); + value += 1; +} +``` + +Use uncancelable locking for short critical sections where interruption would corrupt state or skip required cleanup: + +```zig +mutex.lockUncancelable(io); +defer mutex.unlock(io); +``` + +Do not replace mutexes with spin loops unless there is a documented, measured special case. + +## Condition + +`std.Io.Condition` waits with an `io` and a `std.Io.Mutex`. + +```zig +var mutex: std.Io.Mutex = .init; +var condition: std.Io.Condition = .init; +var ready = false; + +fn waitUntilReady(io: std.Io) !void { + try mutex.lock(io); + defer mutex.unlock(io); + + while (!ready) { + try condition.wait(io, &mutex); + } +} + +fn setReady(io: std.Io) void { + mutex.lockUncancelable(io); + defer mutex.unlock(io); + + ready = true; + condition.broadcast(io); +} +``` + +## Semaphore + +```zig +var semaphore: std.Io.Semaphore = .{ .permits = 3 }; + +fn usePermit(io: std.Io) !void { + try semaphore.wait(io); + defer semaphore.post(io); + // protected limited-concurrency work +} +``` + +Use `waitUncancelable(io)` only when a cancellation point would be invalid. + +## RwLock + +```zig +var rw: std.Io.RwLock = .init; + +fn read(io: std.Io) !void { + try rw.lockShared(io); + defer rw.unlockShared(io); +} + +fn write(io: std.Io) !void { + try rw.lock(io); + defer rw.unlock(io); +} +``` + +## Event + +`std.Thread.ResetEvent` maps to `std.Io.Event`. + +```zig +var event: std.Io.Event = .unset; + +fn waiter(io: std.Io) !void { + try event.wait(io); +} + +fn signal(io: std.Io) void { + event.set(io); +} +``` + +## Group Instead of WaitGroup / Thread.Pool + +Use `std.Io.Group` to spawn and await related tasks. + +```zig +fn doWork(io: std.Io) !void { + var group: std.Io.Group = .init; + errdefer group.cancel(io); + + group.async(io, worker, .{ io, 0 }); + group.async(io, worker, .{ io, 1 }); + + try group.await(io); +} + +fn worker(io: std.Io, id: usize) void { + _ = .{ io, id }; +} +``` + +Do not mechanically replace `std.Thread.Pool` with `std.Io.Group` if tasks synchronously depend on the caller or mutate shared state without I/O-aware synchronization. Re-evaluate ownership and synchronization first. + +## Cancelation + +Cancelable waits/locks may return `error.Canceled`. + +Rules: + +- Propagate `error.Canceled` by default. +- Only ignore it in code that requested cancelation. +- If you catch it and continue, use `io.recancel()` when the request should remain active. +- Use uncancelable waits/locks sparingly and only where cancellation would violate invariants. + +## Application Guidance + +- Queues, allocators, registries, timers, and worker systems that use blocking sync should accept/store `std.Io`. +- Use `lockUncancelable(io)` for queue integrity and allocator metadata updates when cancellation cannot safely interrupt the critical section. +- Keep OS threads for code that is explicitly thread-owned, such as dedicated worker threads or external API callbacks. +- Prefer an application's existing scheduler for application work unless the standard `std.Io` task API is explicitly the better fit. + +## Review Checklist + +- Is this code in an I/O-aware path? If yes, sync primitives should be `std.Io.*`. +- Does every blocking lock/wait receive an `io`? +- Is cancelation either propagated or deliberately protected? +- Was `std.Thread.Pool` replaced with a design that preserves ordering and synchronization semantics? +- Are atomics used only for lock-free state where blocking is not required? diff --git a/references/std-time.md b/references/std-time.md new file mode 100644 index 0000000..a42b389 --- /dev/null +++ b/references/std-time.md @@ -0,0 +1,128 @@ +# Time and Timing (Zig 0.16.0) + +Primary release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Zig 0.16 moves time APIs that depend on the runtime behind `std.Io`. Use `std.Io.Timestamp`, `std.Io.Duration`, and `std.Io.Clock`. + +## Migration Summary + +Release-note map: + +- `std.time.Instant` -> `std.Io.Timestamp` +- `std.time.Timer` -> `std.Io.Timestamp` +- `std.time.timestamp` -> `std.Io.Timestamp.now` +- `{D}` duration formatting -> format `std.Io.Duration` with `{f}` + +`std.time` still provides constants and calendar helpers such as `ns_per_ms`, `ns_per_s`, and `std.time.epoch`. + +## Clocks + +`std.Io.Clock` values: + +- `.real`: wall-clock Unix/POSIX time, affected by clock changes. +- `.awake`: monotonic-style clock intended to exclude suspend time where possible. +- `.boot`: monotonic-style clock intended to include suspend time where possible. +- `.cpu_process`: CPU time used by current process. +- `.cpu_thread`: CPU time used by current thread. + +## Current Timestamp + +```zig +const wall = std.Io.Timestamp.now(io, .real); +const boot = std.Io.Timestamp.now(io, .boot); +const awake = std.Io.Timestamp.now(io, .awake); +``` + +Clock-specific wrapper: + +```zig +const start = std.Io.Clock.Timestamp.now(io, .boot); +``` + +## Duration + +```zig +const d1 = std.Io.Duration.fromMilliseconds(16); +const d2 = std.Io.Duration.fromSeconds(1); +const ns = d1.toNanoseconds(); +_ = .{ d2, ns }; +``` + +Formatting: + +```zig +try writer.print("{f}", .{std.Io.Duration.fromMilliseconds(250)}); +``` + +Do not use the removed `{D}` format specifier. + +## Elapsed Time + +```zig +const start = std.Io.Timestamp.now(io, .boot); + +// work + +const end = std.Io.Timestamp.now(io, .boot); +const elapsed = start.durationTo(end); +``` + +Or with clock-tagged timestamps: + +```zig +const start = std.Io.Clock.Timestamp.now(io, .boot); + +// work + +const elapsed = start.untilNow(io); +``` + +## Sleeping + +Use clock-aware durations/timestamps rather than `std.Thread.sleep` when the code should cooperate with the selected `std.Io` backend. + +```zig +try std.Io.Clock.Duration{ + .raw = std.Io.Duration.fromMilliseconds(10), + .clock = .boot, +}.sleep(io); +``` + +For low-level OS-thread code that deliberately blocks a thread and is not part of I/O task scheduling, `std.Thread.sleep` is still available. + +## Resolution + +Clock resolution may fail or return zero for unsupported clocks. + +```zig +const clock: std.Io.Clock = .boot; +const resolution = try clock.resolution(io); +if (resolution.nanoseconds == 0) { + return error.ClockUnavailable; +} +``` + +## Epoch and Calendar Helpers + +Use `std.time.epoch` for calendar conversion. Timestamps can be converted to seconds: + +```zig +const now = std.Io.Timestamp.now(io, .real); +const seconds: u64 = @intCast(now.toSeconds()); +const epoch_seconds = std.time.epoch.EpochSeconds{ .secs = seconds }; +``` + +## Application Guidance + +- Put common wall-clock timestamp reads behind a shared application helper when consistent clock selection matters. +- Store/pass `std.Io` on timer systems that sample time repeatedly. +- Avoid direct `std.time.microTimestamp`-style callsites in new code; route through the shared helper. +- Use `.boot` or `.awake` for elapsed-time measurement; use `.real` for timestamps intended to correspond to wall-clock time. + +## Review Checklist + +- Is the code using `std.Io.Timestamp.now(io, clock)` rather than old `std.time.timestamp` helpers? +- Is the clock choice documented by usage (`.real` for wall time, monotonic clocks for elapsed time)? +- Is duration formatting using `{f}` with `std.Io.Duration`? +- Does the API receive or store `io` rather than constructing a fallback locally? +- Is `std.Thread.sleep` only used for deliberate OS-thread blocking? diff --git a/references/std-treap.md b/references/std-treap.md new file mode 100644 index 0000000..877b033 --- /dev/null +++ b/references/std-treap.md @@ -0,0 +1,171 @@ +# std.Treap + +A self-balancing binary search tree using randomized priorities. Combines BST ordering with heap-based balancing for expected O(log n) operations. + +## When to Use + +- Need ordered key storage with fast lookup/insert/delete +- Require in-order iteration +- Need min/max access +- Predecessor/successor queries + +## Initialization + +```zig +const std = @import("std"); + +// Define treap with key type and comparator +const MyTreap = std.Treap(u64, std.math.order); + +var treap: MyTreap = .{}; +``` + +## Node Structure + +Nodes are user-managed (intrusive design): + +```zig +var nodes: [100]MyTreap.Node = undefined; + +// Node fields (managed by treap): +// - key: Key +// - priority: usize (random, for balancing) +// - parent: ?*Node +// - children: [2]?*Node +``` + +## Insert via Entry API + +```zig +// Get entry for a key (like a "slot" in the treap) +var entry = treap.getEntryFor(key); + +if (entry.node == null) { + // Key not present, insert new node + entry.set(&nodes[i]); // node content initialized by treap +} +``` + +## Lookup + +```zig +// Find by key +var entry = treap.getEntryFor(key); +if (entry.node) |node| { + // found, node.key == key +} + +// Get entry for existing node (O(1) if you have the node) +var entry = treap.getEntryForExisting(node); +``` + +## Remove + +```zig +var entry = treap.getEntryFor(key); +entry.set(null); // removes the node + +// Or if you have the node: +var entry = treap.getEntryForExisting(node); +entry.set(null); +``` + +## Replace + +```zig +var entry = treap.getEntryForExisting(old_node); +entry.set(&new_node); // replaces old with new (same key) +``` + +## Min/Max Access + +```zig +// Get smallest key +if (treap.getMin()) |min_node| { + std.debug.print("min key: {}\n", .{min_node.key}); +} + +// Get largest key +if (treap.getMax()) |max_node| { + std.debug.print("max key: {}\n", .{max_node.key}); +} +``` + +## Predecessor/Successor + +```zig +// Next larger key +if (node.next()) |successor| { + // successor.key > node.key +} + +// Previous smaller key +if (node.prev()) |predecessor| { + // predecessor.key < node.key +} +``` + +## In-Order Iteration + +```zig +// Iterate keys in sorted order (smallest to largest) +var iter = treap.inorderIterator(); +while (iter.next()) |node| { + std.debug.print("key: {}\n", .{node.key}); +} +``` + +## Custom Comparator + +```zig +fn compareStrings(a: []const u8, b: []const u8) std.math.Order { + return std.mem.order(u8, a, b); +} + +const StringTreap = std.Treap([]const u8, compareStrings); +``` + +## Complete Example + +```zig +const std = @import("std"); +const Treap = std.Treap(u64, std.math.order); + +pub fn main() !void { + var treap: Treap = .{}; + var nodes: [10]Treap.Node = undefined; + + // Insert keys 0-9 + for (0..10) |i| { + var entry = treap.getEntryFor(@intCast(i)); + entry.set(&nodes[i]); + } + + // Find key 5 + var entry = treap.getEntryFor(5); + if (entry.node) |node| { + std.debug.print("found: {}\n", .{node.key}); + + // Get neighbors + if (node.prev()) |p| std.debug.print("prev: {}\n", .{p.key}); + if (node.next()) |n| std.debug.print("next: {}\n", .{n.key}); + } + + // Iterate in order + var iter = treap.inorderIterator(); + while (iter.next()) |node| { + std.debug.print("{} ", .{node.key}); + } + // Output: 0 1 2 3 4 5 6 7 8 9 + + // Remove key 5 + entry.set(null); +} +``` + +## Notes + +- No allocator needed (nodes are user-managed) +- Balancing uses randomized priorities (xorshift PRNG) +- `node.priority == 0` indicates node is not in treap +- Entry API allows atomic check-and-modify patterns diff --git a/references/std-tz.md b/references/std-tz.md new file mode 100644 index 0000000..b81da3e --- /dev/null +++ b/references/std-tz.md @@ -0,0 +1,237 @@ +# std.Tz - TZif Timezone Database Parsing (Zig 0.16.0) + +Parse IANA Time Zone Database files (TZif format, RFC 8536). Used to look up UTC offsets, DST rules, and timezone abbreviations. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +When reading timezone files in Zig 0.16, use `std.Io.Dir`/`std.Io.File` and explicit `std.Io`; use `std.Io.Reader.fixed(bytes)` style patterns instead of old `std.io.fixedBufferStream`. + +## Quick Reference + +| Type | Description | +|------|-------------| +| `Tz` | Parsed timezone with transitions, time types, and leap seconds | +| `Transition` | Point in time when timezone rules change | +| `Timetype` | Timezone offset, DST flag, and abbreviation | +| `Leapsecond` | Leap second occurrence and cumulative correction | + +## Basic Usage + +```zig +const std = @import("std"); + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + const allocator = gpa.allocator(); + + // Open system timezone file + const file = try std.fs.openFileAbsolute("/usr/share/zoneinfo/America/New_York", .{}); + defer file.close(); + + // Parse TZif data + var tz = try std.Tz.parse(allocator, file.reader()); + defer tz.deinit(); + + // Access timezone information + std.debug.print("Transitions: {}\n", .{tz.transitions.len}); + std.debug.print("Footer (POSIX TZ): {s}\n", .{tz.footer orelse "(none)"}); +} +``` + +## Parsing from Embedded Data + +```zig +const std = @import("std"); + +// Embed TZif file at compile time +const tokyo_tz = @embedFile("tz/asia_tokyo.tzif"); + +pub fn main() !void { + var stream = std.io.fixedBufferStream(tokyo_tz); + var tz = try std.Tz.parse(std.heap.page_allocator, stream.reader()); + defer tz.deinit(); + + // Use timezone data... +} +``` + +## Tz Struct + +```zig +pub const Tz = struct { + allocator: std.mem.Allocator, + transitions: []const Transition, // Sorted by timestamp + timetypes: []const Timetype, + leapseconds: []const Leapsecond, + footer: ?[]const u8, // POSIX TZ string for future dates + + pub fn parse(allocator: std.mem.Allocator, reader: anytype) !Tz + pub fn deinit(self: *Tz) void +}; +``` + +## Transition + +A transition marks when timezone rules change (e.g., DST start/end): + +```zig +pub const Transition = struct { + ts: i64, // Unix timestamp (seconds since epoch) + timetype: *Timetype, // Pointer to active time type after this transition +}; +``` + +## Timetype + +Describes timezone offset and DST status: + +```zig +pub const Timetype = struct { + offset: i32, // UTC offset in seconds (e.g., -18000 for EST = UTC-5) + flags: u8, // Packed flags + name_data: [6:0]u8, // Null-terminated abbreviation (e.g., "EST", "PDT") + + pub fn name(self: *const Timetype) [:0]const u8 // Get abbreviation + pub fn isDst(self: Timetype) bool // Is daylight saving time? + pub fn standardTimeIndicator(self: Timetype) bool + pub fn utIndicator(self: Timetype) bool +}; +``` + +## Leapsecond + +Leap second corrections for TAI-UTC: + +```zig +pub const Leapsecond = struct { + occurrence: i48, // Unix timestamp when leap second occurs + correction: i16, // Cumulative TAI-UTC difference +}; +``` + +## Look Up Current Timezone Offset + +```zig +fn getUtcOffset(tz: *const std.Tz, unix_timestamp: i64) i32 { + // Find the last transition before or at the given timestamp + var result: ?*const std.Timetype = null; + + for (tz.transitions) |t| { + if (t.ts <= unix_timestamp) { + result = t.timetype; + } else { + break; + } + } + + // Return offset or default to first timetype + if (result) |tt| { + return tt.offset; + } else if (tz.timetypes.len > 0) { + return tz.timetypes[0].offset; + } + return 0; +} + +// Usage +const offset = getUtcOffset(&tz, std.time.timestamp()); +const local_time = unix_timestamp + offset; +``` + +## Check if DST is Active + +```zig +fn isDstActive(tz: *const std.Tz, unix_timestamp: i64) bool { + for (tz.transitions) |t| { + if (t.ts <= unix_timestamp) { + if (t.timetype.isDst()) return true; + } else { + break; + } + } + return false; +} +``` + +## Get Timezone Abbreviation + +```zig +fn getTimezoneAbbrev(tz: *const std.Tz, unix_timestamp: i64) []const u8 { + var result: ?*const std.Timetype = null; + + for (tz.transitions) |t| { + if (t.ts <= unix_timestamp) { + result = t.timetype; + } else { + break; + } + } + + if (result) |tt| { + return tt.name(); + } else if (tz.timetypes.len > 0) { + return tz.timetypes[0].name(); + } + return "UTC"; +} + +// Returns "EST", "EDT", "PST", "PDT", "JST", etc. +``` + +## List All Transitions + +```zig +fn printTransitions(tz: *const std.Tz) void { + for (tz.transitions) |t| { + std.debug.print("{d}: {s} (offset {d}s, DST: {})\n", .{ + t.ts, + t.timetype.name(), + t.timetype.offset, + t.timetype.isDst(), + }); + } +} +``` + +## Parse Errors + +| Error | Cause | +|-------|-------| +| `error.BadHeader` | Invalid TZif magic bytes (not "TZif") | +| `error.BadVersion` | Unsupported TZif version (only 0, 2, 3 supported) | +| `error.Malformed` | RFC 8536 validation failure | +| `error.OverlargeFooter` | POSIX TZ string exceeds 128 bytes | + +## System Timezone Paths + +| Platform | Path | +|----------|------| +| Linux/BSD | `/usr/share/zoneinfo//` | +| macOS | `/var/db/timezone/zoneinfo//` | + +Common timezone identifiers: +- `America/New_York`, `America/Los_Angeles`, `America/Chicago` +- `Europe/London`, `Europe/Paris`, `Europe/Berlin` +- `Asia/Tokyo`, `Asia/Shanghai`, `Asia/Kolkata` +- `UTC`, `Etc/GMT` + +## POSIX TZ Footer + +Modern TZif files (v2+) include a POSIX TZ string in the footer for calculating offsets beyond the last transition: + +```zig +if (tz.footer) |posix_tz| { + // e.g., "EST5EDT,M3.2.0,M11.1.0" for US Eastern + std.debug.print("POSIX TZ: {s}\n", .{posix_tz}); +} +``` + +## Notes + +- Allocates memory for transitions, timetypes, leapseconds, and footer +- Call `deinit()` to free allocated memory +- Supports TZif version 0 (legacy 32-bit), 2, and 3 (64-bit timestamps) +- Timezone abbreviations are limited to 6 characters (POSIX compliance) +- Transition timestamps are Unix epoch seconds (signed i64) +- Offset is in seconds, negative for west of UTC (e.g., -18000 = UTC-5) diff --git a/references/std-unicode.md b/references/std-unicode.md new file mode 100644 index 0000000..f0f1e9e --- /dev/null +++ b/references/std-unicode.md @@ -0,0 +1,267 @@ +# std.unicode + +Unicode encoding/decoding for UTF-8, UTF-16, and WTF-8/WTF-16. For ASCII-only operations, use `std.ascii`. + +## Quick Reference + +| Task | Function | +|------|----------| +| Validate UTF-8 | `utf8ValidateSlice(s)` | +| Count codepoints | `utf8CountCodepoints(s)` | +| Iterate codepoints | `Utf8View.init(s)` then `.iterator()` | +| UTF-8 ↔ UTF-16 | `utf8ToUtf16LeAlloc`, `utf16LeToUtf8Alloc` | +| Encode codepoint | `utf8Encode(codepoint, buf)` | + +## UTF-8 Validation + +```zig +const std = @import("std"); +const unicode = std.unicode; + +// Check if string is valid UTF-8 +if (unicode.utf8ValidateSlice(input)) { + // valid UTF-8 +} + +// Count codepoints (not bytes) +const count = try unicode.utf8CountCodepoints("héllo"); // 5 + +// Check if codepoint is valid +unicode.utf8ValidCodepoint('é') // true +unicode.utf8ValidCodepoint(0xD800) // false (surrogate) +unicode.utf8ValidCodepoint(0x110000) // false (too large) +``` + +## Iterating Codepoints + +```zig +// Create validated view +const view = try unicode.Utf8View.init("héllo 世界"); +var it = view.iterator(); + +while (it.nextCodepoint()) |codepoint| { + // codepoint is u21: 'h', 'é', 'l', 'l', 'o', ' ', '世', '界' +} + +// Or get UTF-8 slices +var it2 = view.iterator(); +while (it2.nextCodepointSlice()) |slice| { + // slice is []const u8: "h", "é", "l", "l", "o", " ", "世", "界" +} + +// Peek ahead without advancing +const next3 = it.peek(3); // next 3 codepoints as UTF-8 bytes + +// Comptime-validated view +const view = unicode.Utf8View.initComptime("hello"); + +// Unchecked (when you know it's valid) +const view = unicode.Utf8View.initUnchecked(trusted_utf8); +``` + +## Encoding/Decoding Codepoints + +```zig +// Encode codepoint to UTF-8 +var buf: [4]u8 = undefined; +const len = try unicode.utf8Encode('é', &buf); // len = 2 +// buf[0..len] contains UTF-8 bytes + +// Comptime encoding (returns fixed-size array) +const bytes = unicode.utf8EncodeComptime('世'); // [3]u8 + +// Get UTF-8 sequence length for a codepoint +const len = try unicode.utf8CodepointSequenceLength('世'); // 3 + +// Get sequence length from first byte +const len = try unicode.utf8ByteSequenceLength(0xE4); // 3 (for 3-byte sequence) +``` + +## UTF-8 ↔ UTF-16 Conversion + +### UTF-8 to UTF-16LE (Allocating) + +```zig +// Returns []u16 +const utf16 = try unicode.utf8ToUtf16LeAlloc(allocator, "hello 世界"); +defer allocator.free(utf16); + +// Returns [:0]u16 (null-terminated, for Windows APIs) +const utf16z = try unicode.utf8ToUtf16LeAllocZ(allocator, "hello"); +defer allocator.free(utf16z); +``` + +### UTF-16LE to UTF-8 (Allocating) + +```zig +// Returns []u8 +const utf8 = try unicode.utf16LeToUtf8Alloc(allocator, utf16_data); +defer allocator.free(utf8); + +// Returns [:0]u8 (null-terminated) +const utf8z = try unicode.utf16LeToUtf8AllocZ(allocator, utf16_data); +defer allocator.free(utf8z); +``` + +### Non-Allocating Conversion + +```zig +// UTF-8 to UTF-16LE (caller provides buffer) +var utf16_buf: [128]u16 = undefined; +const len = try unicode.utf8ToUtf16Le(&utf16_buf, "hello"); +const utf16 = utf16_buf[0..len]; + +// UTF-16LE to UTF-8 (caller provides buffer) +var utf8_buf: [256]u8 = undefined; +const len = try unicode.utf16LeToUtf8(&utf8_buf, utf16_data); +const utf8 = utf8_buf[0..len]; +``` + +### ArrayList Conversion + +```zig +var list = std.ArrayList(u16).empty; +defer list.deinit(allocator); +try unicode.utf8ToUtf16LeArrayList(&list, "hello"); + +var list8 = std.ArrayList(u8).empty; +defer list8.deinit(allocator); +try unicode.utf16LeToUtf8ArrayList(&list8, utf16_data); +``` + +### Comptime String Literals + +```zig +// Convert UTF-8 literal to UTF-16LE at comptime +const utf16 = unicode.utf8ToUtf16LeStringLiteral("hello"); +// Type: *const [5:0]u16 (null-terminated) + +// Calculate UTF-16 length +const len = try unicode.calcUtf16LeLen("hello 世界"); // 8 (code units) +``` + +## UTF-16 Utilities + +```zig +// Check surrogate code units +unicode.utf16IsHighSurrogate(0xD800) // true (0xD800-0xDBFF) +unicode.utf16IsLowSurrogate(0xDC00) // true (0xDC00-0xDFFF) + +// Decode surrogate pair +const codepoint = try unicode.utf16DecodeSurrogatePair(&[_]u16{ 0xD801, 0xDC37 }); +// codepoint = 0x10437 + +// UTF-16 sequence length for codepoint +const len = try unicode.utf16CodepointSequenceLength(0x10000); // 2 + +// Iterate UTF-16LE +var it = unicode.Utf16LeIterator.init(utf16_slice); +while (try it.nextCodepoint()) |cp| { + // cp is u21 +} +``` + +## WTF-8/WTF-16 (Windows Encoding) + +WTF-8 is like UTF-8 but allows unpaired surrogates (for Windows compatibility). + +```zig +// Validate WTF-8 (allows surrogates) +unicode.wtf8ValidateSlice(data) // bool + +// WTF-8 iteration +const view = try unicode.Wtf8View.init(wtf8_data); +var it = view.iterator(); +while (it.nextCodepoint()) |cp| { + // cp might be a surrogate codepoint +} + +// WTF-8 ↔ WTF-16 conversion +const wtf8 = try unicode.wtf16LeToWtf8Alloc(allocator, wtf16_data); +const wtf16 = try unicode.wtf8ToWtf16LeAlloc(allocator, wtf8_data); + +// Convert WTF-8 to UTF-8 (lossy - replaces surrogates with U+FFFD) +const utf8 = try unicode.wtf8ToUtf8LossyAlloc(allocator, wtf8_data); + +// In-place lossy conversion +try unicode.wtf8ToUtf8Lossy(buffer, wtf8_data); +``` + +## Formatting + +```zig +// Format potentially ill-formed UTF-8 (replaces invalid sequences with U+FFFD) +try stdout.print("{f}", .{unicode.fmtUtf8(possibly_invalid_utf8)}); + +// Format UTF-16LE as UTF-8 (replaces unpaired surrogates with U+FFFD) +try stdout.print("{f}", .{unicode.fmtUtf16Le(utf16_data)}); +``` + +## Constants + +```zig +unicode.replacement_character // U+FFFD (u21) +unicode.replacement_character_utf8 // [3]u8 for U+FFFD +``` + +## Common Patterns + +### Safe string processing +```zig +fn processText(input: []const u8) !void { + if (!unicode.utf8ValidateSlice(input)) { + return error.InvalidUtf8; + } + const view = unicode.Utf8View.initUnchecked(input); + var it = view.iterator(); + while (it.nextCodepoint()) |cp| { + // process each codepoint + } +} +``` + +### Windows API interop +```zig +fn callWindowsApi(path: []const u8) !void { + const wide = try unicode.utf8ToUtf16LeAllocZ(allocator, path); + defer allocator.free(wide); + // wide is [:0]u16, ready for Windows API + windows.CreateFileW(wide.ptr, ...); +} +``` + +### Grapheme-aware truncation +```zig +fn truncateCodepoints(s: []const u8, max_codepoints: usize) ![]const u8 { + const view = try unicode.Utf8View.init(s); + var it = view.iterator(); + var count: usize = 0; + var end: usize = 0; + while (it.nextCodepointSlice()) |slice| { + if (count >= max_codepoints) break; + end = it.i; + count += 1; + } + return s[0..end]; +} +``` + +## Error Types + +| Error | Meaning | +|-------|---------| +| `InvalidUtf8` | Input is not valid UTF-8 | +| `InvalidWtf8` | Input is not valid WTF-8 | +| `Utf8InvalidStartByte` | Invalid first byte in sequence | +| `Utf8ExpectedContinuation` | Missing continuation byte | +| `Utf8OverlongEncoding` | Overlong encoding detected | +| `Utf8EncodesSurrogateHalf` | Surrogate in UTF-8 (use WTF-8) | +| `CodepointTooLarge` | Codepoint > 0x10FFFF | + +## Notes + +- UTF-8 uses 1-4 bytes per codepoint +- UTF-16 uses 1-2 code units (2-4 bytes) per codepoint +- Surrogates (U+D800-U+DFFF) are invalid in UTF-8 but valid in WTF-8 +- Use `fmtUtf8`/`fmtUtf16Le` for safe display of potentially invalid data +- Windows uses UTF-16LE (little-endian) for wide strings diff --git a/references/std-uri.md b/references/std-uri.md new file mode 100644 index 0000000..ad8f6e7 --- /dev/null +++ b/references/std-uri.md @@ -0,0 +1,484 @@ +# std.Uri Reference (Zig 0.16.0) + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +URI parsing remains mostly independent of the I/O migration. For network or file operations derived from URIs, pass/use `std.Io`. + +URI parsing and formatting conforming to RFC 3986, with percent-encoding/decoding and resolution support. + +## Table of Contents +- [Parsing URIs](#parsing-uris) +- [URI Components](#uri-components) +- [Formatting URIs](#formatting-uris) +- [Percent Encoding/Decoding](#percent-encodingdecoding) +- [URI Resolution](#uri-resolution) +- [Common Patterns](#common-patterns) + +## Parsing URIs + +### Basic Parsing + +```zig +const std = @import("std"); + +pub fn main() !void { + const uri = try std.Uri.parse("https://user:pass@example.com:8080/path?query=1#fragment"); + + std.debug.print("Scheme: {s}\n", .{uri.scheme}); // "https" + std.debug.print("User: {s}\n", .{uri.user.?.percent_encoded}); // "user" + std.debug.print("Password: {s}\n", .{uri.password.?.percent_encoded}); // "pass" + std.debug.print("Host: {s}\n", .{uri.host.?.percent_encoded}); // "example.com" + std.debug.print("Port: {d}\n", .{uri.port.?}); // 8080 + std.debug.print("Path: {s}\n", .{uri.path.percent_encoded}); // "/path" + std.debug.print("Query: {s}\n", .{uri.query.?.percent_encoded}); // "query=1" + std.debug.print("Fragment: {s}\n", .{uri.fragment.?.percent_encoded}); // "fragment" +} +``` + +### Parse After Scheme + +For URIs where scheme is already known (e.g., HTTP redirects): + +```zig +// Parse "//example.com/path" as an HTTP URI +const uri = try std.Uri.parseAfterScheme("http", "//example.com/path"); +std.debug.print("Scheme: {s}, Host: {s}\n", .{ + uri.scheme, + uri.host.?.percent_encoded, +}); +``` + +### Error Handling + +```zig +const uri = std.Uri.parse(input) catch |err| switch (err) { + error.UnexpectedCharacter => { + std.debug.print("Invalid character in URI\n", .{}); + return err; + }, + error.InvalidFormat => { + std.debug.print("Malformed URI\n", .{}); + return err; + }, + error.InvalidPort => { + std.debug.print("Port not a valid u16\n", .{}); + return err; + }, +}; +``` + +## URI Components + +### Uri Struct + +```zig +const Uri = struct { + scheme: []const u8, + user: ?Component = null, + password: ?Component = null, + host: ?Component = null, + port: ?u16 = null, + path: Component = Component.empty, + query: ?Component = null, + fragment: ?Component = null, + + pub const host_name_max = 255; +}; +``` + +### Component Union + +Components can be raw (needs encoding) or already percent-encoded: + +```zig +const Component = union(enum) { + /// Needs percent encoding before use in URI + raw: []const u8, + /// Already percent-encoded, can be used directly + percent_encoded: []const u8, + + pub const empty: Component = .{ .percent_encoded = "" }; + + pub fn isEmpty(component: Component) bool; +}; +``` + +### Getting Host + +```zig +var buffer: [std.Uri.host_name_max]u8 = undefined; +const host = uri.getHost(&buffer) catch |err| switch (err) { + error.UriMissingHost => return error.NoHost, + error.UriHostTooLong => return error.HostTooLong, +}; +``` + +With allocation: + +```zig +var arena = std.heap.ArenaAllocator.init(allocator); +defer arena.deinit(); + +const host = try uri.getHostAlloc(arena.allocator()); +``` + +### Creating URIs Programmatically + +```zig +const uri: std.Uri = .{ + .scheme = "https", + .host = .{ .raw = "example.com" }, + .port = 8080, + .path = .{ .raw = "/api/users" }, + .query = .{ .raw = "page=1&limit=10" }, +}; +``` + +### Component Methods + +```zig +const component: std.Uri.Component = .{ .percent_encoded = "hello%20world" }; + +// Check if empty +if (component.isEmpty()) { + // ... +} + +// Get raw (decoded) value with buffer +var buf: [256]u8 = undefined; +const raw = try component.toRaw(&buf); // "hello world" + +// Get raw (decoded) value with allocation (only allocates if needed) +const raw_alloc = try component.toRawMaybeAlloc(allocator); // "hello world" +``` + +## Formatting URIs + +### Full URI + +```zig +const uri = try std.Uri.parse("https://example.com:8080/path?query#frag"); + +var buf: [1024]u8 = undefined; +var writer: std.Io.Writer = .fixed(&buf); +try uri.format(&writer); +const formatted = writer.buffered(); // "https://example.com:8080/path?query#frag" +``` + +### Selective Formatting + +```zig +var buf: [1024]u8 = undefined; +var writer: std.Io.Writer = .fixed(&buf); + +// Format only specific parts +try std.fmt.format(&writer, "{f}", .{uri.fmt(.{ + .scheme = true, + .authority = true, + .path = true, + .query = true, + .fragment = false, // omit fragment +})}); +``` + +### Format Flags + +```zig +const Flags = struct { + scheme: bool = false, // Include scheme (e.g., "https:") + authentication: bool = false, // Include user:password@ (requires authority) + authority: bool = false, // Include host and port + path: bool = false, // Include path + query: bool = false, // Include ?query (requires path) + fragment: bool = false, // Include #fragment (requires path) + port: bool = true, // Include :port (requires authority) + + pub const all: Flags = .{ + .scheme = true, + .authentication = true, + .authority = true, + .path = true, + .query = true, + .fragment = true, + .port = true, + }; +}; +``` + +### Component Formatting Methods + +```zig +var buf: [256]u8 = undefined; +var writer: std.Io.Writer = .fixed(&buf); + +const component: std.Uri.Component = .{ .raw = "hello world" }; + +// Different encoding rules for different URI parts +try component.formatEscaped(&writer); // General: unreserved chars only +try component.formatUser(&writer); // User: unreserved + sub-delims +try component.formatPassword(&writer); // Password: user chars + ':' +try component.formatHost(&writer); // Host: password chars + '[' + ']' +try component.formatPath(&writer); // Path: user chars + '/' + ':' + '@' +try component.formatQuery(&writer); // Query: path chars + '?' +try component.formatFragment(&writer); // Fragment: same as query + +// Get raw (decoded) output +try component.formatRaw(&writer); // Decodes percent-encoded chars +``` + +## Percent Encoding/Decoding + +### Decode In Place + +```zig +var buffer = "hello%20world%21".*; +const decoded = std.Uri.percentDecodeInPlace(&buffer); +// decoded == "hello world!" +``` + +### Decode Backwards (Safe for Aliasing) + +```zig +const input = "%48%65%6C%6C%6F"; +var output: [5]u8 = undefined; +const decoded = std.Uri.percentDecodeBackwards(&output, input); +// decoded == "Hello" +``` + +### Encode with Component + +```zig +var buf: [256]u8 = undefined; +var writer: std.Io.Writer = .fixed(&buf); + +// Raw component will be percent-encoded when formatted +const component: std.Uri.Component = .{ .raw = "hello world!" }; +try component.formatPath(&writer); +const encoded = writer.buffered(); // "hello%20world%21" +``` + +### Custom Percent Encoding + +```zig +var buf: [256]u8 = undefined; +var writer: std.Io.Writer = .fixed(&buf); + +// Encode with custom character validation +std.Uri.Component.percentEncode(&writer, "custom data", struct { + fn isValid(c: u8) bool { + return std.ascii.isAlphanumeric(c); + } +}.isValid); +``` + +## URI Resolution + +### Resolve Relative URI + +Resolves a relative URI against a base URI per RFC 3986 Section 5: + +```zig +const base = try std.Uri.parse("http://a/b/c/d;p?q"); + +var aux_buf: [1024]u8 = undefined; +var aux_slice: []u8 = &aux_buf; + +// Copy relative URI to start of aux_buf +const relative = "../g"; +@memcpy(aux_buf[0..relative.len], relative); + +const resolved = try std.Uri.resolveInPlace(base, relative.len, &aux_slice); +// resolved.path.percent_encoded == "/a/g" +``` + +### Resolution Examples (RFC 3986) + +| Base: `http://a/b/c/d;p?q` | Reference | Result | +|---------------------------|-----------|--------| +| | `g` | `http://a/b/c/g` | +| | `./g` | `http://a/b/c/g` | +| | `g/` | `http://a/b/c/g/` | +| | `/g` | `http://a/g` | +| | `//g` | `http://g` | +| | `?y` | `http://a/b/c/d;p?y` | +| | `g?y` | `http://a/b/c/g?y` | +| | `#s` | `http://a/b/c/d;p?q#s` | +| | `g#s` | `http://a/b/c/g#s` | +| | `../` | `http://a/b/` | +| | `../g` | `http://a/b/g` | +| | `../../g` | `http://a/g` | + +## Common Patterns + +### Extract Query Parameters + +```zig +fn getQueryParam(uri: std.Uri, key: []const u8) ?[]const u8 { + const query = uri.query orelse return null; + const query_str = query.percent_encoded; + + var iter = std.mem.splitScalar(u8, query_str, '&'); + while (iter.next()) |pair| { + if (std.mem.indexOfScalar(u8, pair, '=')) |eq_pos| { + if (std.mem.eql(u8, pair[0..eq_pos], key)) { + return pair[eq_pos + 1 ..]; + } + } else if (std.mem.eql(u8, pair, key)) { + return ""; // Key exists with no value + } + } + return null; +} + +// Usage +const uri = try std.Uri.parse("https://example.com?name=alice&age=30"); +const name = getQueryParam(uri, "name"); // "alice" +``` + +### Build URL with Query Parameters + +```zig +fn buildUrl(allocator: Allocator, base: []const u8, params: []const [2][]const u8) ![]u8 { + var result: std.ArrayList(u8) = .empty; + defer result.deinit(allocator); + + try result.appendSlice(allocator, base); + + for (params, 0..) |param, i| { + try result.append(allocator, if (i == 0) '?' else '&'); + + // Encode key + for (param[0]) |c| { + if (std.Uri.isUnreserved(c)) { + try result.append(allocator, c); + } else { + try result.appendSlice(allocator, try std.fmt.allocPrint(allocator, "%{X:0>2}", .{c})); + } + } + + try result.append(allocator, '='); + + // Encode value + for (param[1]) |c| { + if (std.Uri.isUnreserved(c)) { + try result.append(allocator, c); + } else { + try result.appendSlice(allocator, try std.fmt.allocPrint(allocator, "%{X:0>2}", .{c})); + } + } + } + + return result.toOwnedSlice(allocator); +} +``` + +### Normalize URI + +```zig +fn normalizeUri(allocator: Allocator, uri_str: []const u8) ![]u8 { + const uri = try std.Uri.parse(uri_str); + + var buf: [4096]u8 = undefined; + var writer: std.Io.Writer = .fixed(&buf); + + // Format with all components to normalize encoding + try uri.format(&writer); + + return try allocator.dupe(u8, writer.buffered()); +} +``` + +### Validate URI + +```zig +fn isValidUri(str: []const u8) bool { + _ = std.Uri.parse(str) catch return false; + return true; +} + +fn isValidHttpUri(str: []const u8) bool { + const uri = std.Uri.parse(str) catch return false; + return std.mem.eql(u8, uri.scheme, "http") or std.mem.eql(u8, uri.scheme, "https"); +} +``` + +### Join Path Segments + +```zig +fn joinPath(allocator: Allocator, base_uri: std.Uri, segments: []const []const u8) !std.Uri { + var path: std.ArrayList(u8) = .empty; + defer path.deinit(allocator); + + // Start with base path (remove trailing slash if any) + const base_path = base_uri.path.percent_encoded; + if (base_path.len > 0 and base_path[base_path.len - 1] == '/') { + try path.appendSlice(allocator, base_path[0 .. base_path.len - 1]); + } else { + try path.appendSlice(allocator, base_path); + } + + // Append segments + for (segments) |seg| { + try path.append(allocator, '/'); + try path.appendSlice(allocator, seg); + } + + var result = base_uri; + result.path = .{ .percent_encoded = try path.toOwnedSlice(allocator) }; + result.query = null; + result.fragment = null; + return result; +} +``` + +### Extract Base URL + +```zig +fn getBaseUrl(allocator: Allocator, uri: std.Uri) ![]u8 { + var buf: [1024]u8 = undefined; + var writer: std.Io.Writer = .fixed(&buf); + + try std.fmt.format(&writer, "{f}", .{uri.fmt(.{ + .scheme = true, + .authority = true, + .port = true, + })}); + + return try allocator.dupe(u8, writer.buffered()); +} + +// Usage +const uri = try std.Uri.parse("https://example.com:8080/path?query#frag"); +const base = try getBaseUrl(allocator, uri); // "https://example.com:8080" +``` + +## Error Types + +### ParseError + +```zig +pub const ParseError = error{ + UnexpectedCharacter, // Invalid character in URI component + InvalidFormat, // Malformed URI structure + InvalidPort, // Port not a valid u16 +}; +``` + +### ResolveInPlaceError + +```zig +pub const ResolveInPlaceError = ParseError || error{ + NoSpaceLeft, // Auxiliary buffer too small +}; +``` + +### Component Errors + +```zig +// getHost errors +error.UriMissingHost // URI has no host component +error.UriHostTooLong // Host exceeds host_name_max (255) + +// toRaw errors +error.NoSpaceLeft // Buffer too small for decoded string +``` diff --git a/references/std-zig.md b/references/std-zig.md new file mode 100644 index 0000000..a0d6f2c --- /dev/null +++ b/references/std-zig.md @@ -0,0 +1,724 @@ +# std.zig - Zig Compiler Utilities Reference (Zig 0.16.0) + +Utilities for parsing, tokenizing, and working with Zig source code. Used for tooling, linters, formatters, and custom analysis. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +When tooling reads/writes files or buffers output in Zig 0.16, use `std.Io.Reader`, `std.Io.Writer`, `std.Io.Dir`, and explicit `std.Io`. When searching source text, prefer `std.mem.find*` and `std.mem.cut*` APIs. + +## Table of Contents +- [Quick Start](#quick-start) +- [Tokenizer](#tokenizer) +- [AST Parsing](#ast-parsing) +- [AST Navigation](#ast-navigation) +- [AST Full Node Types](#ast-full-node-types) +- [Error Handling](#error-handling) +- [String/Number Literals](#stringnumber-literals) +- [Identifier Formatting](#identifier-formatting) +- [Source Utilities](#source-utilities) + +## Quick Start + +### Parse and Analyze Zig Source +```zig +const std = @import("std"); + +pub fn analyzeSource(allocator: std.mem.Allocator, source: [:0]const u8) !void { + // Parse source into AST + var tree = try std.zig.Ast.parse(allocator, source, .zig); + defer tree.deinit(allocator); + + // Check for parse errors + if (tree.errors.len > 0) { + for (tree.errors) |err| { + var buf: [256]u8 = undefined; + var w: std.io.Writer = .fixed(&buf); + try tree.renderError(err, &w); + std.debug.print("Error: {s}\n", .{w.buffered()}); + } + return error.ParseError; + } + + // Iterate root declarations + for (tree.rootDecls()) |decl| { + const tag = tree.nodeTag(decl); + std.debug.print("Declaration: {s}\n", .{@tagName(tag)}); + } +} +``` + +### Parse ZON Data +```zig +var tree = try std.zig.Ast.parse(allocator, zon_source, .zon); +defer tree.deinit(allocator); +// Root node contains the ZON expression +``` + +## Tokenizer + +### std.zig.Tokenizer + +Converts source text into tokens. + +```zig +const source: [:0]const u8 = "const x = 42;"; +var tokenizer = std.zig.Tokenizer.init(source); + +while (true) { + const token = tokenizer.next(); + if (token.tag == .eof) break; + + const lexeme = source[token.loc.start..token.loc.end]; + std.debug.print("{s}: '{s}'\n", .{ @tagName(token.tag), lexeme }); +} +// Output: +// keyword_const: 'const' +// identifier: 'x' +// equal: '=' +// number_literal: '42' +// semicolon: ';' +``` + +### Token Structure +```zig +const Token = struct { + tag: Tag, + loc: Loc, + + const Loc = struct { + start: usize, + end: usize, + }; +}; +``` + +### Common Token Tags +```zig +.identifier // Variable/function names +.number_literal // 42, 0xff, 3.14 +.string_literal // "hello" +.char_literal // 'a' +.builtin // @import, @as +.keyword_const // const +.keyword_var // var +.keyword_fn // fn +.keyword_pub // pub +.keyword_if // if +.keyword_for // for +.keyword_while // while +.keyword_return // return +.equal // = +.equal_equal // == +.l_paren // ( +.r_paren // ) +.l_brace // { +.r_brace // } +.semicolon // ; +.comma // , +.period // . +.doc_comment // /// comment +.eof // End of file +``` + +### Keyword Lookup +```zig +// Check if identifier is a keyword +if (std.zig.Token.getKeyword("const")) |tag| { + // tag == .keyword_const +} +``` + +## AST Parsing + +### std.zig.Ast + +Abstract Syntax Tree for Zig source code. + +```zig +const Ast = struct { + source: [:0]const u8, // Original source + tokens: TokenList.Slice, // All tokens + nodes: NodeList.Slice, // All AST nodes + extra_data: []u32, // Additional node data + mode: Mode, // .zig or .zon + errors: []const Error, // Parse errors +}; +``` + +### Parsing +```zig +// Parse Zig source +var tree = try std.zig.Ast.parse(allocator, source, .zig); +defer tree.deinit(allocator); + +// Parse ZON +var zon_tree = try std.zig.Ast.parse(allocator, zon_source, .zon); +defer zon_tree.deinit(allocator); +``` + +### Rendering (Formatting) +```zig +// Format AST back to source +const formatted = try tree.renderAlloc(allocator); +defer allocator.free(formatted); + +// Or render to writer +var buf: [8192]u8 = undefined; +var writer = std.fs.File.stdout().writer(&buf); +try tree.render(allocator, &writer.interface, .{}); +try writer.interface.flush(); +``` + +## AST Navigation + +### Basic Node Access +```zig +// Get node tag (what kind of node) +const tag = tree.nodeTag(node_index); + +// Get main token for node +const main_token = tree.nodeMainToken(node_index); + +// Get node data +const data = tree.nodeData(node_index); + +// Get token slice (the actual text) +const text = tree.tokenSlice(token_index); + +// Get token tag +const token_tag = tree.tokenTag(token_index); +``` + +### Root Declarations +```zig +// Get all top-level declarations +for (tree.rootDecls()) |decl| { + switch (tree.nodeTag(decl)) { + .fn_decl => handleFunction(tree, decl), + .global_var_decl, .simple_var_decl => handleVariable(tree, decl), + .container_decl, .container_decl_two => handleStruct(tree, decl), + else => {}, + } +} +``` + +### Token Location +```zig +// Get line/column from token +const loc = tree.tokenLocation(0, token_index); +std.debug.print("Line {d}, Column {d}\n", .{ loc.line + 1, loc.column + 1 }); +``` + +### Node Span (First/Last Token) +```zig +// Get first and last token of a node (for error highlighting) +const first = tree.firstToken(node); +const last = tree.lastToken(node); + +// Get source text for entire node +const node_source = tree.getNodeSource(node); +``` + +## AST Full Node Types + +The AST uses compact representations. Use `full*` methods to get structured access. + +### Function Declarations +```zig +var buf: [1]std.zig.Ast.Node.Index = undefined; +if (tree.fullFnProto(&buf, node)) |fn_proto| { + // Name + if (fn_proto.name_token) |name| { + std.debug.print("Function: {s}\n", .{tree.tokenSlice(name)}); + } + + // Parameters + var it = fn_proto.iterate(&tree); + while (it.next()) |param| { + if (param.name_token) |name| { + std.debug.print(" Param: {s}\n", .{tree.tokenSlice(name)}); + } + } + + // Return type + if (fn_proto.ast.return_type.unwrap()) |ret_type| { + // Process return type node + } +} +``` + +### Variable Declarations +```zig +fn processVarDecl(tree: *const std.zig.Ast, node: std.zig.Ast.Node.Index) void { + const var_decl = switch (tree.nodeTag(node)) { + .global_var_decl => tree.globalVarDecl(node), + .local_var_decl => tree.localVarDecl(node), + .simple_var_decl => tree.simpleVarDecl(node), + .aligned_var_decl => tree.alignedVarDecl(node), + else => return, + }; + + // Name is token after mut_token (var/const) + const name = tree.tokenSlice(var_decl.ast.mut_token + 1); + std.debug.print("Variable: {s}\n", .{name}); + + // Type annotation + if (var_decl.ast.type_node.unwrap()) |type_node| { + // Process type + } + + // Initializer + if (var_decl.ast.init_node.unwrap()) |init_node| { + // Process initializer + } + + // Visibility + if (var_decl.visib_token != null) { + // pub + } +} +``` + +### Container (Struct/Enum/Union) +```zig +var buf: [2]std.zig.Ast.Node.Index = undefined; +if (tree.fullContainerDecl(&buf, node)) |container| { + // Get container keyword (struct/enum/union) + const keyword = tree.tokenSlice(container.ast.main_token); + + // Iterate members + for (container.ast.members) |member| { + switch (tree.nodeTag(member)) { + .container_field, .container_field_init, .container_field_align => { + const field = tree.containerField(member); + const name = tree.tokenSlice(field.ast.main_token); + std.debug.print(" Field: {s}\n", .{name}); + }, + .fn_decl => { + // Method + }, + else => {}, + } + } +} +``` + +### If Expressions +```zig +if (tree.nodeTag(node) == .@"if" or tree.nodeTag(node) == .if_simple) { + const if_full = if (tree.nodeTag(node) == .@"if") + tree.ifFull(node) + else + tree.ifSimple(node); + + // Condition + const cond = if_full.ast.cond_expr; + + // Then branch + const then_expr = if_full.ast.then_expr; + + // Else branch (if present) + if (if_full.ast.else_expr.unwrap()) |else_expr| { + // Process else + } + + // Payload capture (if |x|) + if (if_full.payload_token) |payload| { + std.debug.print("Payload: {s}\n", .{tree.tokenSlice(payload)}); + } +} +``` + +### While/For Loops +```zig +if (tree.fullWhile(node)) |while_loop| { + // Condition + const cond = while_loop.ast.cond_expr; + + // Continue expression (: (i += 1)) + if (while_loop.ast.cont_expr.unwrap()) |cont| { + // Process continue expr + } + + // Payload (|item|) + if (while_loop.payload_token) |payload| { + std.debug.print("Payload: {s}\n", .{tree.tokenSlice(payload)}); + } + + // Label + if (while_loop.label_token) |label| { + std.debug.print("Label: {s}\n", .{tree.tokenSlice(label)}); + } +} + +if (tree.fullFor(node)) |for_loop| { + // Inputs (iterables) + for (for_loop.ast.inputs) |input| { + // Process each iterable + } + + // Body + const body = for_loop.ast.then_expr; + + // Else branch + if (for_loop.ast.else_expr.unwrap()) |else_expr| { + // Process else + } +} +``` + +### Switch +```zig +if (tree.fullSwitch(node)) |switch_full| { + // Condition being switched on + const cond = switch_full.ast.condition; + + // Cases + for (switch_full.ast.cases) |case_node| { + if (tree.fullSwitchCase(case_node)) |case| { + // Case values + for (case.ast.values) |val| { + // Each case value + } + + // Case body + const body = case.ast.target_expr; + + // Capture (|x|) + if (case.payload_token) |payload| { + std.debug.print("Capture: {s}\n", .{tree.tokenSlice(payload)}); + } + } + } +} +``` + +### Function Calls +```zig +var buf: [1]std.zig.Ast.Node.Index = undefined; +if (tree.fullCall(&buf, node)) |call| { + // Callee (function being called) + const callee = call.ast.fn_expr; + + // Arguments + for (call.ast.params) |arg| { + // Process each argument + } +} +``` + +### Struct/Array Init +```zig +var buf: [2]std.zig.Ast.Node.Index = undefined; + +// Struct init: .{ .x = 1, .y = 2 } or Type{ .x = 1 } +if (tree.fullStructInit(&buf, node)) |init| { + // Type (if explicit) + if (init.ast.type_expr.unwrap()) |type_node| { + // Process type + } + + // Field initializers + for (init.ast.fields) |field| { + // Each field init node + } +} + +// Array init: .{ 1, 2, 3 } or [3]u8{ 1, 2, 3 } +if (tree.fullArrayInit(&buf, node)) |init| { + // Type (if explicit) + if (init.ast.type_expr.unwrap()) |type_node| { + // Process type + } + + // Elements + for (init.ast.elements) |elem| { + // Process each element + } +} +``` + +### Slices +```zig +if (tree.fullSlice(node)) |slice| { + // Sliced expression + const slicee = slice.ast.sliced; + + // Start index + const start = slice.ast.start; + + // End index (if present) + if (slice.ast.end.unwrap()) |end| { + // Process end + } + + // Sentinel (if present) + if (slice.ast.sentinel.unwrap()) |sentinel| { + // Process sentinel + } +} +``` + +### Pointer Types +```zig +if (tree.fullPtrType(node)) |ptr| { + // Child type + const child = ptr.ast.child_type; + + // Size (.one, .many, .slice, .c) + const size = ptr.size; + + // Sentinel + if (ptr.ast.sentinel.unwrap()) |sentinel| { + // Process sentinel + } + + // Alignment + if (ptr.ast.align_node.unwrap()) |align_node| { + // Process alignment + } + + // Const/volatile + const is_const = ptr.const_token != null; + const is_volatile = ptr.volatile_token != null; +} +``` + +## Error Handling + +### Check for Parse Errors +```zig +var tree = try std.zig.Ast.parse(allocator, source, .zig); +defer tree.deinit(allocator); + +if (tree.errors.len > 0) { + for (tree.errors) |err| { + // Get error location + const token = err.token; + const loc = tree.tokenLocation(0, token); + + // Format error message + var buf: [512]u8 = undefined; + var w: std.io.Writer = .fixed(&buf); + try tree.renderError(err, &w); + + std.debug.print("{s}:{d}:{d}: error: {s}\n", .{ + filename, + loc.line + 1, + loc.column + 1, + w.buffered(), + }); + } +} +``` + +### ErrorBundle + +Structured error collection for compiler diagnostics. + +```zig +const ErrorBundle = std.zig.ErrorBundle; + +// Create error bundle from AST errors +var wip_errors: ErrorBundle.Wip = undefined; +try wip_errors.init(allocator); +defer wip_errors.deinit(); + +try std.zig.putAstErrorsIntoBundle(allocator, tree, "file.zig", &wip_errors); + +var bundle = try wip_errors.toOwnedBundle(""); +defer bundle.deinit(allocator); + +// Render to stderr +bundle.renderToStdErr(.{ .ttyconf = .no_color }); + +// Or iterate errors +for (bundle.getMessages()) |msg_idx| { + const msg = bundle.getErrorMessage(msg_idx); + const text = bundle.nullTerminatedString(msg.msg); + std.debug.print("Error: {s}\n", .{text}); + + // Get source location + if (msg.src_loc != .none) { + const loc = bundle.getSourceLocation(msg.src_loc); + std.debug.print(" at line {d}\n", .{loc.line + 1}); + } +} +``` + +## String/Number Literals + +### Parse Character Literal +```zig +const result = std.zig.string_literal.parseCharLiteral("'\\n'"); +switch (result) { + .success => |codepoint| { + std.debug.print("Codepoint: {d}\n", .{codepoint}); // 10 + }, + .failure => |err| { + std.debug.print("Error: {f}\n", .{err.fmt("'\\n'")}); + }, +} +``` + +### Parse Number Literal +```zig +const result = std.zig.number_literal.parseNumberLiteral("0xFF_AB"); +switch (result) { + .int => |value| std.debug.print("Integer: {d}\n", .{value}), + .big_int => |base| std.debug.print("Big int, base {d}\n", .{@intFromEnum(base)}), + .float => |base| std.debug.print("Float, base {d}\n", .{@intFromEnum(base)}), + .failure => |err| std.debug.print("Invalid number\n", .{}), +} +``` + +## Identifier Formatting + +### Escape Identifiers +```zig +// Format identifier, escaping if needed +var buf: [256]u8 = undefined; +var w: std.io.Writer = .fixed(&buf); +try w.print("{f}", .{std.zig.fmtId("while")}); // @"while" +try w.print("{f}", .{std.zig.fmtId("hello")}); // hello +try w.print("{f}", .{std.zig.fmtId("123abc")}); // @"123abc" +``` + +### Check Valid Identifier +```zig +std.zig.isValidId("foo") // true +std.zig.isValidId("while") // false (keyword) +std.zig.isValidId("3d") // false (starts with digit) +std.zig.isValidId("a b") // false (contains space) +``` + +### String Escaping +```zig +// Escape string for Zig string literal +var buf: [256]u8 = undefined; +var w: std.io.Writer = .fixed(&buf); +try w.print("\"{f}\"", .{std.zig.fmtString("hello\nworld")}); +// Output: "hello\nworld" + +// Escape character for char literal +try w.print("'{f}'", .{std.zig.fmtChar('\t')}); +// Output: '\t' +``` + +## Source Utilities + +### Find Line/Column +```zig +const loc = std.zig.findLineColumn(source, byte_offset); +std.debug.print("Line {d}, Column {d}\n", .{ loc.line + 1, loc.column + 1 }); +std.debug.print("Source line: {s}\n", .{loc.source_line}); +``` + +### Source Hash +```zig +// Hash source for caching/comparison +const hash = std.zig.hashSrc(source); + +// Compare hashes +if (std.zig.srcHashEql(hash1, hash2)) { + // Sources are identical +} +``` + +### Read Source File +```zig +// Read and decode source file (handles UTF-16LE BOM) +const file = try std.fs.cwd().openFile("source.zig", .{}); +defer file.close(); + +var reader = file.reader(&buf); +const source = try std.zig.readSourceFileToEndAlloc(allocator, &reader); +defer allocator.free(source); +``` + +### Binary Name Generation +```zig +// Get output filename for compilation target +const name = try std.zig.binNameAlloc(allocator, .{ + .root_name = "myapp", + .target = &target, + .output_mode = .Exe, + .link_mode = .dynamic, +}); +defer allocator.free(name); +// "myapp" (Linux), "myapp.exe" (Windows), etc. +``` + +## Common Patterns + +### Walk AST Recursively +```zig +fn walkNode(tree: *const std.zig.Ast, node: std.zig.Ast.Node.Index) void { + const tag = tree.nodeTag(node); + + switch (tag) { + .fn_decl => { + // Process function + const data = tree.nodeData(node); + walkNode(tree, data.node_and_node[0]); // fn_proto + walkNode(tree, data.node_and_node[1]); // body + }, + .block, .block_semicolon => { + var buf: [2]std.zig.Ast.Node.Index = undefined; + if (tree.blockStatements(&buf, node)) |stmts| { + for (stmts) |stmt| { + walkNode(tree, stmt); + } + } + }, + // ... handle other node types + else => {}, + } +} + +// Start from root +for (tree.rootDecls()) |decl| { + walkNode(&tree, decl); +} +``` + +### Extract All Function Names +```zig +fn extractFunctionNames(allocator: Allocator, tree: *const std.zig.Ast) ![][]const u8 { + var names: std.ArrayList([]const u8) = .empty; + defer names.deinit(allocator); + + for (tree.rootDecls()) |decl| { + var buf: [1]std.zig.Ast.Node.Index = undefined; + if (tree.fullFnProto(&buf, decl)) |fn_proto| { + if (fn_proto.name_token) |name_tok| { + try names.append(allocator, tree.tokenSlice(name_tok)); + } + } + } + + return names.toOwnedSlice(allocator); +} +``` + +### Simple Linter Example +```zig +fn checkForTodos(tree: *const std.zig.Ast) void { + const tags = tree.tokens.items(.tag); + const starts = tree.tokens.items(.start); + + for (tags, 0..) |tag, i| { + if (tag == .doc_comment) { + const start = starts[i]; + const slice = tree.source[start..]; + const end = std.mem.indexOfScalar(u8, slice, '\n') orelse slice.len; + const comment = slice[0..end]; + + if (std.mem.indexOf(u8, comment, "TODO")) |_| { + const loc = tree.tokenLocation(0, @intCast(i)); + std.debug.print("TODO found at line {d}\n", .{loc.line + 1}); + } + } + } +} +``` diff --git a/references/std-zip.md b/references/std-zip.md new file mode 100644 index 0000000..18b8b0c --- /dev/null +++ b/references/std-zip.md @@ -0,0 +1,350 @@ +# std.zip - ZIP Archive API Reference (Zig 0.16.0) + +ZIP archive reading and extraction. Zig 0.16 file and stream APIs use `std.Io.Dir`, `std.Io.File`, `std.Io.Reader`, and `std.Io.Writer`. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +Old examples below may contain 0.15 `std.fs` or `std.io` patterns; translate them to `std.Io` before using in new code. + +## Table of Contents +- [Module Structure](#module-structure) +- [Extracting ZIP Archives](#extracting-zip-archives) +- [Iterating Over Entries](#iterating-over-entries) +- [Entry Extraction](#entry-extraction) +- [Diagnostics](#diagnostics) +- [Low-Level Structures](#low-level-structures) +- [Common Patterns](#common-patterns) + +## Module Structure + +```zig +std.zip.extract() // Extract entire archive to directory +std.zip.Iterator // Iterate over archive entries +std.zip.Iterator.Entry // Single archive entry +std.zip.Diagnostics // Track extraction metadata +std.zip.ExtractOptions // Extraction configuration +std.zip.CompressionMethod // .store, .deflate +``` + +## Extracting ZIP Archives + +### Basic Extraction + +Extract all files from a ZIP archive to a directory: + +```zig +const file = try std.fs.cwd().openFile("archive.zip", .{}); +defer file.close(); + +var buf: [4096]u8 = undefined; +var file_reader = file.reader(&buf); + +try std.zip.extract(output_dir, &file_reader, .{}); +``` + +### With Options and Diagnostics + +```zig +var diagnostics: std.zip.Diagnostics = .{ .allocator = allocator }; +defer diagnostics.deinit(); + +try std.zip.extract(output_dir, &file_reader, .{ + .allow_backslashes = true, // normalize \ to / + .diagnostics = &diagnostics, +}); + +// Check common root directory +if (diagnostics.root_dir.len > 0) { + std.debug.print("Archive root: {s}\n", .{diagnostics.root_dir}); +} +``` + +### ExtractOptions + +```zig +pub const ExtractOptions = struct { + allow_backslashes: bool = false, // normalize \ to / in filenames + diagnostics: ?*Diagnostics = null, // track extraction metadata + verify_checksums: bool = false, // TODO: not yet implemented +}; +``` + +## Iterating Over Entries + +### Iterator API + +For more control, iterate over entries individually: + +```zig +const file = try std.fs.cwd().openFile("archive.zip", .{}); +defer file.close(); + +var buf: [4096]u8 = undefined; +var file_reader = file.reader(&buf); + +var iter = try std.zip.Iterator.init(&file_reader); + +var filename_buf: [std.fs.max_path_bytes]u8 = undefined; +while (try iter.next()) |entry| { + // Read filename from archive + try file_reader.seekTo(entry.header_zip_offset + @sizeOf(std.zip.CentralDirectoryFileHeader)); + const filename = filename_buf[0..entry.filename_len]; + try file_reader.interface.readSliceAll(filename); + + std.debug.print("{s}: {d} bytes (compressed: {d})\n", .{ + filename, + entry.uncompressed_size, + entry.compressed_size, + }); +} +``` + +### Iterator.Entry Structure + +```zig +pub const Entry = struct { + version_needed_to_extract: u16, + flags: GeneralPurposeFlags, + compression_method: CompressionMethod, // .store or .deflate + last_modification_time: u16, // DOS time format + last_modification_date: u16, // DOS date format + header_zip_offset: u64, // offset to central directory header + crc32: u32, // CRC-32 checksum + filename_len: u32, + compressed_size: u64, + uncompressed_size: u64, + file_offset: u64, // offset to local file header +}; +``` + +## Entry Extraction + +### Extract Single Entry + +```zig +var iter = try std.zip.Iterator.init(&file_reader); + +var filename_buf: [std.fs.max_path_bytes]u8 = undefined; +while (try iter.next()) |entry| { + // Extract this entry to destination directory + try entry.extract(&file_reader, .{}, &filename_buf, output_dir); +} +``` + +### Selective Extraction + +Extract only specific files: + +```zig +var iter = try std.zip.Iterator.init(&file_reader); + +var filename_buf: [std.fs.max_path_bytes]u8 = undefined; +while (try iter.next()) |entry| { + // Read filename first + try file_reader.seekTo(entry.header_zip_offset + @sizeOf(std.zip.CentralDirectoryFileHeader)); + const filename = filename_buf[0..entry.filename_len]; + try file_reader.interface.readSliceAll(filename); + + // Only extract .zig files + if (std.mem.endsWith(u8, filename, ".zig")) { + try entry.extract(&file_reader, .{}, &filename_buf, output_dir); + } +} +``` + +## Diagnostics + +Track metadata during extraction: + +```zig +var diagnostics: std.zip.Diagnostics = .{ .allocator = allocator }; +defer diagnostics.deinit(); + +try std.zip.extract(dest, &file_reader, .{ + .diagnostics = &diagnostics, +}); + +// root_dir is the common directory prefix for all files (if any) +// e.g., if all files are under "project/", root_dir will be "project" +if (diagnostics.root_dir.len > 0) { + std.debug.print("Common root: {s}\n", .{diagnostics.root_dir}); +} +``` + +## Low-Level Structures + +### CompressionMethod + +```zig +pub const CompressionMethod = enum(u16) { + store = 0, // no compression + deflate = 8, // DEFLATE algorithm + _, // other methods (unsupported) +}; +``` + +### EndRecord + +Find and parse the end-of-central-directory record: + +```zig +// From file +const end_record = try std.zip.EndRecord.findFile(&file_reader); + +// From buffer +const end_record = try std.zip.EndRecord.findBuffer(zip_bytes); + +// Check if ZIP64 extensions needed +if (end_record.need_zip64()) { + // Parse ZIP64 end locator and record +} +``` + +### Header Structures + +```zig +// Central directory file header (46 bytes) +std.zip.CentralDirectoryFileHeader + +// Local file header (30 bytes) +std.zip.LocalFileHeader + +// End of central directory record (22 bytes) +std.zip.EndRecord + +// ZIP64 end of central directory record +std.zip.EndRecord64 + +// ZIP64 end of central directory locator +std.zip.EndLocator64 +``` + +### Signature Constants + +```zig +std.zip.central_file_header_sig // "PK\x01\x02" +std.zip.local_file_header_sig // "PK\x03\x04" +std.zip.end_record_sig // "PK\x05\x06" +std.zip.end_record64_sig // "PK\x06\x06" +std.zip.end_locator64_sig // "PK\x06\x07" +``` + +## Common Patterns + +### Extract ZIP to Directory + +```zig +fn extractZip(allocator: Allocator, zip_path: []const u8, dest_path: []const u8) !void { + const file = try std.fs.cwd().openFile(zip_path, .{}); + defer file.close(); + + var buf: [4096]u8 = undefined; + var file_reader = file.reader(&buf); + + var dest = try std.fs.cwd().makeOpenPath(dest_path, .{}); + defer dest.close(); + + var diagnostics: std.zip.Diagnostics = .{ .allocator = allocator }; + defer diagnostics.deinit(); + + try std.zip.extract(dest, &file_reader, .{ + .allow_backslashes = true, + .diagnostics = &diagnostics, + }); +} +``` + +### List ZIP Contents + +```zig +fn listZip(zip_path: []const u8) !void { + const file = try std.fs.cwd().openFile(zip_path, .{}); + defer file.close(); + + var buf: [4096]u8 = undefined; + var file_reader = file.reader(&buf); + + var iter = try std.zip.Iterator.init(&file_reader); + + var filename_buf: [std.fs.max_path_bytes]u8 = undefined; + var total_size: u64 = 0; + var file_count: u64 = 0; + + while (try iter.next()) |entry| { + try file_reader.seekTo(entry.header_zip_offset + @sizeOf(std.zip.CentralDirectoryFileHeader)); + const filename = filename_buf[0..entry.filename_len]; + try file_reader.interface.readSliceAll(filename); + + const method: []const u8 = switch (entry.compression_method) { + .store => "stored", + .deflate => "deflated", + else => "unknown", + }; + + std.debug.print("{s:40} {d:>10} {s}\n", .{ + filename, + entry.uncompressed_size, + method, + }); + + total_size += entry.uncompressed_size; + file_count += 1; + } + + std.debug.print("\n{d} files, {d} bytes total\n", .{ file_count, total_size }); +} +``` + +### Extract Single File by Name + +```zig +fn extractFile( + file_reader: *std.fs.File.Reader, + target_name: []const u8, + dest: std.fs.Dir, +) !bool { + var iter = try std.zip.Iterator.init(file_reader); + + var filename_buf: [std.fs.max_path_bytes]u8 = undefined; + while (try iter.next()) |entry| { + try file_reader.seekTo(entry.header_zip_offset + @sizeOf(std.zip.CentralDirectoryFileHeader)); + const filename = filename_buf[0..entry.filename_len]; + try file_reader.interface.readSliceAll(filename); + + if (std.mem.eql(u8, filename, target_name)) { + try entry.extract(file_reader, .{}, &filename_buf, dest); + return true; + } + } + return false; // not found +} +``` + +### Check if File is ZIP + +```zig +fn isZipFile(path: []const u8) !bool { + const file = std.fs.cwd().openFile(path, .{}) catch return false; + defer file.close(); + + var buf: [4096]u8 = undefined; + var file_reader = file.reader(&buf); + + _ = std.zip.EndRecord.findFile(&file_reader) catch return false; + return true; +} +``` + +## Supported Features + +**Formats**: ZIP, ZIP64 (large files > 4GB, > 65535 entries) + +**Compression**: Store (uncompressed), Deflate + +**Not supported**: +- Encryption (returns `error.ZipEncryptionUnsupported`) +- Multi-disk archives (returns `error.ZipMultiDiskUnsupported`) +- Other compression methods (LZMA, BZip2, etc.) +- Writing ZIP archives (read-only API) + +**Path handling**: Optional backslash normalization, directory traversal protection (rejects `..` paths) diff --git a/references/std-zon.md b/references/std-zon.md new file mode 100644 index 0000000..392a73d --- /dev/null +++ b/references/std-zon.md @@ -0,0 +1,489 @@ +# std.zon - ZON Parsing and Serialization + +ZON ("Zig Object Notation") parsing and stringification. ZON's grammar is a subset of Zig's syntax. In Zig 0.16, examples that read/write files should use `std.Io.Dir`/`std.Io.File` and an explicit `std.Io`. + +Primary Zig 0.16 release-note source: https://ziglang.org/download/0.16.0/release-notes.html + +## Table of Contents +- [ZON Format Overview](#zon-format-overview) +- [Parsing ZON](#parsing-zon) +- [Serializing to ZON](#serializing-to-zon) +- [Low-Level Serializer API](#low-level-serializer-api) +- [Supported Types](#supported-types) +- [Common Patterns](#common-patterns) + +## ZON Format Overview + +ZON is a data format using Zig's literal syntax: + +```zig +// Example ZON file +.{ + .name = "my-project", + .version = .{ 0, 1, 0 }, + .dependencies = .{ + .@"std-lib" = .{ .url = "https://...", .hash = "abc123" }, + }, + .build_options = .{ + .optimize = .release_safe, + .strip = true, + }, +} +``` + +### Supported Primitives +- Boolean literals: `true`, `false` +- Number literals: `42`, `-3.14`, `0xFF`, `nan`, `inf`, `-inf` +- Character literals: `'a'`, `'\n'`, `'\u{1F600}'` +- Enum literals: `.foo`, `.bar` +- `null` literal +- String literals: `"hello"`, multiline strings + +### Supported Containers +- Anonymous struct literals: `.{ .x = 1, .y = 2 }` +- Anonymous tuple literals: `.{ 1, 2, 3 }` + +**Note:** ZON may not contain type names. Use `@import` for compile-time ZON parsing. + +## Parsing ZON + +### Parse into Struct (Runtime) + +```zig +const std = @import("std"); + +const Config = struct { + name: []const u8, + port: u16 = 8080, + debug: bool = false, +}; + +const zon_str: [:0]const u8 = + \\.{ + \\ .name = "server", + \\ .port = 3000, + \\} +; + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + const allocator = gpa.allocator(); + + const config = try std.zon.parse.fromSlice(Config, allocator, zon_str, null, .{}); + defer std.zon.parse.free(allocator, config); + + // config.name == "server" + // config.port == 3000 + // config.debug == false (default) +} +``` + +### Parse with Diagnostics + +```zig +var diag: std.zon.parse.Diagnostics = .{}; +defer diag.deinit(allocator); + +const result = std.zon.parse.fromSlice(Config, allocator, zon_str, &diag, .{}) catch |err| { + // Print diagnostic errors + var errors = diag.iterateErrors(); + while (errors.next()) |parse_err| { + const loc = parse_err.getLocation(&diag); + std.debug.print("{d}:{d}: {f}\n", .{ + loc.line + 1, + loc.column + 1, + parse_err.fmtMessage(&diag), + }); + } + return err; +}; +defer std.zon.parse.free(allocator, result); +``` + +### Parse Options + +```zig +const result = try std.zon.parse.fromSlice(T, allocator, zon_str, diag, .{ + // Ignore unknown fields (default: false - errors on unknown) + .ignore_unknown_fields = true, + + // Free partially parsed values on error (default: true) + // Disable if using arena allocation + .free_on_error = false, +}); +``` + +### Compile-Time Parsing with @import + +```zig +// build.zig.zon is automatically imported at comptime +const build_zon = @import("build.zig.zon"); + +// Access fields directly +const name = build_zon.name; +const version = build_zon.version; +``` + +### Free Parsed Values + +```zig +const result = try std.zon.parse.fromSlice(T, allocator, zon_str, null, .{}); +defer std.zon.parse.free(allocator, result); +``` + +## Serializing to ZON + +### Simple Serialization + +```zig +const std = @import("std"); + +const Config = struct { + name: []const u8, + port: u16, + enabled: bool, +}; + +pub fn main() !void { + var gpa: std.heap.DebugAllocator(.{}) = .init; + defer _ = gpa.deinit(); + const allocator = gpa.allocator(); + + const config = Config{ + .name = "server", + .port = 8080, + .enabled = true, + }; + + // Serialize to allocated buffer + var aw: std.Io.Writer.Allocating = .init(allocator); + defer aw.deinit(); + + try std.zon.stringify.serialize(config, .{}, &aw.writer); + const zon_str = aw.written(); + // .{ + // .name = "server", + // .port = 8080, + // .enabled = true, + // } +} +``` + +### Serialize Options + +```zig +try std.zon.stringify.serialize(value, .{ + // Include whitespace for readability (default: true) + .whitespace = true, // false for minified output + + // Emit codepoints as character literals (default: .never) + .emit_codepoint_literals = .never, // always emit as integers + // .emit_codepoint_literals = .printable_ascii, // 'a' for printable ASCII + // .emit_codepoint_literals = .always, // '⚡' for all valid codepoints + + // Emit []u8 as tuple instead of string (default: false) + .emit_strings_as_containers = false, + + // Skip fields equal to their default value (default: true) + .emit_default_optional_fields = true, // false to omit defaults +}, &writer); +``` + +### Serialization with Depth Limits (Recursive Types) + +```zig +// For potentially recursive types, use depth-limited versions: + +// Returns error.ExceededMaxDepth if depth exceeded +try std.zon.stringify.serializeMaxDepth(value, .{}, &writer, 16); + +// No depth checking - caller must ensure no cycles +try std.zon.stringify.serializeArbitraryDepth(value, .{}, &writer); +``` + +## Low-Level Serializer API + +Use `std.zon.Serializer` for fine-grained control over output. + +### Manual Struct Serialization + +```zig +var aw: std.Io.Writer.Allocating = .init(allocator); +defer aw.deinit(); + +var s: std.zon.Serializer = .{ .writer = &aw.writer }; + +var container = try s.beginStruct(.{}); +try container.field("x", 10, .{}); +try container.field("y", 20, .{}); +try container.field("name", "point", .{}); +try container.end(); + +// Output: .{ +// .x = 10, +// .y = 20, +// .name = "point", +// } +``` + +### Manual Tuple Serialization + +```zig +var s: std.zon.Serializer = .{ .writer = &aw.writer }; + +var tuple = try s.beginTuple(.{}); +try tuple.field(1, .{}); +try tuple.field(2, .{}); +try tuple.field(3, .{}); +try tuple.end(); + +// Output: .{ +// 1, +// 2, +// 3, +// } +``` + +### Container Options + +```zig +// Control wrapping behavior +var container = try s.beginStruct(.{ + .whitespace_style = .{ .wrap = true }, // Always wrap fields + // .whitespace_style = .{ .wrap = false }, // Never wrap (single line) + // .whitespace_style = .{ .fields = 2 }, // Auto-wrap if > 2 fields +}); +``` + +### Nested Containers + +```zig +var s: std.zon.Serializer = .{ .writer = &aw.writer }; + +var root = try s.beginStruct(.{}); + +// Nested tuple +var coords = try root.beginTupleField("coords", .{}); +try coords.field(10, .{}); +try coords.field(20, .{}); +try coords.end(); + +// Nested struct +var meta = try root.beginStructField("meta", .{}); +try meta.field("id", 42, .{}); +try meta.end(); + +try root.end(); + +// Output: .{ +// .coords = .{ +// 10, +// 20, +// }, +// .meta = .{ +// .id = 42, +// }, +// } +``` + +### Primitive Serialization + +```zig +var s: std.zon.Serializer = .{ .writer = &aw.writer }; + +// Integer +try s.int(42); + +// Float +try s.float(3.14); + +// String +try s.string("hello\nworld"); // "hello\nworld" + +// Multiline string +try s.multilineString("line1\nline2", .{}); +// \\line1 +// \\line2 + +// Identifier/enum literal +try s.ident("foo"); // .foo +try s.ident("var"); // .@"var" (escaped keyword) + +// Unicode codepoint +try s.codePoint('a'); // 'a' +try s.codePoint('⚡'); // '\u{26a1}' +``` + +### Value Serialization with Options + +```zig +var s: std.zon.Serializer = .{ .writer = &aw.writer }; + +try s.value(my_value, .{ + .emit_codepoint_literals = .always, + .emit_strings_as_containers = false, + .emit_default_optional_fields = true, +}); +``` + +## Supported Types + +### Parse-able Types + +| Zig Type | ZON Syntax | +|----------|------------| +| `bool` | `true`, `false` | +| `i32`, `u64`, etc. | `42`, `-5`, `0xFF` | +| `f32`, `f64` | `3.14`, `-0.0`, `nan`, `inf` | +| `?T` | value or `null` | +| `[]const u8` | `"string"`, multiline strings | +| `[]T` | `.{ item1, item2, ... }` | +| `[N]T` | `.{ item1, item2, ... }` (exact length) | +| `struct` | `.{ .field = value, ... }` | +| `struct (tuple)` | `.{ value1, value2, ... }` | +| `union(enum)` | `.tag` or `.{ .tag = value }` | +| `enum` | `.variant` | +| `*T` | value (auto-allocated) | +| `@Vector(N, T)` | `.{ elem1, elem2, ... }` | + +### Non-serializable Types + +These types cannot be serialized: +- `type`, `void` (except as union payload), `noreturn` +- Error sets/error unions +- Untagged unions +- Non-exhaustive enums +- Many-pointers (`[*]T`) or C-pointers (`[*c]T`) +- Opaque types (`anyopaque`) +- Async frame types (`anyframe`) +- Functions + +## Common Patterns + +### Build Configuration File + +```zig +// build.zig.zon +.{ + .name = "my-project", + .version = "0.1.0", + .dependencies = .{ + .zap = .{ + .url = "https://github.com/...", + .hash = "...", + }, + }, + .paths = .{ "src", "build.zig", "build.zig.zon" }, +} +``` + +```zig +// build.zig - reading build.zig.zon at comptime +const build_zon = @import("build.zig.zon"); +const project_name = build_zon.name; +``` + +### Config File with Defaults + +```zig +const Config = struct { + host: []const u8 = "localhost", + port: u16 = 8080, + workers: u8 = 4, + debug: bool = false, +}; + +fn loadConfig(allocator: std.mem.Allocator, path: []const u8) !Config { + const file = std.fs.cwd().openFile(path, .{}) catch |err| switch (err) { + error.FileNotFound => return Config{}, + else => return err, + }; + defer file.close(); + + const content = try file.readToEndAllocOptions( + allocator, + 1024 * 1024, + null, + @alignOf(u8), + 0, // null terminator + ); + defer allocator.free(content); + + return std.zon.parse.fromSlice(Config, allocator, content, null, .{ + .ignore_unknown_fields = true, + .free_on_error = true, + }); +} +``` + +### Serialize to File + +```zig +fn saveConfig(allocator: std.mem.Allocator, config: Config, path: []const u8) !void { + var aw: std.Io.Writer.Allocating = .init(allocator); + defer aw.deinit(); + + try std.zon.stringify.serialize(config, .{ .whitespace = true }, &aw.writer); + + const file = try std.fs.cwd().createFile(path, .{}); + defer file.close(); + + try file.writeAll(aw.written()); +} +``` + +### Union Serialization + +```zig +const Value = union(enum) { + int: i64, + float: f64, + string: []const u8, + none, // void payload +}; + +const v1 = Value{ .int = 42 }; +// Serializes as: .{ .int = 42 } + +const v2 = Value.none; +// Serializes as: .none +``` + +### Skip Default Fields + +```zig +const Settings = struct { + theme: []const u8 = "dark", + font_size: u8 = 12, + custom_value: u32, +}; + +const settings = Settings{ .custom_value = 100 }; + +try std.zon.stringify.serialize(settings, .{ + .emit_default_optional_fields = false, +}, &writer); + +// Output: .{ .custom_value = 100 } +// (theme and font_size omitted because they equal defaults) +``` + +### Round-Trip ZON Data + +```zig +fn roundTrip(comptime T: type, allocator: std.mem.Allocator, value: T) !T { + // Serialize + var aw: std.Io.Writer.Allocating = .init(allocator); + defer aw.deinit(); + try std.zon.stringify.serialize(value, .{}, &aw.writer); + + // Add null terminator for parsing + try aw.writer.writeByte(0); + const zon_str = aw.written(); + const terminated: [:0]const u8 = zon_str[0 .. zon_str.len - 1 :0]; + + // Parse back + return std.zon.parse.fromSlice(T, allocator, terminated, null, .{}); +} +``` diff --git a/references/style-guide.md b/references/style-guide.md new file mode 100644 index 0000000..4575bf2 --- /dev/null +++ b/references/style-guide.md @@ -0,0 +1,215 @@ +# Zig Style Guide + +Official coding conventions from the Zig language reference. These are implemented and enforced by `zig fmt`. + +## Naming Conventions + +### Summary Table + +| Element | Convention | Example | +|---------|-----------|---------| +| Types | `TitleCase` | `XmlParser`, `HashMap` | +| Namespace structs (0 fields) | `snake_case` | `std.json`, `std.mem` | +| Functions | `camelCase` | `readU32Be`, `parseJson` | +| Functions returning `type` | `TitleCase` | `ArrayList`, `HashMap` | +| Variables/constants | `snake_case` | `const_name`, `global_var` | +| File names (types) | `TitleCase.zig` | `ArrayList.zig` | +| File names (namespaces) | `snake_case.zig` | `mem.zig`, `json.zig` | +| Directories | `snake_case` | `std/`, `hash_map/` | + +### Rules in Detail + +**Types use `TitleCase`:** +```zig +const StructName = struct { field: i32 }; +const TypeName = @import("dir_name/TypeName.zig"); +``` + +**Exception: Namespace structs (0 fields) use `snake_case`:** +```zig +const namespace_name = @import("dir_name/file_name.zig"); +``` + +**Functions use `camelCase`:** +```zig +fn functionName(param_name: TypeName) void { } +fn readU32Be() u32 { } // Acronyms treated as words +``` + +**Functions returning `type` use `TitleCase`:** +```zig +fn ListTemplateFunction(comptime ChildType: type, comptime fixed_size: usize) type { + return List(ChildType, fixed_size); +} + +fn ShortList(comptime T: type, comptime n: usize) type { + return struct { + field_name: [n]T, + fn methodName() void {} + }; +} +``` + +**Variables and constants use `snake_case`:** +```zig +var global_var: i32 = undefined; +const const_name = 42; +const primitive_type_alias = f32; +const string_alias = []u8; +``` + +### Acronyms and Initialisms + +Acronyms follow normal casing rules—they're treated as regular words: + +```zig +// XML loses its all-caps when used in identifiers +const XmlParser = struct { field: i32 }; +fn parseXml() void {} +const xml_document = "..."; + +// BE (Big Endian) treated as a word +fn readU32Be() u32 {} + +// URL, HTTP, etc. follow the same rule +const HttpClient = struct {}; +fn parseUrl() void {} +const api_url = "..."; +``` + +### Established Conventions + +Follow established conventions when they exist (e.g., `ENOENT` from POSIX): +```zig +const ENOENT = error.FileNotFound; +``` + +## Avoid Redundancy in Names + +### Words to Avoid in Type Names + +Don't use these words—they apply to everything and communicate nothing: +- `Value` +- `Data` +- `Context` +- `Manager` +- `utils`, `misc`, or somebody's initials + +```zig +// BAD +const JsonValue = union(enum) { ... }; +const DataManager = struct { ... }; +const misc = @import("misc.zig"); + +// GOOD +const Value = union(enum) { ... }; // In json namespace: json.Value +const Store = struct { ... }; +// Put utilities at module root, no namespace needed +``` + +### Avoid Redundancy in Fully-Qualified Namespaces + +Don't repeat the namespace in the type name: + +```zig +// BAD - "json" appears twice in json.JsonValue +pub const json = struct { + pub const JsonValue = union(enum) { number: f64, boolean: bool }; +}; + +// GOOD - json.Value is clear and non-redundant +pub const json = struct { + pub const Value = union(enum) { number: f64, boolean: bool }; +}; +``` + +The same applies to files (which are implicit structs): +```zig +// In json.zig: +// BAD +pub const JsonParser = struct { ... }; + +// GOOD +pub const Parser = struct { ... }; // Used as json.Parser +``` + +## Whitespace + +- **Indentation:** 4 spaces (not tabs) +- **Braces:** Opening brace on same line, unless wrapping is needed +- **Line length:** Aim for ~100 characters; use common sense +- **Trailing commas:** Use trailing commas for lists with more than 2 items + +```zig +// Short list - can be on one line +const pair = .{ a, b }; + +// Longer list - one item per line with trailing comma +const Config = struct { + name: []const u8, + port: u16, + timeout: u32, + max_connections: usize, // trailing comma +}; +``` + +**Line wrapping:** +```zig +// When arguments don't fit, wrap and align +fn processRequest( + allocator: Allocator, + request: *const Request, + options: ProcessOptions, +) !Response { + // ... +} +``` + +## Doc Comments + +- **Omit redundant information** that's already clear from the name +- **Duplicate information** across similar functions (helps IDEs) +- Use **"assume"** for invariants that cause *unchecked* illegal behavior when violated +- Use **"assert"** for invariants that cause *safety-checked* illegal behavior when violated + +```zig +/// Reads a little-endian u32 from the buffer. +/// +/// Caller must **assume** buffer has at least 4 bytes remaining. +/// This is not checked and will cause undefined behavior if violated. +fn readU32Le(buf: []const u8) u32 { + return std.mem.readInt(u32, buf[0..4], .little); +} + +/// Pops the last element from the list. +/// +/// **Asserts** the list is not empty. In safe modes, returns an error +/// or panics if the list is empty. +fn pop(self: *Self) T { + std.debug.assert(self.items.len > 0); + // ... +} +``` + +## Source Encoding + +- **UTF-8** encoding required +- **LF** (`\n`, 0x0a) line endings (CRLF discouraged but tolerated) +- End files with a newline +- No hard tabs (spaces only) +- `zig fmt` enforces all these conventions + +## Applying the Style Guide + +Run `zig fmt` to automatically format code according to these conventions: + +```bash +# Format a single file +zig fmt src/main.zig + +# Format entire project +zig fmt . + +# Check without modifying (useful for CI) +zig fmt --check src/ +``` diff --git a/references/zig-0.16-release-notes.md b/references/zig-0.16-release-notes.md new file mode 100644 index 0000000..d0c285a --- /dev/null +++ b/references/zig-0.16-release-notes.md @@ -0,0 +1,642 @@ +# Zig 0.16.0 Release Notes Migration Reference + +Primary source: https://ziglang.org/download/0.16.0/release-notes.html + +Use this file as the first stop when upgrading or reviewing Zig 0.16 code. It is organized in the same broad order as the official release notes and converts the release-note material into practical coding guidance. + +## Version Posture + +Zig 0.16.0 is not a small stdlib polish release. It changes ownership of I/O, synchronization, file system access, process spawning, time, randomness, containers, C translation, and several language edge cases. + +General rule for new Zig 0.16 code: + +- APIs that can block, touch the OS, use entropy, query time, spawn work, or otherwise introduce nondeterminism should accept or store a `std.Io`. +- Application entry points should get `io` from Juicy Main (`std.process.Init`) when possible. +- Tests should prefer `std.testing.io`. +- A local `std.Io.Threaded.init_single_threaded` is acceptable only as a temporary adapter at a boundary that cannot yet receive an `io`. + +## Target Support + +The release notes broaden native CI coverage and add/remove several targets. The important migration guidance is: + +- Do not assume an untested target is broken just because old Zig versions were rough there; 0.16 improves stack traces and weakly-ordered architecture behavior. +- Minimum OS versions changed. Current notable minimums include Linux 5.10, macOS 13, Windows 10, FreeBSD 14.0, NetBSD 10.1, OpenBSD 7.8, and DragonFly BSD 6.0. +- If a product's target matrix depends on older OS versions, check the official support table before promising support. +- More targets have usable stack traces, which makes debug/unwind behavior more valuable but also means verbose crash/log paths may become more expensive if left in hot paths. + +## Language Changes + +### switch + +0.16 extends valid switch prong expressions and fixes several switch edge cases. + +Practical notes: + +- Packed structs/unions can appear as switch prong items and compare by backing integer. +- Decl literals and result-typed expressions such as `@enumFromInt` are more broadly usable in prongs. +- Union tag captures are allowed for all prongs, not just inline prongs. +- Switch prong captures cannot all be discarded. +- Error switches have stricter and more consistent unreachable-else handling. + +### @cImport Moves Toward the Build System + +`@cImport` is deprecated as the long-term C translation API. The official upgrade path is: + +1. Put includes in a real header. +2. Add a build-system `b.addTranslateC(...)` step. +3. Import `translate_c.createModule()` into the Zig module graph. + +For ABI-sensitive bindings: + +- Prefer build-system translation for new C bindings. +- Keep translated bindings stable and check generated ABI-sensitive structs with comptime size/alignment/offset assertions. +- If generated C output differs across translation approaches, treat it as a bug-risk investigation, not cosmetic churn. + +### @Type Removed + +`@Type` is replaced by specific type-constructing builtins: + +- `@EnumLiteral` +- `@Int` +- `@Tuple` +- `@Pointer` +- `@Fn` +- `@Struct` +- `@Union` +- `@Enum` +- `@Opaque` + +Migration rule: + +- Use the narrow builtin that matches the type you are constructing. +- Keep `@typeInfo` for reflection. +- Replace old `@Type(.{ .@"struct" = ... })` helpers with `@Struct(...)`, and similarly for unions/enums/pointers/functions. + +### Numeric and Vector Changes + +0.16 allows small integer types to coerce to floats in more cases, but it also tightens vector and array representation rules. + +Practical notes: + +- Runtime vector indexes are forbidden. Use scalar extraction patterns, compile-time indexes, or restructure the vector operation. +- Vectors and arrays no longer support in-memory coercion. Make conversions explicit with loads, stores, or element-wise construction. +- Unary float builtins forward result type. Code such as `const x: f64 = @sqrt(@floatFromInt(n));` now works as intended. +- `@floor`, `@ceil`, `@round`, and `@trunc` can convert floats to integer result types. `@intFromFloat` is now redundant with `@trunc` and is deprecated. + +### Returning Local Addresses + +The compiler now diagnoses trivially returning the address of an expired local variable. + +Review rule: + +- If a function returns a pointer, confirm the pointee outlives the function. +- Prefer caller-provided output buffers, allocator-owned results, or stable owner structs. + +### Packed and Extern Type Tightening + +0.16 makes packed/extern layout more explicit: + +- Packed union fields must have an unambiguous backing bit size. +- Pointers are forbidden in packed structs and packed unions; store an integer address only when that is actually the ABI. +- Packed unions may specify explicit backing integers. +- Enum and packed types in extern contexts need explicit backing types. + +Binding rule: + +- For ABI-sensitive C bindings, preserve or add comptime checks for size, alignment, and field offsets. +- Do not accept layout churn in translated bindings without checking the C header contract. + +### Type Resolution + +0.16 reworks type resolution. Some dependency loops disappear, while other previously accepted self-dependent constructs are now rejected with clearer diagnostics. + +Practical notes: + +- Do not assume every new dependency-loop diagnostic is a false positive. +- Prefer moving self-referential size/alignment queries behind explicit helper types or runtime fields. +- Lazy field analysis means more namespace-like types can exist without forcing full field resolution. +- Pointers to comptime-only types are no longer themselves comptime-only, but dereferencing them at runtime is still invalid. +- Explicitly aligned pointer types are distinct from naturally aligned pointer types, even if they coerce easily. +- Zero-bit tuple fields are no longer implicitly marked comptime in type info. + +## Standard Library + +### Top-Level Additions, Removals, and Renames + +Important removals/renames: + +- `SegmentedList` removed. +- `std.meta.declList` removed. +- `std.Io.GenericWriter`, `std.Io.AnyWriter`, `std.Io.null_writer`, and `std.Io.CountingReader` removed. +- `std.Thread.Mutex.Recursive` removed. +- `std.fmt.format` is replaced by `std.Io.Writer.print`. +- `std.fmt.Formatter` is renamed to `std.fmt.Alt`. +- `std.fmt.FormatOptions` is renamed to `std.fmt.Options`. +- `std.fmt.bufPrintZ` is renamed to `std.fmt.bufPrintSentinel`. +- `std.DynLib` removed Windows support; use platform APIs (`LoadLibraryExW`, `GetProcAddress`) directly or through a local abstraction. +- `BitSet` and `EnumSet` use decl literals instead of `initEmpty` / `initFull`. + +Important error changes: + +- `error.RenameAcrossMountPoints` and `error.NotSameFileSystem` become `error.CrossDevice`. +- `error.SharingViolation` becomes `error.FileBusy`. +- `error.EnvironmentVariableNotFound` becomes `error.EnvironmentVariableMissing`. +- `std.Io.Dir.rename` returns `error.DirNotEmpty` rather than `error.PathAlreadyExists` for non-empty destination directories. + +### I/O as an Interface + +The core 0.16 rule: all input/output functionality requires an `std.Io` instance. + +Anything that might block, interact with the outside world, depend on the OS, wait on concurrency, use entropy, or introduce nondeterminism belongs under the `Io` interface. + +Main patterns: + +```zig +const std = @import("std"); + +pub fn main(init: std.process.Init) !void { + const gpa = init.gpa; + const io = init.io; + _ = gpa; + _ = io; +} +``` + +Temporary adapter when no owner can pass `io` yet: + +```zig +var threaded: std.Io.Threaded = .init_single_threaded; +const io = threaded.io(); +``` + +Testing: + +```zig +const io = std.testing.io; +``` + +I/O implementations mentioned by the release notes: + +- `std.Io.Threaded`: feature-complete threaded implementation and closest behavior to 0.15 blocking APIs. +- `std.Io.Evented`: experimental M:N/user-space stack switching implementation. +- `std.Io.Uring`, `std.Io.Kqueue`, `std.Io.Dispatch`: early or platform-specific implementations. +- `std.Io.failing`: no-operation/failing implementation for unsupported contexts and tests. + +### Future, Group, Cancelation, and Batch + +New task-level and operation-level APIs live under `std.Io`. + +Guidance: + +- `io.async(...)` creates a future for function-level task independence. +- `std.Io.Group` manages many tasks and can await or cancel them together. +- `std.Io.Batch` is lower-level and operation-oriented; use it when you need efficient independence among I/O operations rather than arbitrary Zig functions. +- Propagate `error.Canceled` unless the code that requested cancelation is the code handling it. +- If you handle `error.Canceled` locally and continue, use `io.recancel()` when the cancelation should remain active. +- Prefer `errdefer group.cancel(io)` after spawning grouped work so task resources are released on all exits. + +### Synchronization Primitives + +Synchronization that can block must migrate to `std.Io` equivalents so it cooperates with the chosen I/O backend. + +Map: + +- `std.Thread.ResetEvent` -> `std.Io.Event` +- `std.Thread.WaitGroup` -> `std.Io.Group` +- `std.Thread.Futex` -> `std.Io.Futex` +- `std.Thread.Mutex` -> `std.Io.Mutex` +- `std.Thread.Condition` -> `std.Io.Condition` +- `std.Thread.Semaphore` -> `std.Io.Semaphore` +- `std.Thread.RwLock` -> `std.Io.RwLock` +- `std.once` removed + +Use cancelable locks when cancelation should be honored: + +```zig +try mutex.lock(io); +defer mutex.unlock(io); +``` + +Use uncancelable locks only for short critical sections or cleanup paths: + +```zig +mutex.lockUncancelable(io); +defer mutex.unlock(io); +``` + +Lock-free atomics do not need `std.Io`. + +### Entropy and Random + +Entropy moved under `std.Io`. + +Patterns: + +```zig +var bytes: [32]u8 = undefined; +io.random(&bytes); + +const rng_source: std.Random.IoSource = .{ .io = io }; +const rng = rng_source.interface(); +``` + +Use `io.randomSecure(...)` when fresh OS-backed cryptographic entropy is required and failures should be reported. + +### Time + +The old wall-clock/monotonic split is now routed through `std.Io` time types for clock operations that may depend on the runtime. + +Migration highlights from the release notes: + +- `std.time.Instant` -> `std.Io.Timestamp` +- `std.time.Timer` -> `std.Io.Timestamp` +- `std.time.timestamp` -> `std.Io.Timestamp.now` + +Application preference: + +- Put common timestamp reads behind a shared application helper so callsites do not reinvent `Io.Timestamp` clock-selection or conversion rules. +- Store `io` on timing systems that need repeated timestamps instead of constructing local fallback `Io.Threaded` instances. + +### File System + +Most file system APIs moved from `std.fs` handles to `std.Io.Dir` and `std.Io.File`. + +Basic patterns: + +```zig +const cwd = std.Io.Dir.cwd(); +const file = try cwd.openFile(io, "data.txt", .{}); +defer file.close(io); + +var buf: [4096]u8 = undefined; +var reader = file.reader(io, &buf); +const bytes = try reader.interface.allocRemaining(gpa, .limited(1024 * 1024)); +defer gpa.free(bytes); +``` + +```zig +const out = try std.Io.Dir.cwd().createFile(io, "out.txt", .{}); +defer out.close(io); + +var buf: [4096]u8 = undefined; +var writer = out.writer(io, &buf); +try writer.interface.print("value={d}\n", .{42}); +try writer.interface.flush(); +``` + +Convenience methods: + +```zig +const bytes = try std.Io.Dir.cwd().readFileAlloc(io, "data.txt", gpa, .limited(1024 * 1024)); +defer gpa.free(bytes); + +try std.Io.Dir.cwd().writeFile(io, .{ + .sub_path = "out.txt", + .data = "hello\n", +}); +``` + +Common migrations: + +- `file.close()` -> `file.close(io)` +- `std.fs.cwd()` -> `std.Io.Dir.cwd()` +- `std.fs.File.stdout()` -> `std.Io.File.stdout()` +- `std.fs.Dir.readFileAlloc(...)` -> `std.Io.Dir.readFileAlloc(io, ..., .limited(max))` +- `std.fs.File.readToEndAlloc(...)` -> file reader + `reader.interface.allocRemaining(...)` +- `fs.copyFileAbsolute` and other absolute helpers move to `std.Io.Dir.*Absolute` +- Many `Z` and `W` path-specific helpers were removed; use the cross-platform `[]u8` path APIs. + +### Networking and HTTP + +Networking moved under `std.Io.net`, and higher-level clients hold an `io`. + +Patterns: + +```zig +var client: std.http.Client = .{ + .allocator = gpa, + .io = io, +}; +defer client.deinit(); +``` + +The release notes call out that DNS, parallel connection attempts, and cancelation now work through the chosen `Io` implementation. Do not bypass this with direct platform sockets in new generic code. + +### Process, Args, Env, and Preopens + +Process APIs now use `std.Io`, and Juicy Main makes args/env/preopens non-global. + +Patterns: + +```zig +pub fn main(init: std.process.Init) !void { + const io = init.io; + const gpa = init.gpa; + const args = try init.minimal.args.toSlice(init.arena.allocator()); + _ = args; + _ = gpa; + _ = io; +} +``` + +Run a child and capture output: + +```zig +const result = try std.process.run(gpa, io, .{ + .argv = &.{ "git", "status", "--short" }, + .stdout_limit = .limited(64 * 1024), + .stderr_limit = .limited(64 * 1024), +}); +defer gpa.free(result.stdout); +defer gpa.free(result.stderr); +``` + +Spawn a child: + +```zig +var child = try std.process.spawn(io, .{ + .argv = &.{ "tool", "--flag" }, + .stdout = .pipe, + .stderr = .pipe, +}); +defer child.kill(io); + +const term = try child.wait(io); +_ = term; +``` + +Environment guidance: + +- Prefer `init.environ_map` from `std.process.Init` at app boundaries. +- `error.EnvironmentVariableNotFound` is now `error.EnvironmentVariableMissing`. +- Current directory is now `std.process.currentPath(io, buffer)` or `std.process.currentPathAlloc(io, allocator)`. +- WASI preopens moved to `std.process.Preopens` and are available through Juicy Main. + +### File.MemoryMap + +Memory-mapped file APIs moved under `std.Io.File.MemoryMap`. Treat maps as file-backed I/O resources and keep their lifetime explicit. + +### posix and os.windows Removals + +Several low-level stdlib wrappers were removed as part of moving blocking/nondeterministic operations behind `std.Io`. Prefer the `std.Io` abstraction for portable code. Drop down to `std.posix` or `std.os.windows` only for explicit platform-specific code. + +### Allocators + +`heap.ArenaAllocator` is now thread-safe and lock-free. + +`heap.ThreadSafe` allocator was removed. If shared allocation still needs protection, choose an allocator that is already thread-safe or place synchronization at the owner of the shared data. + +### Compression + +Compression/decompression APIs continue moving toward `std.Io.Reader` and `std.Io.Writer`. LZMA, LZMA2, and XZ are specifically called out as updated. Deflate compression was added and decompression simplified. + +Review rule: + +- Do not reintroduce old `std.io.GenericReader` adapter patterns for compressors. + +### Debug Information + +Debug information was reworked and improved for several kinds of types. Expect better type names and better error-set runtime names, but also keep debug/unwind cost in mind for hot logging paths. + +### Inter-Process Progress on Windows + +Progress reporting now has better Windows inter-process support. Prefer std build/progress APIs over custom ad hoc pipes for build step progress where practical. + +### Windows Networking and NtDll + +Networking no longer requires `ws2_32.dll` in the same way old code did, and more Windows implementation moved toward NtDll. Prefer std `Io.net` when writing portable networking code. + +### mem cut/find APIs + +The mem API emphasizes `find*` naming and adds `cut*` helpers. + +Common names in 0.16: + +- `std.mem.find` +- `std.mem.findLast` +- `std.mem.findScalar` +- `std.mem.findScalarLast` +- `std.mem.findAny` +- `std.mem.findNone` +- `std.mem.cut` +- `std.mem.cutLast` +- `std.mem.cutScalar` +- `std.mem.cutPrefix` +- `std.mem.cutSuffix` + +Avoid adding new `indexOf` / `lastIndexOf` style calls in 0.16 code. + +### Directory Walking and Paths + +`std.Io.Dir.walkSelectively` was added for recursive walks where the walker decides which directories to enter. + +`std.fs.path` behavior changed for Windows path handling: + +- UNC, rooted, and drive-relative paths are handled more consistently. +- `relative`, `relativeWindows`, and `relativePosix` are now pure and require the current directory path and sometimes an environment map as input. + +Pattern: + +```zig +const cwd_path = try std.process.currentPathAlloc(io, gpa); +defer gpa.free(cwd_path); + +const relative = try std.fs.path.relative(gpa, cwd_path, init.environ_map, from, to); +defer gpa.free(relative); +``` + +### File.Stat + +Access time is optional: + +```zig +const stat = try file.stat(io); +const atime = stat.atime orelse return error.FileAccessTimeUnavailable; +``` + +Timestamp-setting APIs also use structured options rather than positional atime/mtime values. + +### Atomic and Temporary Files + +Atomic temporary file handling now uses `std.Io.File.Atomic` and the `Io` entropy path. Do not hand-roll random temp names when std already provides an atomic file helper. + +### Current Directory API + +Use: + +- `std.process.currentPath(io, buffer)` +- `std.process.currentPathAlloc(io, allocator)` + +Avoid old `std.process.getCwd*` patterns. + +### Migration to Unmanaged Containers + +0.16 continues the container migration toward allocator-free fields and allocator-at-call-site methods. + +Important changes: + +- `ArrayHashMap`, `AutoArrayHashMap`, and `StringArrayHashMap` removed. +- `AutoArrayHashMapUnmanaged` -> `std.array_hash_map.Auto` +- `StringArrayHashMapUnmanaged` -> `std.array_hash_map.String` +- `ArrayHashMapUnmanaged` -> `std.array_hash_map.Custom` +- `PriorityQueue` and `PriorityDequeue` no longer store allocators. + +### PriorityQueue and PriorityDequeue + +Priority containers now prefer `.empty` and push/pop terminology. + +Common migrations: + +- `init` -> `.empty` or `initContext` +- `add` -> `push` +- `addSlice` -> `pushSlice` +- `addUnchecked` -> `pushUnchecked` +- `remove` / `removeOrNull` -> `pop` +- `removeIndex` -> `popIndex` +- PriorityDequeue min/max variants become `popMin` / `popMax` + +### Thread.Pool Removed + +`std.Thread.Pool` is removed. Migrate simple independent jobs to `std.Io.async` or `std.Io.Group.async` when they are actually asynchronous from the caller. + +If tasks synchronize with the caller or each other, re-evaluate the design rather than doing a mechanical replacement. Any blocking synchronization used by `Io` tasks must move from `std.Thread.*` primitives to `std.Io.*` primitives. + +### subsystem APIs + +`std.builtin.subsystem` was removed. `std.Target.SubSystem` moved to `std.zig.Subsystem` with field-name updates, while deprecated aliases remain for some build-script compatibility. + +### Reader/Writer Removals + +`std.io` is now `std.Io`. + +Migration map: + +- `std.Io.GenericReader` -> `std.Io.Reader` +- `std.Io.AnyReader` -> `std.Io.Reader` +- `std.io.fixedBufferStream(data).reader()` -> `var r: std.Io.Reader = .fixed(data)` +- `std.io.fixedBufferStream(buffer).writer()` -> `var w: std.Io.Writer = .fixed(buffer)` +- `std.leb.readUleb128` / `readIleb128` -> `std.Io.Reader.takeLeb128` + +### Duration Formatting + +The `{D}` duration format specifier was removed. Format `std.Io.Duration` with `{f}`. + +```zig +try writer.print("{f}", .{std.Io.Duration{ .nanoseconds = ns }}); +``` + +### fs.getAppDataDir Removed + +Application data directory policy is now application-owned. Use app-specific logic or a third-party package such as known-folders if desired. + +### Io.Writer.Allocating + +`std.Io.Writer.Allocating` now stores an `alignment` field. Prefer the provided initializers rather than field literals unless the field set is intentional and complete. + +### Crypto + +0.16 adds AES-SIV, AES-GCM-SIV, and the Ascon AEAD/hash constructions. Existing crypto code also needs the entropy migration described above. + +## Build System + +### Local Package Overrides + +The build system can override packages locally. Use this for development overrides rather than editing dependency cache contents. + +### Project-Local Package Fetching + +Packages can be fetched into a project-local directory. This is useful for reproducible workspaces and local depot workflows. + +### Unit Test Timeouts + +`zig build test --test-timeout ` can bound tests by real time. Use this for runaway tests, but remember scheduler load can make real-time limits flaky. + +### Error Formatting + +New flags: + +- `--error-style verbose|minimal|verbose_clear|minimal_clear` +- `--multiline-errors indent|newline|none` + +`--prominent-compile-errors` was removed. Use `--error-style minimal` for the closest behavior. + +### Temporary Files + +`Build.makeTempPath` and the RemoveDir step are gone. Use: + +- `b.addTempFiles` +- `b.addMutateFiles` +- `b.tmpPath` +- `std.Build.Step.WriteFile` in temporary/mutate modes + +Do not create temporary directories during the configure phase and then mutate them during make. + +## Compiler + +### C Translation + +Translate-c is now based on Aro/translate-c rather than libclang. The change is intended to be non-breaking, but generated code can differ. + +For ABI-sensitive libraries: + +- Regenerate translated C bindings deliberately. +- Compare struct layouts, enum values, constants, calling conventions, and macro translation. +- Keep compile-time asserts next to translated bindings. + +### LLVM Backend + +The LLVM backend has experimental incremental compilation support, smaller bitcode output, some compile-time improvements, and better debug info for several type cases. + +### Type Resolution + +Compiler type resolution changed significantly. See the language section above for source-level effects. + +### Incremental Compilation + +Incremental compilation is more usable but still disabled by default. Try `zig build -fincremental --watch` for local iteration, but do not treat it as a required CI mode yet. + +### x86 and Other Backends + +The x86 backend remains the default for Debug mode and has faster compile times than LLVM, with lower machine-code quality. Other self-hosted backends continue to mature. + +## Linker + +The new ELF linker is available with `-fnew-linker` or build-script options and is default when using incremental compilation for ELF. It is faster for incremental relinks but not feature-complete, especially around debug information. + +Use it for iteration when it works; validate release/QA builds on the intended linker path. + +## Fuzzer + +Fuzz tests now use `*std.testing.Smith` instead of a raw `[]const u8` input. `Smith` generates structured values, bytes, slices, and weighted choices. + +Migration rule: + +- Update fuzz entry points to accept `*std.testing.Smith`. +- Generate inputs through Smith rather than manually slicing the old byte stream. + +## Toolchain + +0.16 updates major toolchain components including LLVM 21, musl 1.2.5, glibc 2.43, Linux 6.19 headers, macOS 26.4 headers, MinGW-w64, FreeBSD 15.0 libc, WASI libc, and zig libc/zig cc updates. + +Practical effects: + +- Cross-compilation behavior can change even when Zig code did not. +- Re-check C ABI integration and platform feature detection after upgrading. +- Treat changed C translation output as worthy of investigation, especially for ABI-sensitive or platform bindings. + +## Application Upgrade Checklist + +When upgrading an application to Zig 0.16: + +1. Prefer `pub fn main(init: std.process.Init) !void` for applications and tools. +2. Pass `init.io` down through setup rather than constructing local `Io.Threaded` instances. +3. Use `std.testing.io` in tests. +4. Add `io: std.Io` fields to long-lived systems, allocators, registries, timers, and queues that must perform blocking work later. +5. Use `std.Io.Mutex`/`Condition`/`Semaphore` when the lock can block. Use `lockUncancelable(io)` only for short critical sections where cancellation must not interrupt cleanup or queue integrity. +6. Migrate file operations to `std.Io.Dir`/`std.Io.File`. +7. Keep platform-specific DynamicLib behavior behind a local abstraction on Windows. +8. Preserve translated-C ABI asserts and expand them when translation output changes. +9. Replace old `std.crypto.random` usage with `io.random` or `std.Random.IoSource`. +10. Replace old process helpers with `std.process.run(gpa, io, ...)` or `std.process.spawn(io, ...)`. +11. Replace old `std.Thread.Pool` assumptions with an application scheduler or carefully designed `std.Io.Group` usage. +12. Keep migration notes updated when the codebase adopts application-specific policy decisions.