Add struct types to Halide - #9416
Draft
alexreinking wants to merge 1 commit into
Draft
Conversation
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 1, 2026 17:16
9f99483 to
069ce04
Compare
alexreinking
changed the base branch from
alexreinking/hoist-splitting
to
alexreinking/rfactor-hoisting
September 1, 2026 17:32
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 1, 2026 18:21
069ce04 to
50683f7
Compare
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 1, 2026 19:06
50683f7 to
e9123a1
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## alexreinking/rfactor-hoisting #9416 +/- ##
================================================================
Coverage ? 69.69%
================================================================
Files ? 262
Lines ? 79998
Branches ? 19531
================================================================
Hits ? 55755
Misses ? 18295
Partials ? 5948 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 3, 2026 17:10
e9123a1 to
425cb95
Compare
Introduce first-class packed struct types (Type::Struct) modeled faithfully in the type system: - A dedicated ABI type code halide_type_struct=5; a struct's packed byte size rides in the halide_type_t reserved field, so struct-typed buffers have correct element size/strides. StructTypeInfo (field layout) is interned like handle metadata, keeping Type at 8 bytes. is_uint()/etc. are honestly false for structs, so no numeric special-casing is needed. - field()/pack_struct() intrinsics with byte-addressed lowering (LowerStructTypes), plus per-field pack_struct ergonomics: an array field is filled by a gather() packet, a gather(extent, gen) generator, a single expression with one swept `_` placeholder (index arithmetic allowed), or a field() copy of a whole same-typed field. - Python bindings for the type, field/pack_struct/gather, and the per-field forms. - Tests: correctness (CPU + GPU), error cases, Python, and an ARM codegen test implementing ggml's q4_0/q8_0 dot product that verifies the packed qs arrays lower to dense 128-bit vector loads. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alexreinking
force-pushed
the
alexreinking/struct-types
branch
from
September 4, 2026 09:09
425cb95 to
2740239
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds struct types to Halide. This allows for inputs and outputs to be defined in terms of popular wire formats, even if they aren't prepared for efficient computation. For example, you can define GGML's
q5_0andq8_0formats like so:Type q5_0 = Type::Struct({{"d", Float(16)}, {"qh", UInt(32)}, {"qs", UInt(8), 16}}); Type q8_0 = Type::Struct({{"d", Float(16)}, {"qs", Int(8), 32}});Then you can write a pipeline that consumes them:
ImageParam x{q4_0_type(), 1, "x"}; ImageParam y{q8_0_type(), 1, "y"}; Var b("b"), k("k"), u("u"); RDom r(0, x.dim(0).extent(), 0, 32, "r"); // block, quant // Dequantize one 4-bit weight to float: quant k is a nibble of the // packed byte k % 16 -- its low nibble for k < 16, its high nibble // otherwise -- biased by -8 to signed [-8, 7], times the block delta. Expr nib = field(x(b), "qs")[k % 16]; // uint8 Func x_wt("x_wt"); x_wt(k, b) = cast<float>(field(x(b), "d")) * (cast<int32_t>(select(k < 16, nib % 16, nib / 16)) - 8); // Dequantize one int8 activation to float: quant k is int8 k, times the // block delta. Func y_wt("y_wt"); y_wt(k, b) = cast<float>(field(y(b), "d")) * cast<int32_t>(field(y(b), "qs")[k]); // Dequantize each side and sum every product. Func qdot{"qdot"}; qdot() = 0.0f; qdot() += x_wt(r[1], r[0]) * y_wt(r[1], r[0]);As you can see, struct types support both individual elements, as well as fixed-size array elements. Fields are accessed with a named
field(expr, "name")intrinsic. The tests include additional cases for struct-typed fields and output structs.This adjusts the
Typeandhalide_type_trepresentations in the following ways:Type: likeHandle, aStructtype has a compile-time known field layout.halide_type_t: structs get their own kind, and the reserved field records the number of bytes in the struct.All layouts are expected to be packed. No padding is inserted around or in between field elements.
Breaking changes
None—it's a new feature.
Checklist