Skip to content

Add struct types to Halide - #9416

Draft
alexreinking wants to merge 1 commit into
alexreinking/rfactor-hoistingfrom
alexreinking/struct-types
Draft

Add struct types to Halide#9416
alexreinking wants to merge 1 commit into
alexreinking/rfactor-hoistingfrom
alexreinking/struct-types

Conversation

@alexreinking

@alexreinking alexreinking commented Sep 1, 2026

Copy link
Copy Markdown
Member

This PR adds struct types to Halide. This allows for inputs and outputs to be defined in terms of popular wire formats, even if they aren't prepared for efficient computation. For example, you can define GGML's q5_0 and q8_0 formats like so:

Type q5_0 = Type::Struct({{"d", Float(16)}, {"qh", UInt(32)}, {"qs", UInt(8), 16}});
Type q8_0 = Type::Struct({{"d", Float(16)}, {"qs", Int(8), 32}});

Then you can write a pipeline that consumes them:

ImageParam x{q4_0_type(), 1, "x"};
ImageParam y{q8_0_type(), 1, "y"};

Var b("b"), k("k"), u("u");
RDom r(0, x.dim(0).extent(), 0, 32, "r");  // block, quant

// Dequantize one 4-bit weight to float: quant k is a nibble of the
// packed byte k % 16 -- its low nibble for k < 16, its high nibble
// otherwise -- biased by -8 to signed [-8, 7], times the block delta.
Expr nib = field(x(b), "qs")[k % 16];  // uint8
Func x_wt("x_wt");
x_wt(k, b) = cast<float>(field(x(b), "d")) * 
             (cast<int32_t>(select(k < 16, nib % 16, nib / 16)) - 8);

// Dequantize one int8 activation to float: quant k is int8 k, times the
// block delta.
Func y_wt("y_wt");
y_wt(k, b) = cast<float>(field(y(b), "d")) * 
             cast<int32_t>(field(y(b), "qs")[k]);

// Dequantize each side and sum every product.
Func qdot{"qdot"};
qdot() = 0.0f;
qdot() += x_wt(r[1], r[0]) * y_wt(r[1], r[0]);

As you can see, struct types support both individual elements, as well as fixed-size array elements. Fields are accessed with a named field(expr, "name") intrinsic. The tests include additional cases for struct-typed fields and output structs.

This adjusts the Type and halide_type_t representations in the following ways:

  1. Type: like Handle, a Struct type has a compile-time known field layout.
  2. halide_type_t: structs get their own kind, and the reserved field records the number of bytes in the struct.

All layouts are expected to be packed. No padding is inserted around or in between field elements.

Breaking changes

None—it's a new feature.

Checklist

  • Tests added or updated (not required for docs, CI config, or typo fixes)
  • Documentation updated (if public API changed)
  • Python bindings updated (if public API changed)
  • Benchmarks are included here if the change is intended to affect performance.
  • Commits include AI attribution where applicable (see Code of Conduct)

@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 9f99483 to 069ce04 Compare September 1, 2026 17:16
@alexreinking
alexreinking changed the base branch from alexreinking/hoist-splitting to alexreinking/rfactor-hoisting September 1, 2026 17:32
@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 069ce04 to 50683f7 Compare September 1, 2026 18:21
@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 50683f7 to e9123a1 Compare September 1, 2026 19:06
@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 18.25558% with 403 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (alexreinking/rfactor-hoisting@3c6ea1b). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/LowerStructTypes.cpp 20.49% 118 Missing and 10 partials ⚠️
src/IROperator.cpp 5.38% 123 Missing ⚠️
src/Type.cpp 25.00% 69 Missing and 12 partials ⚠️
src/ConstantBounds.cpp 0.00% 15 Missing ⚠️
src/Type.h 48.00% 9 Missing and 4 partials ⚠️
src/IRPrinter.cpp 0.00% 10 Missing and 2 partials ⚠️
src/IROperator.h 0.00% 7 Missing ⚠️
src/CodeGen_Vulkan_Dev.cpp 0.00% 4 Missing ⚠️
src/runtime/HalideRuntime.h 0.00% 1 Missing and 2 partials ⚠️
src/CodeGen_C.cpp 77.77% 0 Missing and 2 partials ⚠️
... and 9 more
Additional details and impacted files
@@                       Coverage Diff                        @@
##             alexreinking/rfactor-hoisting    #9416   +/-   ##
================================================================
  Coverage                                 ?   69.69%           
================================================================
  Files                                    ?      262           
  Lines                                    ?    79998           
  Branches                                 ?    19531           
================================================================
  Hits                                     ?    55755           
  Misses                                   ?    18295           
  Partials                                 ?     5948           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from e9123a1 to 425cb95 Compare September 3, 2026 17:10
Introduce first-class packed struct types (Type::Struct) modeled
faithfully in the type system:

- A dedicated ABI type code halide_type_struct=5; a struct's packed byte
  size rides in the halide_type_t reserved field, so struct-typed buffers
  have correct element size/strides. StructTypeInfo (field layout) is
  interned like handle metadata, keeping Type at 8 bytes. is_uint()/etc.
  are honestly false for structs, so no numeric special-casing is needed.

- field()/pack_struct() intrinsics with byte-addressed lowering
  (LowerStructTypes), plus per-field pack_struct ergonomics: an array
  field is filled by a gather() packet, a gather(extent, gen) generator,
  a single expression with one swept `_` placeholder (index arithmetic
  allowed), or a field() copy of a whole same-typed field.

- Python bindings for the type, field/pack_struct/gather, and the
  per-field forms.

- Tests: correctness (CPU + GPU), error cases, Python, and an ARM codegen
  test implementing ggml's q4_0/q8_0 dot product that verifies the packed
  qs arrays lower to dense 128-bit vector loads.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alexreinking
alexreinking force-pushed the alexreinking/struct-types branch from 425cb95 to 2740239 Compare September 4, 2026 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant