[WebGPU] Fix FP16 MatMulNBits wide-tile results on Qualcomm D3D12

[Yang Gu (gyagp)]

Description

Fix incorrect FP16 MatMulNBitsWideTile results on Qualcomm D3D12 while preserving subgroup shuffles and the 32-row tile. Bitcast each vec4<f16> to vec2<u32> for subgroupShuffle, then bitcast the result back. This transports the exact FP16 bits and leaves storage, arithmetic and accumulation precision unchanged.

Enable this only for Qualcomm + D3D12 + FP16, when subgroup shuffles are available. Include the choice in the shader cache key. Other vendors and backends retain their existing shader paths; the NVIDIA direct-read preference is unchanged. Test tolerances and selection are unchanged.

Motivation and Context

The affected path was introduced by #31703, commit 1bc68c1d25b374e13bbf9a0649441af1ad9f1501, merged August 13, 2026. The two tests added in #32796 expose the older issue:

• WebGpuMatMulNBitsTest.Float16_4Bits_SubgroupMatrixTiles
• WebGpuMatMulNBitsTest.Float16_8Bits_SubgroupMatrixTiles

They use synthetic operator inputs, not a language model. At (M,N,K)=(32,128,64), independent reference checks found 2,675 INT4 and 2,677 INT8 mismatches out of 4,096 outputs. Both October 14 and October 19 runtimes reproduce the same errors.

Isolating the cause

Rebuilding the before/after #31703 kernels with the same current ORT/Dawn/compiler dependencies, in two interleaved repetitions, gives zero mismatches before the optimization and the same failures afterward. This identifies the affected source change; the complete historical dependency stack was not rebuilt.

A separate subgroup-broadcast probe removes matrix arithmetic and checks known FP16 payloads loaded from storage and workgroup memory. Qualcomm returns adjacent components swapped: [x,y,z,w] -> [y,x,w,z]. All 32,512 checked values per floating-point mode/source match that exact permutation. Packed u32 transport preserves every value.

Device	Actual subgroup size	FP16 scalar/vector shuffles	Packed u32 shuffles
Adreno X1-85, ARM64 runtime	64	Adjacent components swapped	Pass
Same Adreno, x64 runtime under emulation	64	Same failure	Pass
Radeon RX 9070 XT, x64	64	Pass	Pass
Intel Panther Lake, x64	32	Pass	Pass
GeForce RTX 5080, x64	32	Pass	Pass

The AMD and Qualcomm x64 probes use identical WGSL bodies, ORT/DXC binaries and 64-lane subgroups. All source lane indices in the primitive probe are valid. The failure therefore reproduces independently of MatMul arithmetic and indexing; a 64-lane subgroup alone does not explain it, and it also reproduces outside the ARM64 build.

Reducing the tile to 16 rows, splitting vector loads, and widening FP16 values before a floating-point shuffle did not repair the error. Packed integer transport does, without disabling the optimization. Adreno validation used driver 31.0.133.1, Windows 11 build 26300.9711.

Attribution and workaround scope

A standalone D3D12 reproduction confirms the failure in Qualcomm's D3D12 driver/compiler/execution path. Handwritten HLSL was compiled once on the build host; Qualcomm and AMD then executed the exact same hash-verified DXIL, without ORT, Dawn/Tint or device-side DXC compilation. Both ran 64-lane waves for the shuffle variants. Every one of the 65,536 checked floating-point components per variant/source/repetition is swapped on Qualcomm; AMD has zero mismatches. Packed-integer shuffles and direct reads pass on both devices. Two interleaved repetitions agree.

The inspected DXIL loads and extracts components 0, 1, 2, 3, calls waveReadLaneAt.f16 on the corresponding values with valid source lanes, and stores the results in the original order. It contains no component permutation. This places the observed swap below the offline DXIL compilation boundary. The exact internal Qualcomm compiler pass or GPU instruction responsible is still unknown; a general ORT MatMul algorithm change is not indicated by this evidence.

The new local Dawn end-to-end test, SubgroupsF16ShuffleTests.DistinctComponents, also reproduces the problem on Dawn 160806fc202be2725ca6535605ade24bc9721b1f (October 19). It uses distinct buffer-supplied FP16 values and exact CPU expectations, testing scalar, vector, widened-FP32 and packed-integer shuffles from storage and workgroup memory. Qualcomm has 6 failures / 2 passes / 0 skips; AMD has 8 passes / 0 failures / 0 skips, in each of two repetitions. This test is maintained in the local Dawn checkout and is not part of this ORT source diff.

The direct-D3D12 shader uses -E main -T cs_6_2 -HV 2021 -enable-16bit-types -O3. The storage-vector DXIL SHA-256 is b0ea63a9d0515f348b3d0e09b98e508d289b435d547f6f67577d62ac7b4a93cb. Qualcomm validation used Adreno X1-85 driver 31.0.133.1; the AMD control used Radeon RX 9070 XT driver 32.0.31007.2064.

ORT #31703 introduced the path that exposes this failure on the tested stack. This PR provides an ORT-side compatibility workaround, scoped to Qualcomm D3D12 FP16. It preserves the subgroup algorithm by transporting the payload as integer bits:

bitcast<vec4<f16>>(subgroupShuffle(bitcast<vec2<u32>>(value), lane))

No numerical conversion is involved. FP32 shuffles of values originating from FP16 also failed in the probe, so widening was insufficient; that observation does not establish a failure for arbitrary FP32 inputs.

Validation

Windows Release, based on c1ea5bffb8 with this patch. ORT DLLs and statically linked provider-test executables rebuilt for ARM64 and x64.

Device / architecture	Passed	Failed	Skipped
Qualcomm Adreno X1-85 / ARM64	814	0	9
AMD Radeon RX 9070 XT / x64	816	0	7
Intel Panther Lake / x64	818	0	5
NVIDIA GeForce RTX 5080 / x64	816	0	7

• All 823 compiled WebGPU-only cases accounted for per device; 17 upstream-disabled cases remain excluded.
• Existing MatMulNBits tests cover 80 shape/bias/zero-point combinations, with unchanged tolerances.
• Independent INT4/INT8 fixtures and DP4A controls pass; shader dumps verify that Qualcomm still uses subgroup shuffles, now with packed integer transport.
• C++ formatting, git diff --check, and 23 local runner/build/patch-provenance checks pass.
• Throughput impact has not been benchmarked; this change preserves the shuffle algorithm rather than switching to direct reads.

Targeted reproduction from the provider-test output directory:

.\onnxruntime_provider_test.exe --gtest_filter="WebGpuMatMulNBitsTest.Float16_4Bits_SubgroupMatrixTiles:WebGpuMatMulNBitsTest.Float16_8Bits_SubgroupMatrixTiles"

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.