Skip to content

HUGEINT/UHUGEINT max-value overflows decimal128(38, 0) on RecordBatch return path #32

Description

@rustyconover

Symptom

Round-tripping a HUGEINT or UHUGEINT column through the example.echo
table-in-out fixture (or any other path that re-serialises an Arrow batch
on the way back to DuckDB) raises:

vgi_rpc.utils.IPCError: IPC batch validation failed: In column 1: Invalid:
Decimal value 170141183460469231731687303715884105727 does not fit in
precision of decimal128(38, 0)
[worker: uv run --project vgi-fixture-worker]

Reproduction

Discovered while writing pushdown coverage in
vgi/test/sql/integration/filter_pushdown/hugeint.test.
The HUGEINT max literal 170141183460469231731687303715884105727 (2^127 − 1, 39 digits) overflows the decimal128(38, 0) PyArrow type used on the echo
return path. Same issue with UHUGEINT max
340282366920938463463374607431768211455 (2^128 − 1, 39 digits).

SELECT * FROM example.echo((SELECT * FROM (VALUES
  (1, 170141183460469231731687303715884105727::HUGEINT)
) v(n, h)));

Where to look

The conversion happens on the worker side when echo's pass-through builds
a Record Batch from the input. HUGEINT is registered to round-trip through
the DuckDB.hugeint extension type
(vgi/duckdb/src/common/arrow/arrow_type_extension.cpp:558)
backed by fixed_size_binary(16), so the storage does fit; the
decimal128(38, 0) only appears on echo's return-path serialisation. Look
at how vgi-rpc's IPC validator infers a target Arrow type for the columns
emitted by the worker — decimal128(38, 0) is one digit short of what
HUGEINT/UHUGEINT can hold.

Workaround in tests

The new hugeint.test exercises a 38-digit value
(99999999999999999999999999999999999999) instead of the type max. That
covers the kernel-pair fix without tripping the return-path bug.

Priority

Low — only affects the absolute boundary literals, very rare in real data.
But quietly silent-failing on max values is bad enough to deserve a fix or
a clear error.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions