Skip to content

Releases: vortex-data/vortex

Release list

0.84.0

0.84.0 Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 07 Aug 16:26
0.84.0
7e06a99
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Changes

⚠️ Breaks

✨ Features

🚀 Performance

  • Skip materializing splitless chunk children during split collection (#9271) @AdamGS
  • Improve take on Chunked(FSL) and FSL(Chunked) array alongside implementation of take using PiecewiseSequence (#8881) @robert3005
  • Lazily untranspose DeltaArray validity take #2 (#9226) @robert3005
  • Optimize ChunkedArray take routing (#9224) @gatesn
  • Optimize split calculation in DF source (#9167) @AdamGS
  • Vectorize checked multiplication by reporting evidence, not a comparison (#9210) @connortsui20
  • Add scalar function performance baselines (#9136) @connortsui20
  • duckdb: don't use tokio for local reads (#9175) @myrrc
  • Shrink zstd buffer before returning (#9179) @halvorboe
  • perf(vortex-geo): short-circuit predicates with bounding boxes (#9076) @HarukiMoriarty
  • perf(onpair): adopt onpair 0.2 storage-backed dictionary and safety-only validation (#9117) @gargiulofrancesco
  • perf(array): stop flattening appended list views in ListViewBuilder [builders-child-stack] (#9046) @claude[bot]
  • block_on_stream_thread_safe: increase channel size to number of cores (#9116) @myrrc
  • Use unchecked ListViewArray construction after validation (#9044) @robert3005
  • optimize the array creation (slots) (#9013) @joseph-isaacs
  • Unify numeric compute kernels with vortex-compute iteration functions (#8939) @robert3005
  • Execute until arrays match fast path or canonical before exporting to Arrow (#8935) @mhk197
  • Back EnabledEditions with a session Registry (#8956) @joseph-isaacs
  • feat(expr): fold cast of a literal to a literal in Cast simplify (#8962) @HarukiMoriarty
  • Enable metadata-only mask reduction for constant Boolean masks (#8934) @connortsui20

🐛 Bug Fixes

30 changes
  • duckdb: don't panic on unsupported Value type (#9243) @myrrc
  • duckdb: Restrict pushed aggregations (#9233) @myrrc
  • Fix decimal scalar multiplication and division (#8888) @mhk197
  • duckdb: convert aggregate results to duckdb requested output type (#9231) @myrrc
  • Validate row-index selections (#8574) @gatesn
  • duckdb: TIMESTAMPTZ keeps microsecond precision (#9241) @myrrc
  • duckdb: don't return stats() for aggregate scans and scans with pushed projection expression (#9230) @myrrc
  • Fix .take() overflow on VarBin with >4GB data (#9227) @myrrc
  • Fill in missing functionality for maps in fuzzer (#9208) @AdamGS
  • Remove the unimplemented NotKernel extension point (#9214) @connortsui20
  • fix(buffer): bound alignment exponents read from untrusted input (#9155) @lorenzhs
  • Panic on DType::Null conversion to NonNullable (#9207) @myrrc
  • Upgrade to fsst 0.6.0 and guard against out of bounds codes (#9149) @robert3005
  • fix(spark): write null array elements as nulls instead of zeros (#9169) @jackylee-ch
  • fix(spark): delete aborted output through the native filesystem layer (#9172) @jackylee-ch
  • Use all cores for parquet->vortex conversion in duckdb-bench (#9165) @myrrc
  • fix(zstd): validate frame content sizes (#9164) @lorenzhs
  • fix get_item parent reduce rule (#9151) @mhk197
  • Harden RLE decompression against untrusted indices and offsets (#8953) @joseph-isaacs
  • Reenable float aggregation pushdown for duckdb except for min() (#9094) @myrrc
  • fix: validate unsigned AVX2 take indices (#9036) @connortsui20
  • mean(all null) = NULL (#9079) @myrrc
  • fix: LazyScanStream prepare/execute is CPU-work not I/O (#9078) @robert3005
  • Mark file-level min/max utf8/binary stats exactness correctly (#9088) @AdamGS
  • duckdb: don't push down aggregation over floats (#9069) @myrrc
  • make alp-rd dicts deterministic on equal frequency entries (#9032) @onursatici
  • Fix escaped field names in calculate_physical_schema (#9049) @brancz
  • fix: Fix datafusion benchmark, attaching table ref to table URL (#9014) @AdamGS
  • fix(cuda): pass mutated CUDA device buffers as mut (#9001) @0ax1
  • fix(vortex-row): decimal sort keys are not memcmp-comparable across chunks (#8937) @HarukiMoriarty

📖 Documentation

6 changes

🧰 Maintenance

47 changes
Read more

Contributors

lorenzhs, robert3005, and 19 other contributors
Assets 6
Loading

0.83.0

0.83.0

Choose a tag to compare

View all tags
@0ax1 0ax1 released this 24 Jul 15:07
0.83.0
14a55e9
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

Full Changelog: 0.82.0...0.83.0

Contributors

0ax1 and joseph-isaacs
Loading

0.82.0

0.82.0

Choose a tag to compare

View all tags
@0ax1 0ax1 released this 24 Jul 13:06
0.82.0
299592d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

New Contributors

Full Changelog: 0.81.0...0.82.0

Contributors

robert3005, 0ax1, and 6 other contributors
Loading

0.81.0

0.81.0

Choose a tag to compare

View all tags
@myrrc myrrc released this 23 Jul 10:43
0.81.0
7654891
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

Full Changelog: 0.80.0...0.81.0

Contributors

robert3005, joseph-isaacs, and 3 other contributors
Loading

0.80.0

0.80.0

Choose a tag to compare

View all tags
@0ax1 0ax1 released this 22 Jul 12:36
0.80.0
037b1a8
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

New Contributors

Full Changelog: 0.79.0...0.80.0

Contributors

danking, robert3005, and 11 other contributors
Loading

0.79.0

0.79.0

Choose a tag to compare

View all tags
@0ax1 0ax1 released this 17 Jul 09:58
0.79.0
c981534
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

New Contributors

Full Changelog: 0.78.0...0.79.0

Contributors

claude, a10y, and 15 other contributors
Loading

0.78.0

0.78.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 09 Jul 16:50
0.78.0
aff0bd3
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Changes

✨ Features

  • Implement compare in vortex instead of falling back to arrow (#8661) @robert3005

🚀 Performance

  • Ungrouped aggregate function pushdown for duckdb (#8645) @myrrc

🐛 Bug Fixes

  • fix: accept FixedSizeList dtype when deserializing ListValue scalars (#8696) @brancz
  • Audit panic handling in detached tasks (#8677) @AdamGS

🧰 Maintenance

  • Split TableStrategy dispatcher from StructStrategy (#8638) @mhk197

Contributors

robert3005, brancz, and 3 other contributors
Loading

0.77.0

0.77.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 08 Jul 17:58
0.77.0
c7c28d8
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Changes

NOTE: 0.76.0 was yanked, because of a (hopefully very rare) latent correctness bug, that became much easier to hit in 0.76.0. There was a long-standing incorrect behavior in the TableStrategy writer, wherein it would not correctly propagate panics from the parallel writer subtasks. The fix was in #8672.

If you passed a stream of well-formed valid arrays straight to the builtin vortex writers, you are definitely unaffected.

If any of the following are true, you may have files that were silently truncated, in part (all versions) or in whole (0.76.0):

  • you had a stream of well-formed arrays, AND called map on that ArrayStream with a function that panicked on invocation, then passed the resulting stream to a builtin vortex writer
  • you passed a stream of malformed arrays that could have triggered a panic or assertion failure (e.g., you constructed invalid vortex arrays using unsafe/new_unchecked; to our knowledge you can't hit this path with the safe/regular constructors)

If you believe you may have been affected by this issue, please reach out in slack: https://vortex.dev/slack

⚠️ Breaks

✨ Features

🚀 Performance

🐛 Bug Fixes

  • fix: TableStrategy currently hides panic in spawned column writer tasks. (#8672) @AdamGS
  • Grow list builders as we append elements to it (#8673) @robert3005
  • Fix vortex-jni local path handling, centralise logic in vortex-file (#8650) @robert3005

🧰 Maintenance

13 changes

Contributors

robert3005, 0ax1, and 5 other contributors
Loading

0.76.0

0.76.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 01 Jul 14:56
0.76.0
0a45777
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Changes

⚠️ Breaks

✨ Features

  • Add swimlane view to vortex-web that visualizes physical layout of columns in files (#8634) @robert3005
  • feat(vortex-bench): infra of SpatialBench on DuckDB, plain binary (#8598) @HarukiMoriarty
  • Expose array buffer replacement (#8606) @gatesn
  • bool -> Primitive casts (#8621) @myrrc
  • feat: add metadata bridge for Python CUDA export (#8604) @0ax1
  • Add list_length scalar function (#8495) @mhk197
  • feat(file): make Footer::new public (#8602) @tobias-fire
  • feat(array): push struct validity into children (#8589) @miniex
  • Support session-time source configuration in DataFusion, and clear up precedence between config sources (#8575) @AdamGS
  • feat(python): package CUDA as an optional extension (#8510) @0ax1
  • Decimal casting (#8562) @gatesn
  • Add JSON to Parquet Variant conversion with shredding (#8391) @AdamGS
  • Add extension storage scalar function (#8540) @gatesn
  • Numerical aggregate functions have an option to skip or include nans in calculation, skip by default (#8457) @robert3005
  • feat(vortex-geo): ST_Distance via the geo crate + Polygon type (#8497) @HarukiMoriarty
  • feat[gpu]: arrow device array stream support (#8483) @0ax1
  • feat(vortex-datafusion): struct scalar conversion + extension-over-struct scan (#8453) @HarukiMoriarty
  • feat: DeltaScheme adjustable compression ratio threshold (#8461) @joseph-isaacs
  • Add Delta scheme to the integer compressor (unstable_encodings) (#8212) @joseph-isaacs
  • Push down some expressions to Dict layout reader's cached values (#8341) @myrrc
  • --patterns flag for random-access-bench (#8446) @myrrc
  • ffi: create data source from memory buffer (#8426) @myrrc
  • Constant comparison and byte_length OnPair kernels (#8371) @myrrc
  • REGEX support for sqllogictests (#8392) @myrrc
  • feat(vortex-geo): Arrow import/export for the native Point type (#8374) @HarukiMoriarty

🚀 Performance

🐛 Bug Fixes

21 changes
  • Convert extension type scalars to Value correctly (#8635) @myrrc
  • fix(vortex-bench): map gs:// scheme to gcs storage label (#8630) @brancz
  • fix: CUDA bitpacked sliced output allocation (#8622) @0ax1
  • Fix wasm32 build by gating MultiFileSession on non-wasm targets (#8612) @robert3005
  • fix: support 64K alignment (#8599) @a10y
  • Guard calling is_nan on scalar value on dtype being a float (#8593) @robert3005
  • vortex-datafusion: Pipe session through to converter (#8591) @brancz
  • Save children dtypes/length in ZstdBuffersMetadata (#8572) @myrrc
  • fix(array): don't panic on unsupported arrow types (#8564) @miniex
  • fix(layout): don't panic collecting an empty stream (#8472) @miniex
  • Correctly calculate FSST compressed output size (#8551) @robert3005
  • refactor[fsst]: take ArrayRef in fsst_compress, decide offset width upfront (#7900) @mprammer
  • Move optimizer kernels into a dedicated KernelSession (#8511) @gatesn
  • Fix is_zero() for FixedSizeList and Struct scalar types (#8500) @connortsui20
  • Handle null lanes in primitive take (#8505) @gatesn
  • Preserve dict referenced values metadata (#8499) @gatesn
  • fix: preserve operand width in DecimalValue checked arithmetic (#7380) @abnobdoss
  • fix Fuzz constant OOM for CUDA (#8481) @a10y
  • vortex-tui: query tab should work for non-*.vortex files (#8473) @a10y
  • Ignore PyO3 rustsec (#8381) @connortsui20
  • fix: vx_data_source_new_buffer sem-merge conflict (#8437) @joseph-isaacs

📖 Documentation

🧰 Maintenance

49 changes

Contributors

lukekim, a10y, and 14 other contributors
Loading

0.75.0

0.75.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 12 Jun 15:24
0.75.0
7b53625
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Changes

⚠️ Breaks

🚧 Deprecation

✨ Features

🚀 Performance

🐛 Bug Fixes

13 changes

📖 Documentation

🧰 Maintenance

40 changes

Contributors

a10y, robert3005, and 11 other contributors
Loading