Skip to content

Move ArrayRecord OSS builds to riegeli 0.0.0-20261117-64d553d and Bazel 8. - #236

Merged
copybara-service[bot] merged 1 commit into
mainfrom
test_1025490093
Nov 20, 2026
Merged

copybara-service[bot] merged 1 commit into
mainfrom
test_1025490093

Conversation

@copybara-service

@copybara-service copybara-service Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Move ArrayRecord OSS builds to riegeli 0.0.0-20261117-64d553d and Bazel 8.

The new riegeli release on BCR adds Reader::NewReader(initial_pos, length_hint),
which upcoming ArrayRecord changes use for bounded GCS reads. Moving to it needs
several OSS build changes at once:

  • The other direct dependencies in MODULE.bazel are raised to the versions
    riegeli requires (abseil-cpp, protobuf, rules_cc, rules_python, platforms,
    googletest, pybind11_bazel). The build sets
    --check_direct_dependencies=error, so they must match.
  • CI and the BCR presubmit move from Bazel 7 to Bazel 8: rules_python 2.4.1
    does not load on Bazel 7.
  • Linux wheels are built on manylinux_2_28 instead of manylinux2014,
    because Bazel 8 needs a newer libstdc++ than the manylinux2014 image has.
    Prebuilt Linux wheels now need glibc 2.28 or newer, so CentOS/RHEL 7,
    Amazon Linux 2, and Ubuntu 18.04 lose them. manylinux2014 is end-of-life
    together with CentOS 7.
  • SequencedChunkWriter pads with WritePadding(kImplicitPadding) in OSS
    too, since the new riegeli removed PadToBlockBoundary().
  • macOS CI moves from macos-14, which GitHub Actions no longer runs for
    this repository, to macos-15. pypa/gh-action-pypi-publish is pinned to
    v1.14.3, which fixes GHSA-5xjx-3w67-w9x9.
  • The wheel build skips the *.venv directories rules_python 2.x creates
    for py_test targets. Their python3 symlink does not resolve outside of
    runfiles, and the macOS rsync -L fails on it.

ArrayRecordReader now uses insecure credentials for gs:// paths when
CLOUD_STORAGE_EMULATOR_ENDPOINT is set. google-cloud-cpp 3.x documents this
for storage::Client(), but its default options replace these credentials with
Application Default Credentials. Outside of GCE those look up the metadata
server, and the read is retried until the retry policy expires. With the new
google-cloud-cpp, this made the GCS emulator tests in array_record_module_test
time out. This workaround can be removed once
googleapis/google-cloud-cpp#16678 is fixed (proposed
fix: googleapis/google-cloud-cpp#16687) and the OSS
build uses a google-cloud-cpp release with it.

@google-cla

google-cla Bot commented Nov 19, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@copybara-service
copybara-service Bot force-pushed the test_1025490093 branch 9 times, most recently from 9ef044e to 594b513 Compare November 20, 2026 12:26
…el 8.

The new riegeli release on BCR adds `Reader::NewReader(initial_pos, length_hint)`,
which upcoming ArrayRecord changes use for bounded GCS reads. Moving to it needs
several OSS build changes at once:

*   The other direct dependencies in `MODULE.bazel` are raised to the versions
    riegeli requires (abseil-cpp, protobuf, rules_cc, rules_python, platforms,
    googletest, pybind11_bazel). The build sets
    `--check_direct_dependencies=error`, so they must match.
*   CI and the BCR presubmit move from Bazel 7 to Bazel 8: rules_python 2.4.1
    does not load on Bazel 7.
*   Linux wheels are built on `manylinux_2_28` instead of `manylinux2014`,
    because Bazel 8 needs a newer libstdc++ than the manylinux2014 image has.
    Prebuilt Linux wheels now need glibc 2.28 or newer, so CentOS/RHEL 7,
    Amazon Linux 2, and Ubuntu 18.04 lose them. manylinux2014 is end-of-life
    together with CentOS 7.
*   `SequencedChunkWriter` pads with `WritePadding(kImplicitPadding)` in OSS
    too, since the new riegeli removed `PadToBlockBoundary()`.
*   macOS CI moves from `macos-14`, which GitHub Actions no longer runs for
    this repository, to `macos-15`. `pypa/gh-action-pypi-publish` is pinned to
    v1.14.3, which fixes GHSA-5xjx-3w67-w9x9.
*   The wheel build skips the `*.venv` directories rules_python 2.x creates
    for `py_test` targets. Their `python3` symlink does not resolve outside of
    runfiles, and the macOS `rsync -L` fails on it.

`ArrayRecordReader` now uses insecure credentials for `gs://` paths when
`CLOUD_STORAGE_EMULATOR_ENDPOINT` is set. google-cloud-cpp 3.x documents this
for `storage::Client()`, but its default options replace these credentials with
Application Default Credentials. Outside of GCE those look up the metadata
server, and the read is retried until the retry policy expires. With the new
google-cloud-cpp, this made the GCS emulator tests in `array_record_module_test`
time out. This workaround can be removed once
googleapis/google-cloud-cpp#16678 is fixed (proposed
fix: googleapis/google-cloud-cpp#16687) and the OSS
build uses a google-cloud-cpp release with it.
PiperOrigin-RevId: 1026386371
@copybara-service
copybara-service Bot merged commit 2d1c9c1 into main Nov 20, 2026
3 of 4 checks passed
@copybara-service
copybara-service Bot deleted the test_1025490093 branch November 20, 2026 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants