Move from custom DALI conda recipe to upstream nvidia-dali-feedstock

[JanuszL]

• uses nvidia-dali-feedstock recipe to build DALI in CI
• removes DALI maintained recipe

Category:

Other (e.g. Documentation, Tests, Configuration)

Description:

• uses nvidia-dali-feedstock recipe to build DALI in CI
• removes DALI maintained recipe

Additional information:

Affected modules and functionalities:

• conda build process

Key points relevant for the review:

• NA

Tests:

• Existing tests apply
  • all test *_conda
• New tests added
  • Python tests
  • GTests
  • Benchmark
  • Other
• N/A

Checklist

Documentation

• Existing documentation applies
• Documentation updated
  • Docstring
  • Doxygen
  • RST
  • Jupyter
  • Other
• N/A

DALI team only

Requirements

• Implements new requirements
• Affects existing requirements
• N/A

REQ IDs: N/A

JIRA TASK: DALI-4680

[dali-automaton]

CI MESSAGE: [55681719]: BUILD STARTED

[greptile-apps]

Greptile Summary

This PR replaces DALI's in-tree custom conda recipe (conda-mambabuild + two hand-maintained meta.yaml/build.sh pairs) with the upstream conda-forge/nvidia-dali-python-feedstock, adapted via a local patch and a pair of sed substitutions to build from the local DALI checkout instead of a release tarball.

• build_conda_packages.sh is rewritten: validates inputs (CUDA_VERSION_MAJOR, DALI_VERSION, DALI_TIMESTAMP, NVIDIA_BUILD_ID) with clear error messages, clones the feedstock (optionally at a pinned SHA via DALI_FEEDSTOCK_SHA), applies nvidia-dali-python-feedstock.patch, patches recipe.yaml in-place with verified sed calls, and invokes rattler-build.
• nvidia-dali-python-feedstock.patch adapts the feedstock: adds rm -rf guards before ln -sf for boost/dlpack/cutlass, injects BUILD_FOR_CONDA, DALI_BUILD_FLAVOR, and DALI_TIMESTAMP into CMake, enables tests and benchmarks, and bumps libnvimgcodec from 0.8.* to 0.9.* across all variant config files (including three aarch64 configs that the current build script does not use).
• All seven previously-maintained recipe files are deleted.

Confidence Score: 4/5

Safe to merge for the linux-64 path, but the patch embeds context-sensitive hunks for three aarch64 YAML files that the build script never uses, meaning an unrelated upstream feedstock change to those files would break linux-64 CI.

The linux-64 build path is clean and well-guarded with input validation and sed verification. The one concrete defect is that nvidia-dali-python-feedstock.patch includes hunks for three aarch64 .ci_support YAML files that build_conda_packages.sh never selects. git apply is atomic, so any upstream change to those files' context fails the entire patch and blocks linux-64 CI with no linux-64 change involved. The risk is present whenever DALI_FEEDSTOCK_SHA is not pinned, which is the default for ordinary CI runs.

conda/nvidia-dali-python-feedstock.patch — the aarch64 YAML hunks that create the fragile coupling; conda/build_conda_packages.sh — the git apply call that becomes the failure point.

Important Files Changed

Filename	Overview
conda/build_conda_packages.sh	Replaces conda-mambabuild flow with rattler-build using the upstream nvidia-dali-python-feedstock; adds robust input validation and sed-with-verification recipe patching, but the patch file modifies aarch64 YAML files never used by this script, creating a fragile git apply dependency on unrelated upstream changes.
conda/nvidia-dali-python-feedstock.patch	Adapts the feedstock recipe for local DALI builds: adds rm -rf guards before symlink creation, enables BUILD_FOR_CONDA, enables tests/benchmarks, updates libnvimgcodec to 0.9.* across all variant configs, and injects version/flavor metadata; the three aarch64 config hunks are not needed for the linux-64 CI path.
conda/dali_native_libs/recipe/meta.yaml	Deleted — replaced by the upstream feedstock recipe.
conda/dali_native_libs/recipe/build.sh	Deleted — complex CMake/patchelf logic replaced by the feedstock's own build.sh (with DALI-specific patches applied via nvidia-dali-python-feedstock.patch).
conda/dali_python_bindings/recipe/meta.yaml	Deleted — python bindings recipe superseded by the single upstream feedstock recipe.
conda/dali_python_bindings/recipe/build.sh	Deleted — python bindings build script superseded by the feedstock.
conda/config/conda_build_config.yaml	Deleted — compiler/stdlib/numpy pinnings now governed by the feedstock's .ci_support variant YAML files.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[build_conda_packages.sh] --> B{Validate inputs\nCUDA_VERSION_MAJOR\nDALI_VERSION\nDALI_TIMESTAMP\nNVIDIA_BUILD_ID}
    B -->|invalid| FAIL1[exit 1]
    B -->|valid| C[Select METAL_YAML\nlinux_64 CUDA 12.9 or 13.0]
    C --> D[git clone nvidia-dali-python-feedstock]
    D --> E{DALI_FEEDSTOCK_SHA set?}
    E -->|yes| F[git checkout SHA]
    E -->|no| G[use HEAD]
    F --> H[git apply nvidia-dali-python-feedstock.patch]
    G --> H
    H -->|check fails| FAIL2[exit 1]
    H -->|ok| I[sed: replace tarball URL\nwith /opt/dali path]
    I --> J[sed: add use_gitignore: false]
    J --> K[rattler-build build\n--target-platform linux-64\n-m METAL_YAML]
    K --> L[cp output/linux-64/*.conda\nto /opt/dali/conda/artifacts/]

Loading

_{Reviews (15): Last reviewed commit: "Update conda CI nvImageCodec pins" | Re-trigger Greptile}

[dali-automaton]

CI MESSAGE: [55681719]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [55685089]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [55685089]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [55688559]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [55688559]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [55693916]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [55694466]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [55696184]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [57832399]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [57832399]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [57957810]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [57957810]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [57967556]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [57967556]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [58002631]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [58002631]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [58097163]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [58097163]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [58105433]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [58105433]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [58114829]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [58114829]: BUILD PASSED

[dali-automaton]

CI MESSAGE: [59219619]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [59219619]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [59248913]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [59248913]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [59272399]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [59272399]: BUILD PASSED