Skip to content

ci: skip cmu nav tests - #3559

Draft
aclauer wants to merge 5 commits into
mainfrom
andrew/chore/disable-cmu-nav-tests
Draft

ci: skip cmu nav tests#3559
aclauer wants to merge 5 commits into
mainfrom
andrew/chore/disable-cmu-nav-tests

Conversation

@aclauer

@aclauer aclauer commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Contribution path

  • Small, safe change that does not need a tracking issue
  • Linked issue or discussion: DIM-XXX / #XXX / URL

Problem

We aren't working on cmu nav right now and the tests + caching take a lot of CI time

Solution

Skip the cachix pull stage and skip tests.

How to Test

run ci

AI assistance

Checklist

  • I have read and approved the CLA.

@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

@@            Coverage Diff             @@
##             main    #3559      +/-   ##
==========================================
- Coverage   76.40%   75.56%   -0.84%     
==========================================
  Files        1244     1245       +1     
  Lines      120503   120514      +11     
  Branches    10737    10868     +131     
==========================================
- Hits        92069    91072     -997     
- Misses      25333    26382    +1049     
+ Partials     3101     3060      -41     
Flag Coverage Δ
OS-ubuntu-24.04-arm 70.62% <100.00%> (-0.27%) ⬇️
OS-ubuntu-latest 72.38% <100.00%> (-0.27%) ⬇️
Py-3.10 72.37% <100.00%> (-0.27%) ⬇️
Py-3.11 72.37% <100.00%> (-0.27%) ⬇️
Py-3.12 72.37% <100.00%> (-0.27%) ⬇️
Py-3.13 72.37% <100.00%> (-0.26%) ⬇️
Py-3.14 72.37% <100.00%> (-0.26%) ⬇️
Py-3.14t 72.37% <100.00%> (-0.27%) ⬇️
SelfHosted-Large 29.81% <100.00%> (+<0.01%) ⬆️
SelfHosted-Linux 35.10% <100.00%> (-0.70%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...nav/modules/far_planner/test_far_planner_rosbag.py 40.74% <100.00%> (-54.26%) ⬇️
...modules/local_planner/test_local_planner_rosbag.py 31.52% <100.00%> (-61.89%) ⬇️
...modules/path_follower/test_path_follower_rosbag.py 28.26% <100.00%> (-65.15%) ⬇️
.../navigation/cmu_nav/modules/pgo/test_pgo_rosbag.py 26.27% <100.00%> (-70.05%) ⬇️
..._nav/modules/simple_planner/test_simple_planner.py 21.22% <100.00%> (-78.78%) ⬇️
...s/terrain_analysis/test_terrain_analysis_rosbag.py 32.83% <100.00%> (-62.62%) ⬇️
...les/terrain_map_ext/test_terrain_map_ext_rosbag.py 7.69% <100.00%> (-76.89%) ⬇️
dimos/navigation/cmu_nav/tests/deprecation.py 100.00% <100.00%> (ø)

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aclauer aclauer changed the title Skip cmu nav tests ci: skip cmu nav tests Aug 19, 2026
@aclauer
aclauer marked this pull request as ready for review August 19, 2026 23:49
@aclauer
aclauer marked this pull request as draft August 19, 2026 23:49
@aclauer
aclauer marked this pull request as ready for review August 19, 2026 23:57
@greptile-apps

greptile-apps Bot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This change disables CMU navigation tests by default and adds a shared skip marker to the affected test modules. The downstream test job no longer fetches CMU navigation native modules when the switch is off. However, the separate native-build job still runs its Cachix-authenticated build on trusted cache misses even when CMU navigation tests are disabled, so the intended build avoidance is incomplete.

Confidence Score: 4/5

Not safe to merge as-is because disabling the test suite does not disable the corresponding Cachix-authenticated native build.

A deterministic workflow-condition check reproduced one independent functional issue: the native-build job remains scheduled regardless of the CMU navigation test switch.

Files Needing Attention: .github/workflows/ci.yml needs the CMU navigation switch applied to the cmu-nav-natives job condition while preserving its trusted-event protection.

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex produced a finding-proof for the posted P1 finding by running the deterministic CMU-nav workflow-condition simulator and attaching the simulator source and related logs.
  • T-Rex validated the finding by running two simulations: one with tests disabled and one with tests enabled, to compare outcomes.
  • T-Rex performed a syntax check of the CMU-nav simulator and captured the syntax-check log.
  • T-Rex conducted contract-validation by executing the condition simulation script with false and with true, confirming exit codes of 0 and that the job guard outcome is unchanged by the flag; a live GitHub Actions run was not launched due to secrets and artifact concerns.

View all artifacts

T-Rex Ran code and verified through T-Rex

Comments Outside Diff (2)

  1. .github/workflows/ci.yml, line 552-555 (link)

    P1 CMU-nav native Cachix build remains enabled when tests are disabled

    CMU_NAV_TESTS defaults to "false", but the cmu-nav-natives job condition does not reference that flag. On trusted pushes, merge-group runs, and same-repository PRs with a marker-cache miss, CI therefore still authenticates to Cachix and runs bin/build-cmu-nav-natives. The new guard at line 749 only skips the later test-job fetch, so disabling the suite does not avoid the native build or its artifact publication path. Gate cmu-nav-natives itself on env.CMU_NAV_TESTS == 'true' while retaining the existing trusted-event condition.

    Artifacts

    Deterministic CMU-nav workflow-condition simulator source

    • The exact executable source reads the checked-out CI workflow and evaluates the native job scenario, showing the test flag is absent from the job guard.

    CMU-nav condition simulation with tests disabled

    • Executed with `CMU_NAV_TESTS=false`; the captured output shows the job is scheduled and both Cachix authentication and native build are unguarded.

    CMU-nav condition simulation comparison with tests enabled

    • Executed with `CMU_NAV_TESTS=true`; the identical outcome demonstrates that the native-job guard does not depend on the test flag.

    CMU-nav simulator syntax-check log

    • The exact simulator source was compiled by Python successfully, confirming the executed proof script is valid.

    View artifacts

    T-Rex Ran code and verified through T-Rex

  2. General comment

    P1 CMU-nav native Cachix build remains enabled when CMU_NAV_TESTS is false

    • Bug
      • The workflow sets CMU_NAV_TESTS to "false", but cmu-nav-natives has no condition involving that variable. On a trusted push, merge-group, or same-repository pull request where the marker cache misses, the job is scheduled. Its Cachix authentication and uv run bin/build-cmu-nav-natives steps have no step-level if, so they execute once the job runs.
    • Cause
      • The flag was applied only to the downstream test matrix fetch step (ci.yml:749) and was omitted from the cmu-nav-natives job condition (ci.yml:552-555) and its sensitive build/auth steps.
    • Fix
      • Gate cmu-nav-natives itself with env.CMU_NAV_TESTS == 'true' (while preserving the existing trusted-event protection), or equivalently guard every Cachix-authenticated native-build path so disabled tests cannot create or upload CMU-nav native Cachix artifacts.

    T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "Merge branch 'andrew/chore/disable-cmu-n..." | Re-trigger Greptile

@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Aug 20, 2026
@aclauer
aclauer marked this pull request as draft August 20, 2026 01:27
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant