feat(drone): integrate PX4 autonomous navigation drone and SITL simulation - #3458
Open
weyne-Jiang wants to merge 13 commits into
Open
feat(drone): integrate PX4 autonomous navigation drone and SITL simulation#3458weyne-Jiang wants to merge 13 commits into
weyne-Jiang wants to merge 13 commits into
Conversation
Flight control: - call MAVSDK APIs directly and remove the runtime adapter layer - move module configuration and PX4-specific errors into flight_control.py - simplify FLU/FRD command conversion, telemetry state, lifecycle cleanup, and status RPC - make hold enter the PX4 Hold flight mode and annotate movement skills with CAP_MOVEMENT - keep handler binding local to FlightController so the Gazebo blueprint can omit odometry without changing core Module behavior External vision and MID360: - preserve the hardware-verified odom -> mid360_link -> base_link contract - derive the body pose with the calibrated inverse mount transform before MAVSDK vision forwarding - retain quaternion, timestamp, covariance, and frame validation including lever-arm covariance conversion - collapse the mount publisher to the single calibrated transform implementation GStreamer camera: - replace the multi-layer dynamic GStreamer abstraction with one GsTeeCamera module using Gst directly - accept trusted raw or H.264 input pipelines and always publish BGR over pSHM plus Annex-B H.264 over typed LCM - support NVV4L2 and X264 encoder settings, reject silently ignored H.264 options, and release the pipeline deterministically - retain focused behavior tests for pipeline construction and encoder configuration Blueprints and visualization: - consolidate PX4 basic and Gazebo Harmonic blueprints in px4_basic.py and remove obsolete debug blueprints - add the px4-agentic MCP blueprint with a PX4-specific safety prompt - inline PX4 camera and Rerun configuration, keep lidar visualization at 5 Hz, and isolate the static drone entity from TF paths - give Gazebo a video-only Rerun layout because it has no PointLIO mapping stack - regenerate the built-in blueprint and module registry Cleanup, review fixes, and docs: - remove obsolete config, frame, camera, runtime, GStreamer adapter, debug, and redundant test files - revert the reviewed core Module change to match main and keep PX4-specific behavior within the PX4 package - preserve the tested VisionPositionEstimate interface required by current PX4 firmware - relocate the DJI README, add complete English and Chinese PX4 guides, and document hardware, native builds, simulation, video, frames, and full startup commands Validation: - uv run pytest dimos/robot/drone/px4 -q (18 passed) - CI=1 uv run pytest dimos/robot/test_all_blueprints_generation.py -q (1 passed) - uv run ruff check dimos/robot/drone/px4 - uv run mypy on the five changed PX4 source modules - git diff --check
weyne-Jiang
requested review from
Dreamsorcerer,
leshy,
mustafab0,
paul-nechifor and
spomichter
as code owners
August 13, 2026 10:15
github-actions
Bot
added
the
first-time-contributor
Codecov Report❌ Patch coverage is @@ Coverage Diff @@
## main #3458 +/- ##
==========================================
+ Coverage 74.05% 76.11% +2.06%
==========================================
Files 1283 1231 -52
Lines 124704 119675 -5029
Branches 11141 10723 -418
==========================================
- Hits 92349 91092 -1257
+ Misses 29493 25495 -3998
- Partials 2862 3088 +226
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 84 files with indirect coverage changes 🚀 New features to boost your workflow:
|
weyne-Jiang
changed the title
Remove PyPI PyGObject from the drone extra and lockfile so CI no longer builds PyGObject and pycairo before tests. Document the Ubuntu 24.04 python3-gi, GStreamer GIR/plugin packages, and system-site-packages uv environment required by GsTeeCamera. Pass reset_counter=0 to MAVSDK 3.17.2 VisionPositionEstimate after reproducing the required constructor argument in a fresh locked Ubuntu 24.04 environment.
Remove the forbidden PX4 package initializer and bare underscore assignments flagged by repository checks. Add behavior-focused MAVSDK flight-control tests for commands, frame conversion, telemetry, connection timeout, and shutdown. Expand GStreamer tee coverage for raw and H.264 publication, pipeline construction, and failure cleanup, and cover non-finite external-vision samples. Keep GStreamer config enums compatible with the project's Python 3.10 lower bound.
weyne-Jiang
changed the title
weyne-Jiang
changed the title
Reuse one real FlightController and GsTeeCamera instance per test module while resetting mocked MAVSDK and GStreamer state between cases. This preserves the default Module RPC lifecycle without repeatedly starting LCM for every test. Consolidate the essential external-vision contracts into the flight-control suite and remove redundant blueprint and static-transform test files. The focused suite now keeps behavior coverage with fewer module constructions and less CI lifecycle churn.
Make move commands return to zero velocity when callers stop refreshing the setpoint. Repeated move calls replace the previous watchdog task, while PX4 mode-changing actions remain able to interrupt Offboard control directly. Add deterministic lifecycle and timeout tests, and document the user-visible move behavior.
# Conflicts: # dimos/robot/all_blueprints.py
github-actions
Bot
added
the
ready-to-merge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contribution path
Demonstration
Problem
DimOS did not have a dedicated PX4 drone stack. The existing drone modules were primarily DJI-specific and did not provide MAVSDK-based flight control, PX4 SITL integration, external vision input, or the camera pipeline required by the PX4 hardware and Gazebo workflows.
Solution
Added a dedicated PX4 stack while moving the existing DJI implementation into its own
djipackage.The PX4 integration includes:
px4-basic,px4-agentic, andpx4-gazebo-harmonicblueprints.The Gazebo simulation scene is based on PX4-AeroFusion-Sim, with adaptations for the camera workflow used by the DimOS PX4 blueprint.
Hardware configuration
The PX4 integration was developed and validated on the following hardware:
How to Test
Follow the hardware and simulation procedures in the PX4 Module Quick Start.
Validated on the hardware configuration listed above:
dimos-viewerover the Wi-Fi LAN.Automated validation:
AI assistance
Codex with GPT-5.6 Sol was deeply involved in implementation, debugging, testing, and documentation. All changes were reviewed, understood, and validated by the contributor.
Checklist