Skip to content

Use the automations MCP tool names in the Release Autopilot skills - #246

Merged
sarahlessner merged 1 commit into
mainfrom
devin/1793906562-automations-tool-names
Nov 5, 2026
Merged

sarahlessner merged 1 commit into
mainfrom
devin/1793906562-automations-tool-names

Conversation

@sarahlessner

@sarahlessner sarahlessner commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • The Release Autopilot skills and evals switch to the hosted MCP tool names that match the Automations settings UI: get-automations, update-automations, delete-automations-repo-settings (were get-release-autopilot, update-release-autopilot, delete-release-autopilot-repo-settings). Arguments are unchanged.
  • Both SKILL.md prerequisites now say: on a server that still lists the old names, use those with the same arguments. If a call says the tool was removed, ask the user to reconnect the MCP server and retry. That makes this safe to merge before or after the launchdarkly/mcp rename.
  • Skill names and folders (launchdarkly-release-autopilot-*) are unchanged.
  • Version bumps: settings 1.4.1-experimental, onboarding 0.3.1-experimental, Claude Code plugin 1.0.6, Cursor plugin 1.0.7. skills.json regenerated.

Testing

  • Not applicable
  • Manual (describe below)

python3 scripts/validate_skills.py, python3 -m unittest discover -s tests, python3 scripts/generate_catalog.py --check, and cd evals && npm test (153 pass). Eval tool definitions, mocks, assertions, and promptfoo mock keys were updated. No paid promptfoo runs.

Notes

Link to Devin session: https://app.devin.ai/sessions/44a8cd323e044679b96d97df7aa5796c
Open in Devin Desktop: https://app.devin.ai/desktop/session/44a8cd323e044679b96d97df7aa5796c?variant=devin
Requested by: @sarahlessner


Note

Overview
Release Autopilot onboarding and settings skills now call get-automations, update-automations, and delete-automations-repo-settings instead of the *-release-autopilot-* MCP names, matching the Automations settings UI and the upcoming launchdarkly/mcp rename. Tool arguments are unchanged.

Both skills document backward compatibility: use the old names if the server still exposes them; if a tool was removed, prompt the user to reconnect the MCP server and retry. Skill folder names stay launchdarkly-release-autopilot-*.

The release-autopilot-onboarding eval suite is updated end-to-end—tool definitions, mocks, assertions, promptfoo mock_overrides, and the stateful mock in _mock.js—plus related unit tests. Plugin versions bump to Claude 1.0.6 and Cursor 1.0.7; skills bump to settings 1.4.1 and onboarding 0.3.1 (experimental).

Reviewed by Cursor Bugbot for commit 0e0d909. Bugbot is set up for automated code reviews on this repo. Configure here.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Skill eval results

Skill Before After Δ
agentcontrol/configs-create 100/100 (4/4) 75/100 (3/4) -25
agentcontrol/configs-update 80/100 (4/5) 80/100 (4/5) no change
agentcontrol/configs-variations 80/100 (4/5) 80/100 (4/5) no change
agentcontrol/tools 75/100 (3/4) 75/100 (3/4) no change
aws-devops-agent/experiment-orchestration - 100/100 (9/9) new
experiments/launchdarkly-experiment-guide - 80/100 (4/5) new
experiments/launchdarkly-experiment-hypothesis-builder - 100/100 (13/13) new
experiments/launchdarkly-experiment-setup - 100/100 (8/8) new
feature-flags/flag-and-release-change - 100/100 (4/4) new
feature-flags/flag-release - 100/100 (5/5) new
feature-flags/launchdarkly-flag-command - 100/100 (3/3) new
feature-flags/launchdarkly-flag-create 100/100 (3/3) 100/100 (4/4) no change
feature-flags/launchdarkly-flag-drift - 100/100 (4/4) new
feature-flags/should-flag-change - 100/100 (17/17) new
onboarding - 100/100 (7/7) new
release-autopilot/launchdarkly-release-autopilot-onboarding - 96/100 (24/25) new

Only suites whose source actually changed since their last recorded score were re-run. Soft-failing while we stabilise the baseline.

@fsaresh fsaresh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

@sarahlessner
sarahlessner merged commit 0bf9be1 into main Nov 5, 2026
24 checks passed
@sarahlessner
sarahlessner deleted the devin/1793906562-automations-tool-names branch November 5, 2026 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants