USHIFT-7342: Replace hardcoded sleep with state check before restart#6996
USHIFT-7342: Replace hardcoded sleep with state check before restart#6996pacevedom wants to merge 2 commits into
Conversation
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
|
/test ? |
|
Skipping CI for Draft Pull Request. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
Walkthrough
ChangesMicroShift Restart Reliability
Estimated code review effort: 3 (Moderate) | ~20 minutes 🚥 Pre-merge checks | ✅ 15✅ Passed checks (15 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
openshift-ci
Bot
added
the
approved
|
/test e2e-aws-tests |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/resources/microshift-process.resource`:
- Around line 83-89: The PID snapshot is taken too early in the restart flow, so
the existing MicroShift process and etcd PIDs can become stale before the
restart is triggered. Move the MicroShift Process ID and MicroShift Etcd Process
ID captures in this resource so they happen after `Wait Until Keyword Succeeds`
/ `MicroShift Service Is Not Transitioning` and immediately before `Systemctl
restart microshift.service`, then keep the `Wait Until ... Changes` checks
using those fresh values.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 9704d8bc-0c13-4dad-86cb-b127ce30b0db
📒 Files selected for processing (1)
test/resources/microshift-process.resource
|
/test e2e-aws-tests |
|
/test all |
pacevedom
changed the title
openshift-ci-robot
added
the
jira/valid-reference
|
@pacevedom: This pull request references USHIFT-7342 which is a valid jira issue. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
The Restart MicroShift keyword had an unconditional 10s sleep before every restart to avoid issuing restarts while the service was mid-transition. Replace it with an active check that polls systemd ActiveState for transitional states, passing instantly when the service is already stable and saving ~10s per restart. Also reset the systemd failure counter before restarting so that start-limit-hit from prior rapid failures (e.g. invalid config tests) does not block the restart.
openshift-ci
Bot
removed
the
do-not-merge/work-in-progress
|
/retest |
|
/lgtm |
|
/verified by CI |
openshift-ci-robot
added
the
verified
|
@pacevedom: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/hold |
openshift-ci
Bot
added
the
do-not-merge/hold
openshift-ci-robot
removed
the
verified
|
/hold cancel |
openshift-ci
Bot
removed
the
do-not-merge/hold
|
@pacevedom: This pull request references USHIFT-7342 which is a valid jira issue. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/hold |
openshift-ci
Bot
added
the
do-not-merge/hold
pacevedom
removed
the
do-not-merge/hold
|
/lgtm |
The transition check budget (10x 5s = 50s) was too short — the activating state can last up to TimeoutStartSec=4m (240s) during slow startups. Increase to 30x 10s (300s) to cover the full startup window, matching the poll interval used by every other wait in the file. Reduce Wait For MicroShift Service from 50x 10s (500s) to 30x 10s (300s). TimeoutStartSec=4m (240s) is the hard ceiling — systemd kills the process after that, so the extra 200s could never fire. All waits in the file now use a uniform 30x 10s (300s) budget. Document the full Restart MicroShift flow: the Restart=always auto-restart loop, why the transition check must precede reset-failed (to close the race where pending auto-restarts re-trigger start-limit-hit after we clear it), PID tracking as defense-in-depth, and the readiness verification sequence.
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ggiguash, Neilhamza, pacevedom, suleymanakbas91 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/retest |
1 similar comment
|
/retest |
|
@pacevedom: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
5 participants
The Restart MicroShift keyword had an unconditional 10s sleep before every restart to avoid issuing restarts while the service was mid-transition. Replace it with an active check that polls systemd SubState for transitional states, passing instantly when the service is already stable and saving ~10s per restart.
Summary by CodeRabbit
microshift.servicestate to fully settle, clearing any failed state, restarting, verifying both MicroShift and etcd PIDs changed, then refreshing kubeconfig and confirming readiness.Wait For MicroShift Serviceto use fewer retries and clarified that the timeout budget coversTimeoutStartSec=4m.microshift.serviceis not transitioning (not activating, deactivating, or reloading) before proceeding with the restart.