Repository navigation
Move the Linux CUDA wheel workflow to OSDC runners - #1849
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/meta-pytorch/torchcodec/1849
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 378abdf with merge base 0fc8b84 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
bf4e523 to
0964461
Compare
0964461 to
e9903e8
Compare
| if: github.repository == 'meta-pytorch/torchcodec' && github.event_name == 'pull_request' && | ||
| github.event.pull_request.head.repo.fork == false |
There was a problem hiding this comment.
This is a regression, we rely on this doc PR preview job often and we always submit fork PRs, since it's the normal GitHub workflow.
I could work around this by submitting non-fork PRs, although I'd prefer not to, but external contributors are unable to do so anyway.
Is there a planned fix for this?
There was a problem hiding this comment.
Agree. There is an extra step we need to do to maintain the existing behaviour, but it's doable. Basically, the doc build needs to be trigger via ciflow so that it runs in the context of meta-pytorch/torchcodec instead of the fork. Then the job will have the permission to upload the preview to S3. Let me update the PR for this
There was a problem hiding this comment.
Updated in f491103. Fork PRs can get a preview again: a maintainer adds the ciflow/docs label, and pytorch-bot pushes a ciflow/docs/<PR> tag. That run happens in meta-pytorch/torchcodec, so it can assume the role and upload to the same meta-pytorch/torchcodec/<PR> preview path. It builds the wheel and docs and skips the GPU tests. Same-repo PRs still upload directly. The bot reads .github/pytorch-probot.yml from main, so the label only works after this PR lands, and the ciflow/docs label needs to be created.
There was a problem hiding this comment.
I decide to move doc build logic to a new workflow to make it easier to trigger it instead of keeping it in the CUDA workflow.
f491103 to
443907a
Compare
|
@NicolasHug The new doc build and upload workflow test run https://github.com/meta-pytorch/torchcodec/actions/runs/44192457051/job/134445928509?pr=1849 |
|
Thanks for handling the PR preview. I'd like to go back to the f491103 approach. Docs need to be built on every PR, not only when the label is there: building them is our only check that docs and tutorials don't break, and we want to catch that before the merge. Concretely:
|
|
Let me try to explain a bit better on why we need a work around here to address your points. The key change is that OSDC runners as k8s pods don't have any credential to write anything. Previously, bare metal EC2 runners have a blanket default permission to write anything anywhere they like on the artifact bucket, which is very bad for security reason. Because OSDC runners don't have any credentials now, they need to authenticate to get one before they can write. This applies to all cases, fork, non-fork, main, release. Now, the GitHub OIDC authentication works roughly like this:
Another thing is about GitHub workflow triggers, you can have multiple triggers in the workflow, i.e Now, let me go over your points one by one:
In conclusion, I think we need to move
Note that this new approach has a drawback that |
Keep build-docs and upload-docs in linux_cuda_wheel so docs are built on every PR. Preview uploads need OIDC, which fork pull_request runs can't get, so a workflow_run workflow uploads the Built-Docs artifact instead. This replaces the ciflow/docs workflow and labeler.
bf7b977 to
87f6dd2
Compare
5e23704 to
2149bb6
Compare
Summary: Move the doc preview upload off EC2 (`linux.2xlarge`) so it works for both fork and same-repo PRs. - OSDC pods have no AWS credentials, so the S3 upload needs `role/arc` through GitHub OIDC. A fork's `pull_request` run can't get an OIDC token. - `doc-preview` moves from `docs.yml` to a new `docs_preview.yml`, triggered by `workflow_run` when Generate documentation completes successfully. It runs from `main` in this repo, so it can authenticate for any PR. It finds the PR from the run's head, downloads `Built-Docs` from that run and uploads it to `doc-previews/meta-pytorch/torchrec/<PR>` on `mt-l-x86iavx512-8-64`. - `build_docs_job` and the `gh-pages` deploy are unchanged. - `workflow_run` only uses the copy of the workflow on `main`, so this starts working after it lands. The same workflow is in meta-pytorch/torchcodec#1849, where it was tested on a fork and a same-repo PR. Pull Request resolved: #5092 Reviewed By: aporialiao Differential Revision: D133333925 Pulled By: huydhn fbshipit-source-id: 32f256c265ea8ddcce63baa40123c31d0d07b775
NicolasHug
left a comment
There was a problem hiding this comment.
Thanks for the extra work making sure we preserve the preview! IIUC we can only try it out on the next PR, so I'll merge this and trigger a test PR. Thanks!
Move
linux_cuda_wheel.yamlfrom EC2 to OSDC runners.runner-fleet: osdc. PRs run onmt-l-x86aavx2-11-41-a10g. Release refs (main, nightly,release/*,v*tags) use the same regular runner: meta-pytorch can't use the release runners (Keep non-pytorch orgs on regular OSDC runners on release refs pytorch/test-infra#9485).linux.g5.4xlarge.nvidia.gpu→mt-l-x86aavx2-11-41-a10g. Docs are still built on every PR from the wheel this workflow builds.upload-docs(mainand release tags) is unchanged: it pushes togh-pageswithGITHUB_TOKENand needs no AWS credentials.pull_requestrun can't get an OIDC token, sodoc-previewmoves to a newdocs_preview.yaml. It's triggered byworkflow_runwhen Linux CUDA x86 completes, and runs frommainin this repo. It finds the PR from the run's head, downloadsBuilt-Docsfrom that run, and uploads it withrole/arconmt-l-x86iavx512-8-64. This covers fork and same-repo PRs, and needs no label.workflow_runonly uses the copy of the workflow onmain, so it starts working after this lands.