Skip to content

cmd-import: pull disk images from OCI#4591

Open
jbtrystram wants to merge 2 commits into
coreos:mainfrom
jbtrystram:import_images
Open

cmd-import: pull disk images from OCI#4591
jbtrystram wants to merge 2 commits into
coreos:mainfrom
jbtrystram:import_images

Conversation

@jbtrystram

@jbtrystram jbtrystram commented Jul 1, 2026

Copy link
Copy Markdown
Member

In Konflux we are building the disk images using image-builder and storing them as intermediary artifacts in OCI.
Abusing the OCI manifest spec a little bit we describe the disk images types under platform.os with the artifact name.

This allows to create a big manifest list referencing the regular OCI container images from the base build alongside all the disk images. Here is an example:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:1e24720d2ba7aba2a1cb8d82eb40e4c3fdde109c40d9a0eb118889ce5bedf6e3",
      "size": 771,
      "platform": {
        "architecture": "amd64",
        "os": "qemu"
      },
      "artifactType": "application/vnd.diskimage.qcow2"
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:c7e32f00573f696796dec43df101a00db23e0b4702271fa6c7b8f00dbacca6ea",
      "size": 773,
      "platform": {
        "architecture": "amd64",
        "os": "metal"
      },
      "artifactType": "application/vnd.diskimage.raw.gzip"
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:2f568edad2c832b24a67e362c4056eec49a658228f3f00ac597c9ce206c80261",
      "size": 11457,
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    }
  ]
}

With this change, cosa import knows how to pull a disk image from such a manifest with the --download flag, passing the artifact name. e.g.
cosa import docker://quay.io/konflux-fedora/coreos-tenant/disk-images-rawhide:on-pr-2d54f6ac04172313b4a3eec102d55bbd9def3097 --download qemu will result in a build dir populated with the OCI artifact and the qemu artifact:

cosa list
45.20260701.91.1
   Timestamp: 2026-07-01T06:09:39Z (8:15:24 ago)
   Artifacts: ostree oci-manifest qemu
      Config: c632a672e1b70cc3de53843db2bee58a4edaa367

Assisted-by: Opencode.ai <Opus 4.6>

Summary by CodeRabbit

  • New Features
    • cosa import can now detect disk image artifacts inside OCI image indexes, report discovered disk images, and optionally download them into the imported build.
    • Added --download (comma-separated platforms or all) and --arch to control which disk images are fetched and added.
  • Bug Fixes
    • If a build already exists, providing --download now augments it with the requested disk images instead of blocking.
    • When downloading is requested, the command now fails early if the source isn’t an OCI index or if no matching disk images are found for the selected architecture.

@openshift-ci

openshift-ci Bot commented Jul 1, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support to cosa import for automatically discovering and downloading disk image artifacts from an OCI image index, introducing a new --download command-line option. The review feedback focuses on improving the robustness of the implementation, specifically by ensuring that the oras CLI tool is validated before use, handling cases where requested downloads are unavailable, and adding defensive checks to prevent crashes when parsing potentially malformed JSON payloads or missing manifest keys.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/cmd-import Outdated
Comment thread src/cmd-import
Comment thread src/cmd-import
Comment thread src/cmd-import Outdated
@jbtrystram

Copy link
Copy Markdown
Member Author

/retest

@jbtrystram

Copy link
Copy Markdown
Member Author

/gemini-review

@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Enterprise

Run ID: bcfe5bdf-265d-42dd-986b-1ff7e207bc8f

📥 Commits

Reviewing files that changed from the base of the PR and between cbcb7c7 and 9bb1b4f.

📒 Files selected for processing (2)
  • src/cmd-import
  • src/deps.txt

📝 Walkthrough

Walkthrough

cosa import discovers disk image artifacts in OCI indexes, optionally downloads selected platform images through oras, records their metadata, and includes them in new or existing builds.

Changes

Disk image discovery and download

Layer / File(s) Summary
CLI and OCI artifact discovery/download primitives
src/cmd-import
Adds --download and --arch, OCI index discovery, platform and extension mappings, registry helpers, blob downloading, integrity checks, and the generate_oci_archive signature update.
Import and existing-build augmentation
src/cmd-import
Validates download requests, downloads selected images for new builds, and adds missing images to existing build metadata.
Finalization and runtime dependency
src/cmd-import, src/deps.txt
Moves downloaded images into the build destination and adds python3-oras.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant cosa_import
  participant OCI_registry
  participant oras_client
  participant build_storage
  cosa_import->>OCI_registry: Probe srcimg for OCI index
  OCI_registry-->>cosa_import: Return index manifest
  cosa_import->>cosa_import: Discover and select disk image artifacts
  cosa_import->>oras_client: Fetch platform manifest and blob
  oras_client-->>cosa_import: Return downloaded disk image
  cosa_import->>build_storage: Update metadata and finalize files
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: cmd-import now pulls disk images from OCI sources.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@jbtrystram
jbtrystram marked this pull request as ready for review July 8, 2026 12:29

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/cmd-import`:
- Around line 236-258: Validate registry-controlled artifact fields before they
are used to build filenames or populate build_meta['images']: in the cmd-import
flow where platform = entry.get('platform', {}), oci_goarch_to_basearch(), and
artifact_type_to_extension() are used, reject empty or unsafe platform.os,
platform.architecture, and artifactType-derived suffixes before appending to
artifacts or downloading. Add explicit checks in the manifest-processing path to
fail fast on unsupported platform names and only allow values that are valid for
the build metadata schema, so downstream path/key construction never sees
untrusted components.
- Around line 68-75: The `--download` path in `cmd-import` compares `args.arch`
directly against `disk_image_artifacts` entries, but those entries are
normalized with `oci_goarch_to_basearch()`, so mismatched aliases like `arm64`
vs `aarch64` can fail to match. Update the `download_arch` selection and the
artifact filtering in the `--download` branch to apply the same architecture
normalization used for OCI index entries, likely by normalizing `args.arch`
before comparing in the list comprehension and error message.
- Around line 60-71: The current try_fetch_oci_index()/--download flow is
masking real registry fetch failures as “not an OCI image index.” Update the
src/cmd-import logic around try_fetch_oci_index, registry_base_from_ref, and the
--download check so only known non-registry transports are treated as a missing
index, while actual oras manifest fetch/auth/transport errors are allowed to
surface when args.download is set. Keep the early
disk_image_artifacts/index_manifest handling, but distinguish between “not
supported transport” and genuine fetch failure before printing the OCI index
error.
- Around line 201-207: The ref normalization in registry_base_from_ref() still
leaves tags on refs like registry/repo:tag@sha256:..., so update the stripping
logic to remove the digest first and then always strip any tag from the final
path segment. Keep the fix localized to registry_base_from_ref() and ensure the
returned base is tag-free before later ORAS fetches use it.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 680e01e9-32b8-4731-99f0-fab7de0db016

📥 Commits

Reviewing files that changed from the base of the PR and between a2b4dd6 and 437b7ab.

📒 Files selected for processing (2)
  • src/cmd-import
  • src/deps.txt

Comment thread src/cmd-import
Comment thread src/cmd-import
Comment thread src/cmd-import
Comment thread src/cmd-import
jbtrystram added a commit to jbtrystram/fedora-coreos-pipeline that referenced this pull request Jul 8, 2026
Support using coreos/coreos-assembler#4591 to
download the pre-built QEMU image from the konflux builds instead
of building it manually.
joelcapitao
joelcapitao previously approved these changes Jul 9, 2026

@joelcapitao joelcapitao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
I left an in-line comment that can be addressed in a follow-up patch
There is also a fixup typo in the commit message but I would not trigger pipelines again only for its removal (just in case there is another edit before the merge)

Comment thread src/deps.txt Outdated
jbtrystram added a commit to jbtrystram/fedora-coreos-pipeline that referenced this pull request Jul 15, 2026
Support using coreos/coreos-assembler#4591 to
download the pre-built QEMU image from the konflux builds instead
of building it manually.
In Konflux we are building the disk images using image-builder and
storing them as intermediary artifacts in OCI.
Abusing the OCI manifest spec a little bit we describe the disk images
types under `platform.os` with the artifact name.

This allows to create a big manifest list referencing the regular OCI
container images from the base build alongside all the disk images.
Here is an example:
      ```json
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:1e247...",
      "size": 771,
      "platform": {
        "architecture": "amd64",
        "os": "qemu"
      },
      "artifactType": "application/vnd.diskimage.qcow2"
    },
    ...
  ]
}
```

With this change, `cosa import` knows how to find a disk image from such
an index and download them the `--download` flag.
e.g.
`cosa import docker://quay.io/konflux-fedora/coreos-tenant/disk-images-rawhide:on-pr-2d54f6ac04172313b4a3eec102d55bbd9def3097  --download qemu`
will result in a build dir populated with the OCI artifact and the
qemu artifact:
```
cosa list
45.20260701.91.1
   Timestamp: 2026-07-01T06:09:39Z (8:15:24 ago)
   Artifacts: ostree oci-manifest qemu
      Config: c632a672e1b70cc3de53843db2bee58a4edaa367
```

Downloaded blobs are integrity-checked against the layer digest from
the manifest.

The --arch flag allows downloading disk images for a different
architecture than the host (defaults to host basearch).

Non-registry sources (oci-archive:, containers-storage:) are unaffected
and continue to use the standard skopeo-based flow.

Uses the oras Python bindings (python3-oras) for registry interactions
(manifest fetching and blob downloads) instead of shelling out to the
oras CLI tool.

Assisted-by: Opencode.ai <Opus 4.6>
Authored-by: Opencode.ai <claude-opus-4-6>
Add the ability to pull disk images to an already imported build.
When --download is used on a build that already exists, the new
augment_existing_build() path adds disk images without re-importing the
OCI container image. Platforms already present in the build are skipped.

The platform selection logic is also extracted into
select_platforms_to_download() for reuse by both code paths.

Authored-by: Opencode.ai <claude-opus-4-6>
@jbtrystram

Copy link
Copy Markdown
Member Author

@coderabbitai help

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

CodeRabbit commands

These commands are invoked using PR/Issue comments.

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai rate limit to show your current review rate limit status.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai evaluate custom pre-merge check --instructions <custom-checks-instructions> --name <custom-checks-title> [--mode <error|warning>] to validate the custom pre-merge checks instructions. Defaults to error when --mode is omitted.
  • @coderabbitai ignore pre-merge checks to override pre-merge checks and get an approval on PR.
  • @coderabbitai run pre-merge checks to run pre-merge checks on the pull request.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai emit path instructions to emit generated path-specific review instructions for this repository.
  • @coderabbitai resolve merge conflict to automatically resolve merge conflicts.
  • @coderabbitai autofix to automatically fix issues identified in unresolved review comments.
  • @coderabbitai fix-ci to automatically fix failing CI checks by creating a stacked PR with fixes.
  • @coderabbitai resolve to resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai generate configuration to create a PR that adds the current resolved configuration as .coderabbit.yaml (or show it if already present).
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit configuration file (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • You can also validate your configuration using the online YAML validator.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, support, documentation and community

  • Visit our status page to check the current availability of CodeRabbit.
  • Create a ticket on our support page for assistance with any issues or questions.
  • Visit our documentation site for detailed information on how to use CodeRabbit.
  • Join our Discord community to connect with other users and get help from the community.
  • Follow us on X/Twitter for updates and announcements.

@jbtrystram

Copy link
Copy Markdown
Member Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants