Skip to content

feat: restrict guest capabilities during snapshot checkpoint - #1896

Open
andreiltd wants to merge 3 commits into
virtq-lazy-pool-generationsfrom
virtq-restricted-checkpoint
Open

andreiltd wants to merge 3 commits into
virtq-lazy-pool-generationsfrom
virtq-restricted-checkpoint

Conversation

@andreiltd

@andreiltd andreiltd commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

This patch introduces the execution mode on the host and allows call guest function and prohibit the guest to call host functions. The only user of this mode is snapshotting for now.

Closes: #1883

@andreiltd
andreiltd added this pull request to stack #1897 October 7, 2026 18:22
@andreiltd andreiltd added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Oct 7, 2026
@andreiltd
andreiltd marked this pull request as ready for review October 7, 2026 18:23
Copilot AI balanced review requested due to automatic review settings October 7, 2026 18:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

4 open findings
What changed in this PR

Introduces a host-side execution mode to restrict what guest operations are accepted during snapshot checkpointing, primarily to prevent guest→host function calls during the checkpoint window.

Changes:

  • Added ExecutionMode to SandboxMemoryManager and threaded it into outbound virtqueue handling.
  • Switched snapshot checkpoint dispatch to a restricted guest-execution path.
  • Added post-checkpoint validation of virtqueue/mailbox state and documented the behavior in the changelog.
File Description
src/​hyperlight_host/​src/​sandbox/​outb.rs Rejects guest host-function requests during restricted execution mode.
src/​hyperlight_host/​src/​sandbox/​initialized.rs Dispatches guest execution in restricted mode during snapshot checkpointing.
src/​hyperlight_host/​src/​mem/​mgr.rs Adds ExecutionMode state and validates virtqueues/mailbox after checkpoint.
CHANGELOG.md Documents restricted host-call behavior and added validation during checkpoints.

🧠 Review effort: Lite


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/hyperlight_host/src/mem/mgr.rs Outdated
Comment thread src/hyperlight_host/src/sandbox/initialized.rs Outdated
Comment thread src/hyperlight_host/src/sandbox/outb.rs
Comment thread src/hyperlight_host/src/sandbox/outb.rs
@hyperlight-gh-bot

This comment has been minimized.

@andreiltd
andreiltd force-pushed the virtq-restricted-checkpoint branch 2 times, most recently from 942a49b to 8c55f9d Compare October 7, 2026 19:37
@hyperlight-gh-bot

This comment has been minimized.

@ludfjig ludfjig left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Should we prevent any other outb as well you think or is this sufficient? (debug print, tracing)?

@ludfjig ludfjig left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think we need to reset the mode in restore(). If dispatch panics during the checkpoint, the reset line is skipped and the sandbox stays Restricted even after a restore. Maybe a test for this would be good as well (unless hard to repro).

@andreiltd
andreiltd force-pushed the virtq-restricted-checkpoint branch from 8c55f9d to be8b588 Compare October 8, 2026 08:39
@hyperlight-gh-bot

Copy link
Copy Markdown

Benchmark Results

Measured commit: be8b588844ab
Baseline commit: 9de91ff2c5f2

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 759.98 ns (➖ 1.08x faster)
vec_bytes 576.70 ns (➖ 1.03x faster)
373.69 µs (➖ 1.01x faster)

payload_allocation

slot_pool_segmented
262144 535.03 ns (➖ 1.02x slower)
65536 141.96 ns (➖ 1.02x faster)

sandboxes

create_initialized_and_drop
medium 81.52 ms (➖ 1.00x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
7.97 ns (➖ 1.03x slower) 7.72 ns (➖ 1.00x faster) 7.77 ns (➖ 1.00x slower)

snapshot_files

load_snapshot_unverified
small 92.33 µs (➖ 1.01x faster)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 7.39 µs (➖ 1.08x faster) 7.05 µs (➖ 1.06x faster)
65536 2.08 µs (➖ 1.07x slower) 1.94 µs (➖ 1.08x faster)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 6.28 µs (➖ 1.04x faster) 6.23 µs (➖ 1.04x faster)
8192 1.05 µs (➖ 1.00x slower) 1.10 µs (➖ 1.02x faster)
262144 27.24 µs (➖ 1.00x slower)
kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 762.43 ns (➖ 1.11x slower)
vec_bytes 545.98 ns (➖ 1.05x slower)
702.00 µs (➖ 1.03x slower)

payload_allocation

slot_pool_segmented
262144 512.82 ns (➖ 1.02x slower)
65536 144.82 ns (➖ 1.07x slower)

sandboxes

create_initialized_and_drop
medium 75.75 ms (➖ 1.00x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
7.06 ns (➖ 1.01x faster) 7.06 ns (➖ 1.01x slower) 7.06 ns (➖ 1.01x slower)

snapshot_files

load_snapshot_unverified
small 48.95 µs (➖ 1.02x slower)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 7.71 µs (➖ 1.03x slower) 7.69 µs (➖ 1.02x slower)
65536 2.16 µs (➖ 1.03x slower) 2.13 µs (➖ 1.02x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 7.39 µs (➖ 1.05x slower) 7.25 µs (➖ 1.02x slower)
8192 759.91 ns (➖ 1.07x faster) 753.61 ns (➖ 1.01x slower)
262144 30.03 µs (➖ 1.03x slower)
mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 940.72 ns (➖ 1.01x faster)
vec_bytes 713.25 ns (➖ 1.01x slower)
270.96 µs (➖ 1.01x slower)

payload_allocation

slot_pool_segmented
262144 726.35 ns (➖ 1.00x slower)
65536 197.14 ns (➖ 1.01x slower)

sandboxes

create_initialized_and_drop
medium 58.12 ms (➖ 1.06x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
9.74 ns (➖ 1.00x faster) 9.85 ns (➖ 1.01x slower) 9.68 ns (➖ 1.00x faster)

snapshot_files

load_snapshot_unverified
small 83.51 µs (➖ 1.01x faster)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 10.28 µs (➖ 1.00x faster) 9.58 µs (➖ 1.01x slower)
65536 2.42 µs (➖ 1.03x faster) 2.24 µs (➖ 1.00x faster)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 8.08 µs (➖ 1.17x faster) 8.98 µs (➖ 1.07x faster)
8192 1.36 µs (➖ 1.04x slower) 1.29 µs (➖ 1.03x slower)
262144 38.43 µs (➖ 1.02x slower)
mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 931.79 ns (➖ 1.01x slower)
vec_bytes 672.95 ns (➖ 1.02x slower)
718.11 µs (➖ 1.06x slower)

payload_allocation

slot_pool_segmented
262144 620.79 ns (➖ 1.00x faster)
65536 163.47 ns (➖ 1.01x faster)

sandboxes

create_initialized_and_drop
medium 69.12 ms (➖ 1.03x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
9.66 ns (➖ 1.00x slower) 8.78 ns (➖ 1.00x faster) 9.32 ns (➖ 1.06x slower)

snapshot_files

load_snapshot_unverified
small 46.34 µs (➖ 1.02x slower)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 8.00 µs (➖ 1.00x faster) 8.03 µs (➖ 1.00x slower)
65536 2.32 µs (➖ 1.01x slower) 2.30 µs (➖ 1.00x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 7.65 µs (➖ 1.02x slower) 7.50 µs (➖ 1.02x faster)
8192 941.98 ns (➖ 1.00x slower) 947.09 ns (➖ 1.05x slower)
262144 38.48 µs (➖ 1.04x faster)
hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 1.17 µs (➖ 1.02x slower)
vec_bytes 781.44 ns (➖ 1.01x faster)
2.01 ms (➖ 1.07x faster)

payload_allocation

slot_pool_segmented
262144 784.84 ns (➖ 1.02x faster)
65536 229.75 ns (➖ 1.01x faster)

sandboxes

create_initialized_and_drop
medium 88.99 ms (➖ 1.07x slower)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
10.16 ns (➖ 1.01x slower) 10.23 ns (➖ 1.01x slower) 10.11 ns (➖ 1.00x slower)

snapshot_files

load_snapshot_unverified
small 690.79 µs (➖ 1.36x faster)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 9.53 µs (➖ 1.04x slower) 9.44 µs (➖ 1.03x slower)
65536 2.42 µs (➖ 1.03x slower) 2.39 µs (➖ 1.02x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 8.80 µs (➖ 1.01x faster) 9.01 µs (➖ 1.01x faster)
8192 1.44 µs (➖ 1.00x slower) 1.27 µs (➖ 1.01x faster)
262144 40.08 µs (➖ 1.01x faster)
hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 1.24 µs (➖ 1.02x slower)
vec_bytes 821.64 ns (➖ 1.07x slower)
3.03 ms (➖ 1.01x faster)

payload_allocation

slot_pool_segmented
262144 734.23 ns (➖ 1.01x faster)
65536 214.73 ns (➖ 1.01x slower)

sandboxes

create_initialized_and_drop
medium 109.60 ms (➖ 1.05x slower)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
10.45 ns (➖ 1.02x slower) 10.05 ns (➖ 1.01x slower) 10.76 ns (➖ 1.02x slower)

snapshot_files

load_snapshot_unverified
small 641.01 µs (➖ 1.15x slower)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 7.82 µs (➖ 1.01x slower) 8.20 µs (➖ 1.06x slower)
65536 2.34 µs (➖ 1.01x slower) 2.36 µs (➖ 1.01x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 8.47 µs (➖ 1.02x faster) 8.61 µs (➖ 1.06x faster)
8192 1.41 µs (➖ 1.06x faster) 1.12 µs (➖ 1.00x slower)
262144 41.02 µs (➖ 1.02x slower)

Reported by cargo ci bench-report --candidate run:37751315163 --config-file bench_report.toml.

@andreiltd
andreiltd force-pushed the virtq-restricted-checkpoint branch from be8b588 to 925741c Compare October 8, 2026 09:34
syntactically
syntactically previously approved these changes Oct 8, 2026

@syntactically syntactically left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

self.g2h_consumer = None;
self.h2g_consumer = None;

// A panic during restricted dispatch can skip the mode reset.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm probably missing something obvious, but why can this happen? I see that dispatch_guest_call_restricted will still do the reset even if dispatch_guest_call returns Err(...), and I thought a guest call would show up there as Err(HyperlightError::GuestAborted(...)).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is referring to panic in host. But yeah as I don't think we have any promises if host panics, this might be overly cautions.

Keep transport pools and aliases stable. Pin pages backing live buffers
and exclude overlapping slots until a checkpoint finds those pages
empty.

Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
This patch introduces the execution mode on the host and allows call
guest function and prohibit the guest to call host functions. The only
user of this mode is snapshotting for now.

Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/enhancement For PRs adding features, improving functionality, docs, tests, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Restrict guest execution capabilities during snapshot

4 participants