Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .agents/skills/vllm-rlt-review-pr/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,9 +101,11 @@ changes need relevant regression checks, not a speedup.

Use targeted CPU tests or a small reproducer to resolve a concrete uncertainty.
Check test dependencies and skips; a green suite with GPU cases skipped does not
validate kernels or real serving. GPU validation requires the user's applicable
experiment scope and verified scheduler reservation; never use an unreserved
device or silently expand a benchmark's exhausted run budget. If unavailable,
validate kernels or real serving. Run GPU validation within the user's authorized
experiment scope and budget. A scheduler reservation is not a prerequisite; an
explicit GPU testing request authorizes selecting an available device without
another confirmation solely because a scheduler is unavailable. Do not silently
expand a benchmark's exhausted run budget. If GPU execution is unavailable,
finish the code/evidence review and identify the remaining validation precisely.

For each finding, give priority, a short defect title, the smallest relevant
Expand Down
2 changes: 1 addition & 1 deletion docs/async-speculative.md
Original file line number Diff line number Diff line change
Expand Up @@ -1196,7 +1196,7 @@ Token IDs require exact agreement; KV tolerances are FP32 `atol=rtol=4e-5` and
BF16 `atol=0.08, rtol=0.04`. These are targeted correctness checks, not general
model-quality evidence.

For a current rerun, use the commands above on a scheduler-reserved CUDA device.
For a current rerun, use the commands above on an explicitly selected CUDA device.
Compare native async, synchronous speculation and async speculation with the
same BF16 model, prompts, token budget, backend, capacity and GPU. Warm up each
mode, alternate measurement order and retain five unprofiled observations per
Expand Down
2 changes: 1 addition & 1 deletion docs/runtime-history.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ BF16 `atol=0.08, rtol=0.04`. These are targeted correctness checks, not general
model-quality evidence.

For a current rerun, use the [validation commands](async-speculative.md#validation)
on a scheduler-reserved CUDA device.
on an explicitly selected CUDA device.
Compare native async, synchronous speculation and async speculation with the
same BF16 model, prompts, token budget, backend, capacity and GPU. Warm up each
mode, alternate measurement order and retain five unprofiled observations per
Expand Down
6 changes: 3 additions & 3 deletions docs/serving.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ to `127.0.0.1:8000`; this benchmark frontend has no authentication or TLS.

Prepare the Ouro checkpoint and tokenizer separately. The official model ID
pins revision `574fa66cb8bf5abdc979642d01cf2b79b16bfab1`; local paths are supported.
Use a GPU scheduler reservation on shared hosts:
Select an available GPU explicitly; a scheduler is optional:

```bash
gpu status
gpu run --gpu-ids <available-id> --timeout 30m --note "Ouro HTTP serving" -- \
nvidia-smi
CUDA_VISIBLE_DEVICES=<available-id> \
python -m vllm_rlt.entrypoints.serve \
--model /path/to/prepared/ouro --tokenizer /path/to/prepared/ouro \
--device cuda --dtype bfloat16 --attention-backend triton \
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ license-files = ["LICENSE", "NOTICE"]
[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["."]
markers = ["gpu: requires a scheduler-reserved CUDA device"]
markers = ["gpu: requires an explicitly selected CUDA device and --run-gpu"]

[tool.ruff]
line-length = 100
Expand Down
6 changes: 3 additions & 3 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@

def pytest_addoption(parser):
parser.addoption(
"--run-gpu", action="store_true", help="Run CUDA tests inside a GPU reservation"
"--run-gpu", action="store_true", help="Run CUDA tests on explicitly selected devices"
)


def pytest_collection_modifyitems(config, items):
enabled = config.getoption("--run-gpu")
if enabled and not os.environ.get("CUDA_VISIBLE_DEVICES"):
raise pytest.UsageError("--run-gpu requires scheduler-provided CUDA_VISIBLE_DEVICES")
raise pytest.UsageError("--run-gpu requires explicit CUDA_VISIBLE_DEVICES")
for item in items:
if "gpu" in item.keywords and not enabled:
item.add_marker(pytest.mark.skip(reason="use --run-gpu inside a GPU reservation"))
item.add_marker(pytest.mark.skip(reason="set CUDA_VISIBLE_DEVICES and use --run-gpu"))


@pytest.fixture(autouse=True, scope="session")
Expand Down
Loading