ruvllm 2.1.0: quantize reads 0 tensors from a HuggingFace directory or .safetensors and writes a 75-byte GGUF

[ruvnet]

What happens

ruvllm quantize --model <hf-dir-or-model.safetensors> --quant q4_k_m --output out.gguf (ruvllm 2.1.0, Linux x86_64) prints Input size: 0.00 MB, 0 tensors, 0 elements, and writes a 75-byte file, for both a merged HuggingFace directory (config.json + model.safetensors + tokenizer.json) and the bare .safetensors path. The same weights load fine in candle and convert to a valid GGUF with candle's writer, so the input is well-formed.

Where it was hit

Mission ruOS SLiM (cognitum-one/ruos-desktop PR #83, ADR-061): a candle LoRA fine-tune of Qwen2.5-0.5B-Instruct merged to HF safetensors, then exported. The --help text advertises "Quantize a model to GGUF format … Q4_K_M, Q5_K_M, Q8_0" and gives ruvllm quantize --model ./model.safetensors --quant q8_0 as an example, so callers reasonably expect the safetensors path to work.

Expected

Either a working safetensors/HF-dir loader for quantize, or a loud error ("safetensors input not supported yet") instead of a 75-byte success.

Workaround

candle's native GGUF writer (llama.cpp Q4_K_M mixed layout for hidden sizes that are not multiples of 256, e.g. 896 on Qwen2.5-0.5B).

🤖 Generated with claude-flow

https://claude.ai/code/session_01CmwEhDLBt5MWxBBBTzvQqA

[ruvnet]

Dream evidence review (current default head edaffffb3b85768eb1f3ec1f683b7f46f0506af4):

The reported failure is confirmed against reachable current source. The advertised SafeTensors quantization path advances progress without loading tensor payloads, writes a GGUF header declaring zero tensors, and the GGUF-to-GGUF path currently emits a placeholder payload. This contradicts the CLI's documented conversion behavior. No production deployment or exploitability is inferred.

Frozen acceptance gate:

1. A real, pinned SafeTensors fixture must either produce a loadable GGUF with a non-zero tensor table and preserved metadata, or fail closed before any output artifact is created.
2. A GGUF input must round-trip through the selected quantizer with non-empty tensor data and a successful independent loader check.
3. Unsupported architectures, malformed metadata, truncated data, and interrupted writes must leave no apparently valid output.
4. Tests must bind input/output hashes, tensor counts, shapes, dtypes, quantization type, loader result, peak memory, wall time, and exact candidate commit.
5. Native workspace, supply-chain, regression, and WASM compatibility gates must pass on that exact head.

Until a validated candidate satisfies those checks, the hypothesis that the current CLI performs real quantization is REJECT. A safe patch was not attempted because this cycle had no runnable checkout or representative model fixture; unvalidated replacement of the artifact writer would be higher risk than keeping this issue explicit.

[ruvnet]

Partial fix in #1014 (ruvllm-cli): ruvllm quantize now fails closed. It validates its arguments, then exits with an error pointing at llama.cpp / candle before creating any output file, for SafeTensors, PyTorch, HF-directory and GGUF inputs. The --help text no longer advertises a working conversion. Tests confirm that no output artifact is created.

This covers the 'loud error' branch of gate 1 and gate 3. Gates 2, 4 and 5 need a real GGUF tensor writer, which does not exist yet, so this stays open. ruvllm-cli uses the workspace version (2.3.0 on crates.io), so this change ships with the next workspace release.

[ruvnet]

ruvllm-cli 2.3.1 is published to crates.io with the fail-closed quantize. The issue stays open until a real GGUF tensor writer exists.