[BUG]: The model muse-glimmer is not loading

[LyohaLyoha]

Description

Two weeks ago was released very efficient and interesting Meta model - muse glimmer
https://huggingface.co/meta-models/Muse-Glimmer-30B
https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF

Latest llama.cpp v0.3.0 release already supports it.

Please, consider upgrading to it - this model is actually kind of silent revolution in terms of VRAM efficiency across dense models.

Reproduction Steps

Try to load any gguf from https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF as an LLM model

Environment & Configuration

• Operating system: win
• .NET runtime version: 8
• LLamaSharp version: 0.29
• CUDA version (if you are using cuda backend):
• CPU & GPU device: core i5 10400, amd rx 9070, Vulkan

Known Workarounds

No response

[zsogitbe]

Thanks for reaching out and reporting this.

This is not a bug in LLamaSharp, but a matter of upstream synchronization. LLamaSharp wraps specific native llama.cpp releases. When upstream adds support for new model architectures (like Muse-Glimmer), those changes must be pulled into our native builds and bindings before the model can load. We'll track updating the upstream dependency for an upcoming release.

Regarding the model itself, it's worth tempering expectations before treating it as a revolution:

• Standard Memory Size: At 30B dense parameters, an ~18 GB footprint at 4-bit quantization is completely standard for this size class. It doesn't actually offer new memory-saving magic compared to other 27B–32B models.
• DFlash Overhead: Much of the advertised speed relies on pairing it with a DFlash draft model. In practice, juggling multiple files (base model, drafter, multimodal projector) adds complexity and can easily lead to a slowdown instead of a speedup when the drafter guesses wrong. Standard speculative decoding is generally much more optimal, predictable, and straightforward to run.
• Built-in Retries vs. Code-Level Control: The model tries to automatically diagnose and retry failed tool or bash commands. In practice, this can lead to stubborn retry loops (and token/cost explosion). It's usually much safer and more reliable to handle errors and retries directly in your own application code (like in C#) rather than trusting the model to fix itself.
• Narrow Focus: Because it is heavily fine-tuned for tool calling and terminal scripts, it's quite specialized and won't perform well as a general-purpose model for ordinary chat or writing tasks.

[LyohaLyoha]

Oh, i see. Well i saw that recent report about qwen3.8 not loading due to exactly the same cause - was reported as a bug, without any notes about it being in wrong category - so just reproduced that issue report.

And yep, i reported exactly because llama.cpp already has official 0.3.0 release fully supporting Muse - so upstream is green i suppose - i already use Muse with 0.3.0 llama.cpp for two weeks without any issues.

As to model itself - it's memory footpring magic is KV cache/context, being ~1.7Gb for Muse, instead of ~5.9Gb for Gemma4 - for the same f16 128k context.

Also Muse is ~30% faster at empty context, and up to 500% faster than G4 when more than half context is filled. It's so fast it just doesn't need drafting (and corresponding resources - VRAM first of all) at all!
All the same applies when comparing against qwen3.6 27b

Also Muse is very close to Gemma4 in terms of coding, and somewhat worse but not much as compared to qwen3.6
Taking into account that it decodes literally times faster than Gemma4/qwen3.6 - and thus can just do a few additional code review passes "for free" - as an outcome - it's just best of all three for coding/agentic tasks in terms of efficiency, hands down.
And agentic/coding usage of local models is as popular as chat/writing, i suppose - at least openrouter reported recently that agentic traffic is several times higher than conversational one nowadays.

I used Gemma4/qwen3.6 since their releases.
And use Muse for two weeks. Both G4 and Q3.6 already departed from my SSD :)