Add Nunchaku Lite single-file quantization by rootonchair · Pull Request #14100 · huggingface/diffusers

rootonchair · 2026-07-01T17:08:21Z

What does this PR do?

Adds Nunchaku Lite single-file checkpoint loading for Diffusers models.

This introduces NunchakuLiteQuantizationConfig and a new Nunchaku Lite quantizer that can patch supported nn.Linear modules into runtime SVDQ/AWQ linear layers before strict checkpoint loading. The loader reads safetensors metadata during from_single_file so Nunchaku Lite checkpoints can use their embedded runtime manifest to decide which modules to replace.

Deprecated API

import torch
from diffusers import (
    ErnieImagePipeline,
    ErnieImageTransformer2DModel,
    NunchakuLiteQuantizationConfig,
)

checkpoint = hf_hub_download(
    repo_id="rootonchair/ERNIE-Image-Turbo-nunchaku-lite",
    filename="svdq-int4_r32-ernie-image-turbo-zero-svdq-fix-bias.safetensors",
)

transformer = ErnieImageTransformer2DModel.from_single_file(
    checkpoint,
    config="baidu/ERNIE-Image-Turbo",
    subfolder="transformer",
    quantization_config=NunchakuLiteQuantizationConfig(compute_dtype=torch.bfloat16),
    torch_dtype=torch.bfloat16,
)

pipe = ErnieImagePipeline.from_pretrained(
    "baidu/ERNIE-Image-Turbo",
    transformer=transformer,
    torch_dtype=torch.bfloat16,
)

pipe.to("cuda")

image = pipe(
    prompt="A modern red armchair in a quiet studio, soft window light, realistic product photography",
    height=512,
    width=512,
    num_inference_steps=8,
    guidance_scale=1.0,
    generator=torch.Generator(device="cuda").manual_seed(1234),
    use_pe=False,
).images[0]

image.save("ernie-image-turbo-nunchaku-lite.png")

New API for `from_single_file` use

import torch
from huggingface_hub import hf_hub_download

from diffusers import (
    ErnieImagePipeline,
    ErnieImageTransformer2DModel,
    NunchakuLiteQuantizationConfig,
)


dtype = torch.bfloat16
device = "cuda"

checkpoint = hf_hub_download(
    repo_id="rootonchair/ERNIE-Image-Turbo-nunchaku-lite",
    filename="svdq-int4_r32-ernie-image-turbo-zero-svdq-fix-bias.safetensors",
)

svdq_targets = []
for name in [
    "self_attention.to_q",
    "self_attention.to_k",
    "self_attention.to_v",
    "self_attention.to_out.0",
    "mlp.gate_proj",
    "mlp.up_proj",
    "mlp.linear_fc2",
]:
    svdq_targets.extend([f"layers.{i}.{name}" for i in range(36)])

quantization_config = NunchakuLiteQuantizationConfig(
    compute_dtype=dtype,
    svdq_w4a4={
        "precision": "int4",
        "group_size": 64,
        "rank": 32,
        "targets": svdq_targets,
    },
    awq_w4a16={
        "precision": "int4",
        "group_size": 64,
        "targets": [
            "text_proj",
            "time_embedding.linear_1",
            "time_embedding.linear_2",
            "adaLN_modulation.1",
            "final_norm.linear",
            "final_linear",
        ],
    },
)

transformer = ErnieImageTransformer2DModel.from_single_file(
    checkpoint,
    config="baidu/ERNIE-Image-Turbo",
    subfolder="transformer",
    quantization_config=quantization_config,
    torch_dtype=dtype,
)

pipe = ErnieImagePipeline.from_pretrained(
    "baidu/ERNIE-Image-Turbo",
    transformer=transformer,
    torch_dtype=dtype,
)

pipe.to(device)

image = pipe(
    prompt="A modern red armchair in a quiet studio, soft window light, realistic product photography",
    height=512,
    width=512,
    num_inference_steps=8,
    guidance_scale=1.0,
    generator=torch.Generator(device=device).manual_seed(1234),
    use_pe=False,
).images[0]

image.save("ernie-image-turbo-nunchaku-lite.png")

Fixes # (issue)

Before submitting

Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
- Did you read the Coding with AI agents guide?
- Did you self-review the diff against .ai/review-rules.md?
Did you read the contributor guideline?
Did you read our philosophy doc? (important for complex PRs)
Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?
Are you the author (or part of the team) of the model/pipeline (only applicable for model/pipeline related PRs)?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

sayakpaul

Thanks for getting started! Just did a first pass and left high-level reviews.

sayakpaul · 2026-07-02T04:16:04Z

@@ -0,0 +1,161 @@
+import json


For tests, WDYT of adding a mixin to https://github.com/huggingface/diffusers/blob/main/tests/models/testing_utils/quantization.py and then extending a popular model like Flux to use that mixin?

Yes, let's do it that way

We can remove this test then?

rootonchair · 2026-07-02T07:04:43Z

I just did some benchmark on RTX PRO 6000, here is the visual result between bf16 and nvfp4 checkpoint for ERNIE-Image-Turbo

BF16	Nunchaku NVFP4

Case	Full mean	Denoise mean	Denoise peak alloc	Full peak alloc	Speedup
original	3.003s	2.862s	29.429GB	31.081GB	1.0x
nunchaku_lite NVFP4	2.271s	2.127s	18.926GB	20.578GB	1.35x
nunchaku_lite NVFP4 + compile	1.675s	1.525s	18.672GB	20.578GB	1.8x
nunchaku_lite NVFP4 + bnb text encoder	2.285s	2.132s	14.317GB	15.969GB	1.35x

By replacing Nunchaku Linear, we have reduced the latency of these linear operations by 2x with large shape

Target	Op	Rows	Shape	Normal ms	Nunchaku ms	Speedup
`layers.0.self_attention.to_q`	`svdq_w4a4`	4096	4096 -> 4096	0.3660	0.1563	2.34x
`layers.0.mlp.gate_proj`	`svdq_w4a4`	4096	4096 -> 12288	1.0646	0.4272	2.49x
`layers.0.mlp.linear_fc2`	`svdq_w4a4`	4096	12288 -> 4096	1.0269	0.4596	2.23x

One note here, AWQ only benefit with adaLN layer, so other modules like time embedding or final linear can stay as bf16

Target	Op	Rows	Shape	Normal ms	Nunchaku ms	Speedup
`text_proj`	`awq_w4a16`	1	3072 -> 4096	0.0111	0.0201	0.55x
`time_embedding.linear_1`	`awq_w4a16`	1	4096 -> 4096	0.0129	0.0251	0.52x
`time_embedding.linear_2`	`awq_w4a16`	1	4096 -> 4096	0.0129	0.0247	0.52x
`adaLN_modulation.1`	`awq_w4a16`	1	4096 -> 24576	0.1389	0.0243	5.71x
`final_norm.linear`	`awq_w4a16`	1	4096 -> 8192	0.0220	0.0245	0.90x
`final_linear`	`awq_w4a16`	4114	4096 -> 128	0.0248	0.0635	0.39x

rootonchair · 2026-07-02T10:20:55Z

I have just implemented the native loading feature, which now can load by from_pretrained with converted repo:

import torch
from diffusers import ErnieImagePipeline

pipe = ErnieImagePipeline.from_pretrained(
    "rootonchair/ERNIE-Image-Turbo-nunchaku-lite-int4",
    torch_dtype=torch.bfloat16,
).to("cuda")

image = pipe(
    prompt="A modern red armchair in a quiet studio, soft window light, realistic product photography",
    height=1024,
    width=1024,
    num_inference_steps=8,
    guidance_scale=1.0,
    use_pe=False,
).images[0]

image.save("ernie-image-turbo-nunchaku-lite-int4.png")

Quantization config now change to:

"quantization_config": {
    "awq_w4a16": {
      "group_size": 64,
      "precision": "int4",
      "targets": [
        "text_proj",
        "time_embedding.linear_1",
        "time_embedding.linear_2",
        "adaLN_modulation.1",
        "final_norm.linear",
        "final_linear"
      ]
    },
    "compute_dtype": "bfloat16",
    "quant_method": "nunchaku_lite",
    "svdq_w4a4": {
      "group_size": 16,
      "precision": "fp4",
      "rank": 32,
      "targets": [
        "layers.0.self_attention.to_q",
        "layers.1.self_attention.to_q",
        "layers.2.self_attention.to_q",
        "layers.3.self_attention.to_q",
         ...
      ]
    }

If we agree to use this schema, I will remove the old metadata/from_single_file approach

sayakpaul

Looking good. I think we can remove all metadata related code?

sayakpaul

No rush but let us know once you would like another round of review.

HuggingFaceDocBuilderDev · 2026-07-03T02:24:22Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

rootonchair · 2026-07-03T14:53:23Z

@sayakpaul I think this is ready for the next review

sergereview

🤗 Serge says:

This adds a new nunchaku_lite quantization backend (config, quantizer, runtime linear layers backed by the HF kernels package, docs, and tests). The overall structure follows existing quantizer integrations, but there are a few blocking issues.

Security

_get_ops() in src/diffusers/quantizers/nunchaku/utils.py calls get_kernel(..., trust_remote_code=True) against a personal user repo (rootonchair/nunchaku-lite-kernels). This silently enables remote code execution for anyone who loads a Nunchaku Lite checkpoint, with no user opt-in. No other kernel usage in the repo does this (gguf and attention_dispatch call get_kernel without trust_remote_code). This needs to be removed or made an explicit user decision, and the kernel repo should ideally live under an org namespace with pinned revisions.

Description vs. diff mismatch

The PR description claims "The loader reads safetensors metadata during from_single_file so Nunchaku Lite checkpoints can use their embedded runtime manifest" — but there are no changes to single_file_model.py or any loader in this diff. Correspondingly, the metadata parameter of NunchakuLiteQuantizer._process_model_before_weight_loading is never populated by any call site and is dead code.

Correctness

NunchakuLiteTesterMixin._test_quantized_layers is tautological: it sets expected_quantized_layers = num_quantized_layers and then asserts they're equal, so the count check can never fail. It should compare against the number of targets in the quantization config (like the base mixin compares against linear-layer count).
The docs and the NunchakuLiteQuantizationConfig docstring repeatedly reference model.json, but Diffusers model configs (and the test in this PR) use config.json. As written, users following the doc will put the quantization config in a file Diffusers never reads.

Style

nunchaku_quantizer.py has trailing whitespace (line 64) — make style was apparently not run.
New source files are missing the Apache license header used across src/diffusers.
NunchakuLiteQuantizationConfig is appended to _import_structure via a standalone statement instead of being listed in the dict literal like every other unconditional export.
import itertools is buried inside check_strict_state_dict_match; move it to module top level.
SVDQW4A4Linear re-validates precision/group_size that NunchakuLiteQuantizationConfig.post_init already enforces — per repo guidelines, drop the duplicated defensive checks (the forward path hard-codes group sizes 16/64 anyway, so the group_size argument is effectively unused at runtime).

serge v0.1.0 · model: claude-fable-5 · 10 LLM turns · 22 tool calls · 390.5s · 437374 in / 30106 out tokens

sergereview · 2026-07-03T16:08:46Z

+    if _ops is None:
+        from kernels import get_kernel
+
+        _ops = get_kernel(_HF_KERNEL_REPO, version=_HF_KERNEL_VERSION, trust_remote_code=True).ops


Security: trust_remote_code=True silently enables execution of arbitrary remote code from a personal user repo whenever a Nunchaku Lite checkpoint is loaded — the user never opts in. Neither of the existing get_kernel call sites in this repo (quantizers/gguf/utils.py, models/attention_dispatch.py) passes trust_remote_code. Please drop it (publish the kernel as a standard prebuilt kernels repo that doesn't require remote code), or at minimum surface this as an explicit user-facing opt-in. Hosting under a personal namespace (rootonchair/...) rather than an org also makes this a supply-chain risk for everyone using the backend.

sergereview · 2026-07-03T16:08:46Z

+        self,
+        model: "ModelMixin",
+        state_dict: dict[str, Any] | None = None,
+        metadata: dict[str, str] | None = None,


metadata is never passed by any caller — neither modeling_utils.py nor single_file_model.py forwards safetensors metadata to preprocess_model. The PR description claims the loader "reads safetensors metadata during from_single_file", but no loader changes exist in this diff. Per the repo guidelines (no unused parameters "for API consistency"), remove this parameter.

sergereview · 2026-07-03T16:08:46Z

+                self._verify_if_layer_quantized(name, module, config_kwargs)
+                num_quantized_layers += 1
+
+        expected_quantized_layers = num_quantized_layers


This assertion is tautological: expected_quantized_layers is set to num_quantized_layers, so the num_quantized_layers == expected_quantized_layers check below can never fail, and num_fp32_modules is always 0. The only effective check left is num_quantized_layers > 0. Compute the expected count from the config instead, e.g. the total number of entries in svdq_w4a4["targets"] + awq_w4a16["targets"], so the test actually verifies all targets were replaced.

sergereview · 2026-07-03T16:08:46Z

+    The exported state dict must match the target Diffusers model architecture exactly. Checkpoints quantized with
+    fused QKV projections won't load into a model config that expects separate Q, K, and V projection modules.
+
+    Example compact `model.json` config:


Same as the doc page: Diffusers reads the model config from config.json, not model.json. Please fix the filename here and in the "quantization_config stored in model.json" sentence above so users don't package their checkpoints incorrectly.

sergereview · 2026-07-03T16:08:46Z

    ],
 }

+_import_structure["quantizers.quantization_config"].append("NunchakuLiteQuantizationConfig")


Since this export is unconditional, add "NunchakuLiteQuantizationConfig" directly to the "quantizers.quantization_config" list in the _import_structure dict literal above instead of appending via a standalone statement — that matches how every other unconditional export is declared. Note also that NunchakuLiteQuantizationConfig.__init__ references torch unconditionally, while all other quantization configs exported here are gated behind is_torch_available(); consider whether this one needs the same guard.

sergereview · 2026-07-03T16:08:46Z

+        if device is None:
+            device = torch.device("cpu")
+
+        if precision not in {"int4", "nvfp4"}:


These precision/group_size checks duplicate validation that NunchakuLiteQuantizationConfig.post_init already enforces (and post_init is stricter: it pins group_size to 16 for fp4 / 64 for int4, while this accepts any positive value that forward then ignores — the activation-scale layout hard-codes 16/64). Per the repo's no-defensive-code guideline, drop the re-validation here and rely on the config.

Co-authored-by: sergereview[bot] <283583894+sergereview[bot]@users.noreply.github.com>

sayakpaul

Nice, thanks a lot for the further updates. I will post about the benchmarking I conducted through HF Jobs and Claude Code below.

sayakpaul · 2026-07-09T03:23:38Z

+Compile the quantized transformer after loading the pipeline for faster inference.
+
+```python
+pipe.transformer = torch.compile(pipe.transformer, mode="default", fullgraph=False)


Why can't we do it fullgraph=True? Can you open an issue on the nunchaku-lite repo with the trace?

with the second kernel version, this has been resolved

sayakpaul · 2026-07-09T03:26:02Z

+    use_pe=False,
+).images[0]
+image.save("ernie-image-turbo-nunchaku-lite.png")
+```


Is it compatible with offloading, too?

yes it is compatible with enable_sequential_cpu_offload and enable_model_cpu_offload

sayakpaul · 2026-07-09T03:27:18Z

+| `svdq_w4a4` | `fp4` | 16 | Uses NVFP4 runtime kernels with SVDQ low-rank correction. |
+| `svdq_w4a4` | `int4` | 64 | Uses INT4 W4A4 kernels with SVDQ low-rank correction. |
+| `awq_w4a16` | `int4` | 64 | Uses INT4 weight-only AWQ-style kernels. |
+


I think it was not addressed? I think should mention a matrix of the quant schemes and their supported hardware / CUDA capabilities. Also, the note on hopper since we cannot support it.

sayakpaul · 2026-07-09T03:47:47Z

+
+        self.post_init()
+
+    def post_init(self):


Very nice validation!

sayakpaul · 2026-07-09T04:52:13Z

@@ -0,0 +1,161 @@
+import json


We can remove this test then?

sayakpaul · 2026-07-09T04:54:44Z

Even though the below is generated with Claude Code (I have manually verified them).

Nunchaku Lite (PR #14100) — testing notes

I ran the snippet from docs/source/en/quantization/nunchaku.md end-to-end. TL;DR: the compiled NVFP4 benchmark in the doc reproduces cleanly on the same class of hardware, but I hit two rough edges (hardware/torch-version compatibility) that are worth documenting or guarding.

Reproduced the doc benchmark ✅

Ran the exact doc snippet (rootonchair/ERNIE-Image-Turbo-nunchaku-lite-nvfp4, 1024×1024, 8 steps, use_pe=False) on an RTX PRO 6000 Blackwell (via HF Jobs, --flavor rtx-pro-6000) — the same GPU the doc benchmarks on.

Variant	Latency (mean of 3)
No compile	2.279 s
`torch.compile(mode="default", fullgraph=False)`	1.643 s
Speedup (compile vs eager)	1.39×

First compiled call (compilation): ~51 s
Peak memory: 22.1 GB

These match the doc's reported 2.271 s → 1.675 s almost exactly, so the NVFP4 + torch.compile claim holds up. 👍

Rough edge 1: capability check is more permissive than the shipped kernels

NunchakuLiteQuantizer.validate_environment allows INT4 on any GPU with capability >= (7, 5) (Turing+). But the kernel build actually fetched for INT4 (rootonchair/nunchaku-lite-kernels, torch212-cxx11-cu126) only contains SM images for sm_75 / 80 / 86 / 89. Verified with cuobjdump -lelf on the downloaded .so.

Consequence: on an H100 (sm_90) the INT4 path passes validate_environment and then dies at inference with:

CUDA error: no kernel image is available for execution on the device

which is a confusing failure mode (looks like a runtime bug, not an unsupported-hardware message). So Hopper is effectively unsupported for INT4 today even though the check green-lights it.

Suggestions (pick whatever fits):

Add sm_90 to the INT4 kernel build, or
Tighten/clarify the check so unsupported archs fail early with a clear message (e.g. "no INT4 kernel image for sm_90"), or
Note the actually-supported archs in the docs.

Rough edge 2: kernels only ship for torch 2.11 / 2.12

kernels.get_kernel matches builds by torch major.minor. The repo currently only has builds for torch 2.11 and 2.12 (cu126/cu128/cu130/cu132). Installing any other torch (I first tried 2.8) fails hard:

FileNotFoundError: Cannot find a build variant for this system ...
  torch211-cxx11-cu128-x86_64-linux: Torch version (2.11) does not match environment Torch version (2.8)
  ...

For Blackwell specifically, the working combo is torch 2.11 + cu128 (torch211-cxx11-cu128).

Suggestion: document the supported torch versions in nunchaku.md (and ideally surface a friendlier error / requirement up front), since this isn't obvious and the failure only appears at load time.

Environment

HF Jobs rtx-pro-6000 → NVIDIA RTX PRO 6000 Blackwell, capability (12, 0)
torch 2.11.0+cu128, diffusers @ PR Add Nunchaku Lite single-file quantization #14100 head (452de403), kernels 0.16.x

sayakpaul · 2026-07-09T05:00:23Z

Now running the tests.

sayakpaul · 2026-07-09T05:16:06Z

Testing report (again manually verified).

Nunchaku Lite (PR #14100) — test run on HF Jobs

Ran the PR's test suite tests/quantization/nunchaku/test_nunchaku.py on Hugging Face Jobs.

Setup

Hardware: HF Jobs rtx-pro-6000 — RTX PRO 6000 Blackwell, capability (12, 0)
Stack: torch 2.11.0+cu128, kernels 0.16.0, diffusers @ PR Add Nunchaku Lite single-file quantization #14100 head
Env: RUN_NIGHTLY=1, DIFFUSERS_TEST_DEVICE=cuda (so nothing is skipped, including the @nightly CUDA kernel test)

Result

9 passed in 4.66s ✅ (pytest exit code 0), including NunchakuLiteCudaKernelsTests::test_awq_cuda_kernels, which executes the real AWQ W4A16 kernel on GPU.

NunchakuLiteBasicTests::test_compact_config_round_trips_dtype_and_targets           PASSED
NunchakuLiteBasicTests::test_environment_requires_cuda                              PASSED
NunchakuLiteBasicTests::test_int4_environment_allows_turing_cuda                    PASSED
NunchakuLiteBasicTests::test_int4_environment_requires_turing_cuda                  PASSED
NunchakuLiteBasicTests::test_nvfp4_environment_allows_blackwell_cuda                PASSED
NunchakuLiteBasicTests::test_nvfp4_environment_requires_blackwell_cuda              PASSED
NunchakuLiteBasicTests::test_compact_config_replaces_svdq_and_awq_without_state_dict PASSED
NunchakuLiteBasicTests::test_nunchaku_lite_loads_with_from_pretrained               PASSED
NunchakuLiteCudaKernelsTests::test_awq_cuda_kernels                                 PASSED

One caveat (not a PR bug)

The 3 tests that download the kernel initially failed due to an external dependency bug: kernels passes an empty user_agent to huggingface_hub, producing a User-Agent header with a trailing space:

httpx.LocalProtocolError: Illegal header value b'kernels/0.16.0; hf_hub/1.22.0; python/3.12.12; '

huggingface_hub 1.x uses httpx, which rejects trailing whitespace. It only triggers on a cold first get_kernel call. Worked around by stripping the UA (a small sitecustomize.py patching huggingface_hub.utils._headers). Pinning huggingface_hub<1.0 does not help — it drags kernels down to 0.12.3, which lacks the trust_remote_code= argument the PR uses.

Suggestion: report to kernels upstream, and consider bumping the PR's require_kernels_version_greater_or_equal("0.9.0") to a version that actually provides trust_remote_code.

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

…chair/diffusers into feature/nunchaku-lite-single-file

sayakpaul

Thanks a lot for this!

sayakpaul · 2026-07-13T18:09:47Z

@bot /style

github-actions · 2026-07-13T18:11:11Z

Style bot fixed some files and pushed the changes.

stevhliu

thanks for the docs!

stevhliu · 2026-07-13T19:02:44Z

 Initialize [`~quantizers.PipelineQuantizationConfig`] with the following parameters.

- `quant_backend` specifies which quantization backend to use. Currently supported backends include: `bitsandbytes_4bit`, `bitsandbytes_8bit`, `gguf`, `quanto`, and `torchao`.
+- `quant_backend` specifies which quantization backend to use. Currently supported backends include: `bitsandbytes_4bit`, `bitsandbytes_8bit`, `gguf`, `nunchaku_lite`, `quanto`, and `torchao`.


seems like nunchaku_lite is more of a prequantized loader versus a backend that quantizes pretrained/non-quantized checkpoints on the fly which is what this section mainly demonstrates. it would be better to add a new "Loading prequantized checkpoints" section with this there

stevhliu · 2026-07-13T19:08:54Z

+```bash
+pip install -U kernels
+```


move the installation before the first code snippet which needs it

stevhliu · 2026-07-13T19:09:58Z

+pip install -U kernels
+```
+
+> [!NOTE]


would move this to the loading section where architecture matching is more relevant versus under CUDA kernels

sayakpaul · 2026-07-14T02:35:27Z

@rootonchair there are some CI failures. Would it be possible for you to fix them? The PR is basically ready to be merged.

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> Co-authored-by: Vinh H. Pham <phamvinh257@gmail.com>

rootonchair · 2026-07-14T03:56:37Z

@sayakpaul @stevhliu I just updated the docs and the unit test, could you check again

Add Nunchaku Lite single-file quantization

7f4a3a0

github-actions Bot added size/L PR with diff > 200 LOC quantization tests single-file and removed size/L PR with diff > 200 LOC labels Jul 1, 2026

rootonchair marked this pull request as draft July 1, 2026 17:13

sayakpaul reviewed Jul 2, 2026

View reviewed changes

Support config-backed Nunchaku Lite loading

1a66ac9

github-actions Bot added the size/L PR with diff > 200 LOC label Jul 2, 2026

sayakpaul reviewed Jul 2, 2026

View reviewed changes

Comment thread src/diffusers/quantizers/nunchaku/nunchaku_quantizer.py

Comment thread src/diffusers/quantizers/quantization_config.py

Comment thread src/diffusers/quantizers/quantization_config.py

sayakpaul reviewed Jul 2, 2026

View reviewed changes

Comment thread src/diffusers/quantizers/quantization_config.py

rootonchair added 3 commits July 2, 2026 15:53

Remove Nunchaku runtime manifest metadata loading

36b4fc4

Simplify Nunchaku compact config loading

db05e0b

Add Nunchaku Lite quantization tests

5d4822a

sayakpaul reviewed Jul 3, 2026

View reviewed changes

Document Nunchaku Lite checkpoint loading

c630a88

github-actions Bot added documentation Improvements or additions to documentation and removed single-file labels Jul 3, 2026

Refine Nunchaku Lite quantization docs

ab246af

sayakpaul requested review from SunMarc and asomoza July 3, 2026 15:03

sergereview Bot requested changes Jul 3, 2026

View reviewed changes

rootonchair and others added 3 commits July 3, 2026 16:11

Remove unused Nunchaku smooth factor original weights

57323cd

Update docs/source/en/quantization/nunchaku.md

5fef94f

Co-authored-by: sergereview[bot] <283583894+sergereview[bot]@users.noreply.github.com>

Update src/diffusers/quantizers/nunchaku/utils.py

fb33430

Co-authored-by: sergereview[bot] <283583894+sergereview[bot]@users.noreply.github.com>

sayakpaul reviewed Jul 9, 2026

View reviewed changes

Merge branch 'main' into feature/nunchaku-lite-single-file

b849b18

Apply suggestions from code review

9284dd0

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

rootonchair force-pushed the feature/nunchaku-lite-single-file branch from 03399a0 to 5522954 Compare July 9, 2026 16:15

rootonchair added 3 commits July 9, 2026 16:17

docs: update nunchaku quantization guide

4806585

test: remove nunchaku quantization tests

d687751

fix: reject hopper for nunchaku lite

86cd7c8

rootonchair force-pushed the feature/nunchaku-lite-single-file branch from 5522954 to 86cd7c8 Compare July 9, 2026 16:17

rootonchair commented Jul 9, 2026

View reviewed changes

Comment thread docs/source/en/quantization/overview.md Outdated

rootonchair and others added 4 commits July 9, 2026 23:20

Apply suggestion from @rootonchair

fd58893

docs: update nunchaku lite compile example

f6897af

Merge branch 'feature/nunchaku-lite-single-file' of github.com:rooton…

3df700a

…chair/diffusers into feature/nunchaku-lite-single-file

Merge branch 'main' into feature/nunchaku-lite-single-file

0d77278

sayakpaul requested a review from stevhliu July 13, 2026 18:08

sayakpaul approved these changes Jul 13, 2026

View reviewed changes

Apply style fixes

b23fd3c

stevhliu approved these changes Jul 13, 2026

View reviewed changes

Merge branch 'main' into feature/nunchaku-lite-single-file

10236fe

rootonchair and others added 3 commits July 14, 2026 10:01

Apply suggestions from code review

5fdc026

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> Co-authored-by: Vinh H. Pham <phamvinh257@gmail.com>

update doc to be more reasonable

0d63eea

fix import that requires kernels

d03f80a

sayakpaul approved these changes Jul 14, 2026

View reviewed changes

sayakpaul merged commit 60ec6f7 into huggingface:main Jul 14, 2026
14 of 15 checks passed

Uh oh!

Conversation

rootonchair commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Deprecated API

New API for from_single_file use

Before submitting

Who can review?

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rootonchair commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rootonchair commented Jul 2, 2026

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

HuggingFaceDocBuilderDev commented Jul 3, 2026

Uh oh!

rootonchair commented Jul 3, 2026

Uh oh!

sergereview Bot left a comment

Choose a reason for hiding this comment

Security

Description vs. diff mismatch

Correctness

Style

Uh oh!

sergereview Bot Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

sergereview Bot Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sergereview Bot Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sergereview Bot Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

sergereview Bot Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sergereview Bot Jul 3, 2026

Choose a reason for hiding this comment

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rootonchair commented Jul 1, 2026 •

edited

Loading

New API for `from_single_file` use

rootonchair commented Jul 2, 2026 •

edited

Loading

github-actions Bot commented Jul 13, 2026 •

edited

Loading