Nunchaku for FLUX: Local Installation and Inference Guide

Nunchaku is an inference engine from the Nunchaku and SVDQuant team that runs image models using low-bit computation. You install its Python runtime, obtain compatible quantized weights, and use Python or its ComfyUI plugin. 1, 2

What are the key facts about Nunchaku?

How does Nunchaku run quantized FLUX models?

Nunchaku implements SVDQuant, which uses a higher-precision low-rank component alongside quantized computation. Its purpose is to reduce the memory and computation needed to execute a supported model. 1

The practical benefit is a different way to run an existing FLUX checkpoint. You can keep an image-generation workflow while replacing the transformer implementation with Nunchaku's supported equivalent. 3

The basic tutorial uses Diffusers' FluxPipeline with NunchakuFluxTransformer2dModel. This makes the boundary clear: Diffusers assembles the pipeline, and Nunchaku supplies the quantized transformer. 3

There is also a dedicated ComfyUI plugin maintained by the project. It exposes Nunchaku through workflows, giving users a visual route to the same runtime instead of requiring a custom Python application. 4

For a first evaluation, choose a small set of prompts you already understand. Include one image with lettering, one with a recognizable object, and one with a busy composition. These are suggested checks, not a benchmark dataset.

Record what makes each output usable. A product silhouette, the position of a face, or the spelling on a sign is easier to assess than a general impression that one image looks better.

What are the hardware and model limits of Nunchaku?

The installation documentation lists specific supported NVIDIA GPU architectures. Treat that compatibility list as a requirement; the documented CUDA installation is not an AMD or Apple Silicon setup recipe. 2

A quantized file must match the implementation that loads it. The tutorial loads a named SVDQuant checkpoint and selects INT4 or FP4 precision according to the GPU. An arbitrary compressed FLUX file is not the documented input. 3

The engine's software license does not replace the underlying model license. Check the FLUX model card when choosing weights, especially if the eventual workflow involves operating a commercial service. 5

The FLUX.1 [dev] card also notes that generated images can miss prompt requirements and are not a source of factual information. Changing the inference engine does not remove those model limitations. 5

Avoid treating published speed measurements as a promise for your computer. For your own comparison, record GPU, precision, dimensions, sampling settings, and whether model loading is included in the timer.

A useful decision is whether the complete job fits your workflow. Include setup effort and failed runs in that decision instead of judging the installation from its quickest successful sample alone.

How do you install and use Nunchaku for FLUX?

1. Check your GPU against the installation guide. Select a supported PyTorch, CUDA, and Python combination before choosing the Nunchaku wheel. Wheel names encode compatibility information. 2

2. Install the wheel into the Python environment that will run inference. With portable ComfyUI, use its bundled Python; the project also documents an install_wheel.json workflow. 2

3. Obtain access to FLUX.1 [dev] on Hugging Face and review its conditions. The example pipeline loads components from that repository as well as the quantized transformer. 3, 5

4. Run the documented basic example before introducing extra nodes. The following adaptation uses the tutorial's default CUDA path; Turing GPUs need the separate configuration in the same guide. 3
   

import torch
from diffusers import FluxPipeline
from nunchaku import NunchakuFluxTransformer2dModel
from nunchaku.utils import get_precision

precision = get_precision()
transformer = NunchakuFluxTransformer2dModel.from_pretrained(
    f"nunchaku-tech/nunchaku-flux.1-dev/"
    f"svdq-{precision}_r32-flux.1-dev.safetensors"
)
pipe = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    transformer=transformer,
    torch_dtype=torch.bfloat16,
).to("cuda")
pipe("A blue ceramic teapot on a wooden shelf",
     num_inference_steps=50, guidance_scale=3.5).images[0].save("teapot.png")

1. For ComfyUI, install the official plugin and begin with its supplied workflows. Follow the workflow's model requirements instead of substituting a checkpoint based only on a similar filename. 4

Use the ComfyUI guide for workflow concepts. Keep the first graph small enough that you can explain every model loader and connection.

If a run fails, note which stage failed: importing the package, locating weights, loading the model, or sampling. That record gives you a useful starting point when checking the relevant installation or usage instructions.

Once the basic example works, add your usual components individually. Save a working copy before each addition so that a failure has a small, identifiable set of possible causes.

How does Nunchaku compare with other FLUX inference options?

Compare the first two using the same creative task. For the hosted alternative, compare the cost and usefulness of completed outputs rather than assuming that an API response time measures the same work as a local run.

The sibling FLUX.2 online access guide covers that hosted route. Choose it when you want to evaluate an image workflow before maintaining a local runtime.

Is Nunchaku a separate FLUX model?

No. Nunchaku is inference software; its FLUX examples run compatible quantized versions of Black Forest Labs models. The original checkpoint still determines the underlying model identity. 1, 3

Does installing the ComfyUI plugin install everything?

The Nunchaku runtime and ComfyUI plugin are separate installations. Install the runtime wheel into ComfyUI's Python environment using the project's instructions. 2, 4

Can Nunchaku run every FLUX checkpoint?

Nunchaku's FLUX tutorial requires a compatible SVDQuant transformer checkpoint. Check the project's supported formats before downloading weights for a new workflow. 3

Is there a fixed generation speed or VRAM minimum?

Nunchaku's published performance results describe particular models and hardware configurations. Record your GPU, precision, dimensions, and sampling settings when measuring your own setup. 1, 2

Sources

• Nunchaku engine, license, and project history
• Official installation guide
• Official basic FLUX usage tutorial
• Official ComfyUI plugin
• Black Forest Labs FLUX.1 dev model card
• BFL image-generation API guide

Related guides on PromptZone

• Realistic Photos with FLUX
• Best SDXL Models in 2026