The Muse Spark Open NVFP4 checkpoint is 127 safetensors shards totalling 533,011,023,472 bytes (533.0 GB, 496.4 GiB), and the BF16 checkpoint is 65 shards totalling 1,644.3 GB, so the model card's 700 GB and 2 TB figures are VRAM floors with headroom, not file sizes. Only the routed experts in layers 1 to 60 are FP4; attention, layers 0 and 61, the expert projections, lm_head and the vision and audio towers stay BF16. From config.json, KV cache costs 64 KiB per token on the 16 full-attention layers plus a fixed 368 MiB per sequence on the 46 sliding-window layers, so one 1M-token sequence needs 64.4 GiB at BF16 or 32.2 GiB at FP8. At vLLM 0.33.0, B200 and B300 run NVFP4 natively, H200 runs it through Marlin W4A16, and RTX PRO 6000 is not accepted by the Muse NVFP4 MoE path. An 8x B200 node with FP8 KV holds a ceiling of 176 concurrent 128K sequences. Access to Meta's repos is manually approved and the Muse Model License requires a Built with Muse Spark Open display, so request access and send LICENSE.md to counsel before you order GPUs.
Muse Spark Open VRAM requirements come down to two numbers the model card does not print: the NVFP4 checkpoint is 533,011,023,472 bytes, and the KV cache costs 64 KiB per token. Meta's card says "the model requires a GPU with at least 700GB of VRAM for NVFP4 and 2TB for BF16." Those are floors with headroom baked in. The files themselves, summed from the Hugging Face file listing, are 533.0 GB of NVFP4 across 127 shards and 1,644.3 GB of BF16 across 65 shards. Calculator pages print estimates built from parameter counts, not from the shard bytes or config.json.
Everything above the weights is KV cache, and this model has an unusual KV profile. Muse Spark Open has 62 layers: 46 use a 2,048-token sliding window and 16 use full attention with grouped-query attention. The full layers cost 64 KiB per token at BF16. The sliding layers cost a fixed 368 MiB per sequence no matter how long the context runs. Those two figures, multiplied by your real concurrency and context length, decide which node you buy.
The third input is not technical. Both Meta repos are gated with manual approval, and the Muse Model License carries a mandatory "Built with Muse Spark Open" display, an outputs-based definition of Derivatives and a Usage Policy that names ITAR work and critical-infrastructure operation. For a bank, a hospital group or a defence supplier, those are procurement steps that come before the GPU order, and the last section treats them that way. The model was released on October 15, 2026; everything below is pinned to vLLM 0.33.0, released November 2, 2026.
Muse Spark Open VRAM requirements in one table
Start from the bytes. Every row below is the sum of the safetensors (or GGUF) file sizes in the repository's file API, not a parameter count multiplied by a bit width.
Reconcile the card with the files and the floors make sense. 700 GB minus 533.0 GB leaves 167 GB for KV cache, activations and CUDA graphs. 2 TB minus 1,644.3 GB leaves about 356 GB. The BF16 checkpoint is 1.64 TB on disk; the "2TB" in the card is VRAM, not file size. Meta's own developer post for the release rounds the NVFP4 build to "about 530 GB" and says it "serves on four Blackwell Ultra GPUs or eight H200s."
If you are searching for a quantized build, neither the ggml-org nor the unsloth GGUF repository has a Q4_K_M file; the Q4-class files are Q4_K_S at 470.1 GB and UD-Q4_K_XL at 505.5 GB. And the mlx 4bit build fits the two-Mac-Studio setup Meta's card describes as optimized for MLX; our DGX Spark vs Mac Studio comparison covers whether that class of hardware belongs in a serving plan at all. For the same bytes-first method applied to another large MoE, see how to run GLM-5.2 locally. The rest of this post sizes the Meta NVFP4 checkpoint on NVIDIA datacenter GPUs, because that is the build an 8-GPU NVIDIA node in a regulated data centre runs.
| Checkpoint | Files | Bytes on disk | Target hardware |
|---|---|---|---|
| Meta NVFP4 (Muse-Spark-Open-816B-A42B-NVFP4) | 127 shards | 533.0 GB (496.4 GiB) | NVIDIA B200 / B300 native, H200 via Marlin |
| Meta BF16 (Muse-Spark-Open-816B-A42B) | 65 shards | 1,644.3 GB (1,531.4 GiB) | 8x B300 is the only node below that holds it |
| AMD Quark MXFP4 | 65 shards | 508.9 GB | AMD MI350 / MI355 per its card |
| AMD Quark PTPC FP8 | 832.7 GB | AMD, via the Quark W8A8 FP8 path | |
| mlx-community 4bit | 100 shards | 532.8 GB | two 512 GB M5 Ultra Mac Studios |
| ggml-org GGUF Q2_K / Q4_K_S / Q8_0 / BF16 | split files | 304.1 / 470.1 / 871.6 / 1,640.3 GB | llama.cpp, plus a 4.10 GB BF16 or 2.18 GB Q8_0 mmproj file |
| unsloth GGUF UD-IQ1_S / UD-Q2_K_XL / UD-Q4_K_XL | split files | 175.6 / 311.2 / 505.5 GB | llama.cpp |
What NVFP4 actually quantizes (and what stays BF16)
The NVFP4 build is 3.1x smaller than BF16, not 4x, because most of the non-expert model is left alone. hf_quant_config.json declares quant_method modelopt, quant_algo NVFP4, group_size 16, kv_cache_quant_algo null, and exclude_modules of layers.0., layers.61., self_attn and lm_head. config.json goes further: its modules_to_not_convert list has 762 entries, which add the pre_expert_proj and post_expert_proj matrices in all 62 layers, the full vision tower (50 layers), the audio tower and the vision adapter.
The net effect is that only the routed experts in layers 1 to 60 are FP4. The shard headers decompose the 533.0 GB like this:
Attention alone is 13.52B parameters across 62 layers, all BF16: q and o projections at 8,192 x 8,192, k and v at 1,024 x 8,192, plus an output gate_proj at 8,192 x 8,192, which comes to 218.1M parameters per layer. There are no shared experts (num_shared_experts is 0), no MTP layers and no multi-head latent attention.
That split changes the per-token picture more than the file size suggests. At batch 1, the active weights read per decode step work out, from the shapes above, to about 54.4 GB: 13.6 GB of active FP4 experts, 27.0 GB of attention, 8.3 GB of expert projections, 3.3 GB of lm_head, 1.6 GB of BF16 experts in layers 0 and 61 and 0.5 GB of routers. Only a quarter of the bytes read per token are FP4. This is arithmetic from tensor shapes, not a throughput figure, and it is the reason not to assume that a 4-bit checkpoint decodes four times faster than BF16. The card's own limitation note applies as well: "Quantized inference may show minor quality differences in edge cases compared to full-precision."
| Component | Count | Storage | Bytes |
|---|---|---|---|
| Routed experts, layers 1 to 60 | 773.1B weights (60 x 256 experts) | 4-bit packed + one FP8 E4M3 scale per 16 weights = 4.5 bits | 434.9 GB |
| Everything else (attention, layers 0 and 61, expert projections, lm_head, vision, audio) | 48.8B params | BF16 | 97.6 GB |
| Routers and scales | 130.0M params | F32 | 0.5 GB |
| Total | 533.0 GB |
Muse Spark Open KV cache size per token, from config.json
config.json (text_config) gives everything the KV formula needs: num_hidden_layers 62, num_key_value_heads 16 against 128 query heads, head_dim 64, sliding_window 2048, max_position_embeddings 1,048,576, and a layer_types list of 46 sliding_attention and 16 full_attention entries. The full-attention layers sit at indices 1, 5, 9 and so on up to 61, every fourth layer. layer_rope_theta is 500000.0 on the sliding layers and 0 on the full layers, so the global layers carry no rotary embedding. Since hf_quant_config.json sets no KV quantization, vLLM's default cache dtype of auto stores KV in BF16.
# From config.json (text_config) and the HF file listing full-attention layers = 16 (layer_types == full_attention) sliding layers = 46 (sliding_window = 2048) num_key_value_heads = 16 head_dim = 64 KV per token, full layers = 16 x 2 x 16 x 64 x 2 B = 65,536 B = 64 KiB (BF16) KV per sequence, sliding = 46 x 2 x 16 x 64 x 2 B x 2,048 = 368 MiB (BF16, fixed) 1M-token sequence = 64 GiB + 368 MiB (BF16) = 32 GiB + 184 MiB (FP8) 128K sequence = 8 GiB + 368 MiB (BF16) = 4 GiB + 184 MiB (FP8) NVFP4 weights = 127 shards = 533,011,023,472 B = 533.0 GB BF16 weights = 65 shards = 1,644,333,393,864 B = 1,644.3 GB
Per sequence, that gives the following, before allocator block rounding and before any draft model's own cache:
The sliding-window cap is real in vLLM, not only on paper. At 0.33.0 the Muse decoder passes per_layer_sliding_window to the attention layers marked sliding_attention, and the hybrid KV manager's SlidingWindowManager frees blocks that fall outside the 2,048-token window. Prefix caching on those layers behaves differently from full attention, which our post on prompt re-processing in sliding-window hybrid models explains; plan for it if your workload reuses long shared prefixes.
Why this is the opposite of Kimi K3
Kimi K3 makes a useful contrast for a team choosing a large open-weight MoE, because the memory profiles point in opposite directions. Our Kimi K3 GPU sizing guide derives 27.0 KiB per token at BF16 from its latent attention, with 1,560.9 GB of weights and a fixed recurrent state per sequence. Muse Spark Open has about a third of those weight bytes but pays 64 KiB per token, 2.4x more per token of context. K3 is weight-bound and cheap at long context. Muse fits on fewer GPUs and gets expensive as context and concurrency grow, so the 1M context window is a capacity decision, not a default.
| Context per sequence | BF16 KV | FP8 KV |
|---|---|---|
| 32K tokens | 2.36 GiB | 1.18 GiB |
| 128K tokens | 8.36 GiB | 4.18 GiB |
| 1M tokens | 64.36 GiB | 32.18 GiB |
Which GPUs run Muse Spark Open: 8-GPU node fit by context
The table subtracts 533.0 GB of weights from aggregate memory at vLLM 0.33.0's default --gpu-memory-utilization of 0.92, then divides the remainder by the per-sequence KV cost above. Treat every sequence count as a ceiling: activations, CUDA graphs and the vision tower's runtime buffers come out of the same headroom, and they are not derivable from config.
The 1,440 GB B200 figure is from NVIDIA's DGX B200 user guide. The 288 GB per B300 is from NVIDIA's DGX B300 user guide (8 x 288 GB). Eight H100s at 80 GB would leave about 56 GB after weights, which is not a serving configuration. The BF16 checkpoint fits only the 8x B300 row, with about 475 GB left over.
vllm serve meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \ --tensor-parallel-size 8 \ --enable-expert-parallel \ --max-model-len 131072 \ --kv-cache-dtype fp8 \ --gpu-memory-utilization 0.92 \ --max-num-seqs 64 \ --reasoning-parser muse \ --tool-call-parser muse \ --enable-auto-tool-choice
Which GPU generation loads the NVFP4 path
The fit column matters less than the last column. In vllm/models/muse_spark/nvidia/muse_moe_runner.py at the v0.33.0 tag, the NVFP4 MoE layer accepts exactly two backends. On compute capability 9.x it uses Marlin and logs "MuseSpark NVFP4: Marlin W4A16 GEMMs with Triton finalization.", which means FP4 weights dequantized against BF16 activations, and it requires data-parallel and PCP size 1. Everywhere else it requires FLASHINFER_TRTLLM, whose device check supports only the Blackwell family at capability 10.x, or it raises "MuseSpark NVFP4 MoE requires FLASHINFER_TRTLLM or Marlin on Hopper". So B200 and B300 get native FP4 compute. H200 loads the model but on a dequantization path. RTX PRO 6000 Blackwell is capability 12.0, matches neither branch, and does not load the Muse NVFP4 path at 0.33.0, whatever its 768 GB would otherwise allow. Our RTX PRO 6000 vs H100 vs L40S comparison covers the card trade-offs in general; for this model on this vLLM release the question is already answered. On AMD, the model file states support for Quark W8A8 FP8 or OCP-MXFP4 W4A4 instead.
Recommended node for a regulated site
Use 8x B200 with FP8 KV and a 128K context cap as the default. It runs the native NVFP4 path, keeps a ceiling of 176 concurrent 128K sequences, and leaves room for a drafter. Reserve the full 1M context for a dedicated route with a handful of concurrent sequences: even on 8x B200 the ceiling is 22 at FP8. Buy 8x H200 only if the hardware is already on the floor, and size it at tensor parallel 8. Whether the Marlin route combines with expert parallelism is not stated in the source, so do not plan on it there. The launch below implements that B200 recommendation. It runs on B200 or B300 through the FlashInfer TRT-LLM path. Replace the repo id with your local mirror directory on an air-gapped host. The Muse fused QK-norm, RoPE and KV-insert op handles kv_cache_dtype fp8 and fp8_e4m3, so the FP8 KV flag is supported on this model's own path. --max-num-seqs 64 sits well inside the 176-sequence ceiling and is a starting point, not a tuned value. If startup fails on KV allocation, our guide to vLLM KV cache OOM errors walks through how --gpu-memory-utilization, --max-model-len and --max-num-seqs trade against each other. Meta's own vLLM example uses --tensor-parallel-size 4 with --max-model-len 1048576, and its format table says the NVFP4 build serves on four Blackwell Ultra GPUs; that is the 4x B300 row, where the 1M ceiling is 15 sequences at FP8.
| Node | Aggregate | Headroom after weights | Max sequences, BF16 KV (32K / 128K / 1M) | Max sequences, FP8 KV (32K / 128K / 1M) | NVFP4 path at vLLM 0.33.0 |
|---|---|---|---|---|---|
| 8x H200 (141 GB) | 1,128 GB | 470.1 GiB | 199 / 56 / 7 | 398 / 112 / 14 | Marlin W4A16, DP1 only |
| 8x B200 (180 GB) | 1,440 GB | 737.4 GiB | 312 / 88 / 11 | 625 / 176 / 22 | FlashInfer TRT-LLM, native W4A4 |
| 4x B300 (288 GB) | 1,152 GB | 490.6 GiB | 207 / 58 / 7 | 415 / 117 / 15 | FlashInfer TRT-LLM, native W4A4 |
| 8x B300 (288 GB) | 2,304 GB | 1,477.7 GiB | 626 / 176 / 22 | 1,252 / 353 / 45 | FlashInfer TRT-LLM, native W4A4 |
| 8x RTX PRO 6000 (96 GB) | 768 GB | 161.6 GiB | not loadable | not loadable | No matching backend |
Serving defaults that change memory: xhigh reasoning and top_k
The model card sets reasoning effort through the reasoning_effort request parameter or chat_template_kwargs, with four accepted values: low, medium, high and xhigh. "The default is xhigh," and "any other value returns an error." Every request that leaves it unset therefore asks for the longest reasoning trace, and every one of those output tokens occupies full-attention KV at 64 KiB per token at BF16, the same as a prompt token. No published per-effort token counts exist, so size from the mechanism: set effort per route, and give extraction or classification routes low or medium so they stop consuming KV budget the long-document routes need.
Sampling defaults also differ from what the card recommends. The card says temperature 1.0, top_p 1.0 and top_k disabled. generation_config.json in the repository ships do_sample true, temperature 1.0, top_p 1 and top_k 64, and vLLM 0.33.0 loads generation_config.json from the model path by default. A default vLLM server therefore samples with top_k 64. If you want the card's behaviour, set top_k explicitly in the request.
There is no MTP head in config.json, so speculative decoding needs a separate draft checkpoint with its own weights, its own KV and its own license. Meta's developer example pairs the model with Inferact/Muse-Spark-Open-DSpark, a 5-layer BF16 draft of 10.6 GB used with 7 speculative tokens, and vLLM 0.33.0 ships a DSpark draft model implementation for CUDA and ROCm. NVIDIA publishes Muse-Spark-Open-816B-A42B-DFlash2, a 4.38B BF16 drafter of 8.77 GB for Blackwell under the NVIDIA Open Model Agreement, which is a second license for legal to read. Budget 9 to 11 GB of weights per drafter in the headroom column above, and use our guide on speculative decoding running slower than baseline to check whether it pays for itself at your batch size.
Muse Spark Open license and gated access: clear them before the GPU order
Both Meta repositories, the BF16 and the NVFP4, report gated as manual in the Hugging Face API. Each access request is approved by the repo owner. Before approval only README.md and LICENSE.md are readable; config.json, generation_config.json, USE_POLICY.md and the weights return HTTP 401. Neither the card nor Meta's developer post documents how long approval takes. Treat it the way you treat a vendor security questionnaire: start it before the hardware order, not after the node arrives.
Run the request like a procurement record
- Request access from the account that will own the download and the mirror, not from an engineer's personal account. That ties the approval to the organisation that accepts the license, not to one person.
- Pin the revision. Both repos were last modified on November 10, 2026; the NVFP4 repo's revision at that point is 6b5fa7bb2d5bcf422a01852ae562a8f7f52b55c3. Record the sha you pulled, and re-check it before every mirror refresh, because the change history is not visible until access is granted.
- Mirror the governance files with the weights. The NVFP4 repo carries LICENSE.md and USE_POLICY.md alongside the 127 shards, config.json, generation_config.json and hf_quant_config.json. An air-gapped copy that drops the license files breaks condition 1 of the license.
- Do not use the ungated third-party mirrors to skip the queue. Some carry shards with sha256 values identical to Meta's and the same LICENSE.md, but the license binds anyone "accessing or obtaining a copy" regardless of source, and the audit trail is cleaner when the weights come from Meta's repo under an approved request.
What the license actually says
The Muse Model License (LICENSE.md, dated October 15, 2026) opens with an MIT-style grant: use, copy, modify including fine-tuning, create derivative works, distill, merge, publish, distribute, sublicense and sell. Its numbered sections then attach conditions a regulated buyer must clear, and one clause on outputs: For licensees in the European region, "Meta" means Meta Platforms Ireland Limited; elsewhere it is Meta Platforms, Inc. That matters for an EU bank's contract register. Internal deployment for your own staff sits inside the Section 3 exemption, but the Section 2 display duty has no internal-use carve-out. Whether a line in an internal tool's documentation or settings menu satisfies it for a staff-only assistant is a question for counsel; the text allows those locations, so ask early.
- 1. Include the copyright notice, the license and the Usage Policy in all copies of the software and its Derivatives, and comply with the Usage Policy (Exhibit A).
- 2. "Licensee shall prominently display "Built with Muse Spark Open" in the user interface of each product or service that uses the Software or any Derivatives, or in the accompanying documentation, website, or settings menu." Derivatives include any model trained, fine-tuned or distilled "using the Software or any outputs or results thereof", so a smaller model trained on Muse outputs inherits the obligation.
- 3. Offering the model as a service to third parties via API, with affiliate revenue above USD 20 million in any 12 months, requires notifying Meta and a separate written agreement. That condition "does not apply to (a) internal use, research, or any commercial activity that is not Model as a Service."
- 4. Meta claims no ownership of outputs.
Usage Policy lines a regulated buyer must map
Exhibit A prohibits uses that land squarely in regulated sectors. It bars "unauthorized or unlicensed practice of any profession including, but not limited to, financial, legal, medical/health" practice; collecting or inferring sensitive personal information such as health or demographic data without the legal right to do so; discrimination in employment, credit or housing; "Military, warfare, nuclear industries or applications, espionage, use for materials or activities that are subject to the International Traffic Arms Regulations (ITAR)"; and "Operation of critical infrastructure, transportation technologies, or heavy machinery." It also prohibits representing outputs as human-generated. A defence supplier with ITAR programmes or a utility operating grid assets has to clear those clauses before anything else in this post matters. Banks, insurers and hospital groups should map their intended use cases against the professional-practice and sensitive-data clauses, and design review steps around them. The card's own deployment guidance points the same way: run the model inside a broader system with guardrails for the use case, avoid exposing raw chain-of-thought, and require human confirmation for irreversible actions in agentic workflows. Those are the controls we build into on-premise deployments anyway, and here they double as license hygiene. For how this model fits alongside the rest of the open-weight landscape, the LLMs and models pillar maps the cluster. One naming note: Muse Glimmer is a separate 30B model with its own vLLM architecture entries, and Meta's hosted Muse Spark is not these open weights.
What to do this week
FAQ
Quick answers to the questions this post tends to raise.



