turbo-fieldfare is an open-source Swift and Metal engine that runs Gemma 4 26B-A4B, a Mixture-of-Experts model with roughly 3.9B active parameters per token, on any Apple Silicon Mac with 8GB of RAM. It keeps a small shared core and the FP16 KV cache resident and streams the routed experts each token needs from a custom .gturbo file on disk through a 16-slot per-layer LFU cache, holding process memory near 2GB. Measured throughput is 5.1 to 6.3 tokens per second on an 8GB M2 MacBook Air and 31 to 35 tokens per second on a 24GB M5 Pro. It went from its first commit to 2,225 GitHub stars in under two weeks, and the tradeoff is real: this is a memory-constrained-hardware unlock, not a throughput or production-serving win.
893 points and 328 comments carried a Show HN post to the top of Hacker News on July 29, 2026: "Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac." The claim reads like a typo. A 26-billion-parameter model, the kind that normally needs tens of gigabytes of VRAM just to load, running inference in roughly the memory footprint of a browser tab. It isn't a typo. turbo-fieldfare, the Swift and Metal inference engine behind the post, reached 2,225 GitHub stars within two weeks of its first commit, and the number that matters more than the star count is 5.1 to 6.3 tokens per second on an 8GB MacBook Air, a machine that has no business running a 26B-parameter model at all.
The mechanism is disk streaming applied to a Mixture-of-Experts model, and it's worth understanding on its own terms, not just as a neat demo. This post covers what turbo-fieldfare actually does under the hood, what "26B parameters" means once you know only a fraction are ever resident in memory, the real throughput numbers against fully-resident loading, where the approach breaks down, and how to set it up if you want to run Gemma 4 locally on your own Mac. The repository is under two weeks old as of this writing and still moving fast, so treat the specifics below as accurate at the time of research (2026-07-31) and verify against the live repo before you build anything that depends on the exact numbers holding steady.
The Claim: Running Gemma 4 26B Locally in 2GB of RAM
turbo-fieldfare's own repository description is blunt about what it does: "Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook." The repo was created on 2026-07-17, is written in Swift, and ships under an Apache 2.0 license. At last check (2026-07-31) it sat at 2,225 stars, 94 forks, and 16 open issues, and it is explicitly not affiliated with, sponsored by, or endorsed by Google, despite running Google's model.
The "sounds impossible" reaction is the correct first instinct. A 26-billion-parameter model at 4-bit quantization is still on the order of 13 to 14GB of weights, before you add a KV cache. Loading that into 2GB of process memory should not work by the normal rules of "quantize the model, load it, run it." turbo-fieldfare doesn't break those rules. It sidesteps the assumption underneath them: that all of a model's weights need to be resident in memory at the same time to generate a single token. For a dense model, that assumption holds. For the Mixture-of-Experts architecture Gemma 4 26B-A4B uses, it doesn't, and that gap is the entire trick.
How turbo-fieldfare's Disk-Streamed Weight Loading Works
The installed model lives on disk in a custom .gturbo layout rather than the standard checkpoint format. During installation, turbo-fieldfare never materializes the full source checkpoint: it streams the required byte ranges directly from the pinned Hugging Face revision over range requests and repacks them into .gturbo as they arrive, validating manifest and file hashes before marking the install complete. That first download moves roughly 15GB and installs about 14.3GB to disk, all without ever holding a complete copy of the model in memory or as a separate staging file.
At inference time, the engine splits work across two resources that run concurrently. Metal computes attention and the router from the weights that stay resident: a shared core of roughly 1.35GB plus the KV cache, held in FP16 with a circular buffer for the model's routing-heavy layers and linear storage for its full-attention layers. The CPU takes the router's top-scoring expert IDs for the current token and checks them against a 16-slot LFU (least-frequently-used) cache maintained per transformer layer. Anything not already cached triggers bounded parallel pread calls into Metal-visible buffers, timed to overlap with Metal's work on the shared layers rather than stall behind it. Prefill batches up to 128 tokens per chunk to make that overlap efficient; generation runs one token at a time, since each new token can route to a different set of experts.
The practical effect: you're not loading a 26B-parameter model into RAM. You're loading a roughly 1.35GB shared core plus KV cache, and paging in the specific expert weights each token actually needs, right before Metal needs them, discarding the ones that fall out of the LFU cache.
What Gemma 4's 26B Parameters Actually Means When Only 4B Are Active
The "A4B" in Gemma 4 26B-A4B is the detail that makes all of this legal. It stands for roughly 3.88 billion active parameters per token out of 26 billion total, Google's naming convention for a sparse Mixture-of-Experts model. On any given token, a router selects a small subset of expert sub-networks (turbo-fieldfare's README describes the router picking from the top 8) and only those experts, plus a shared dense path, actually participate in the forward pass. The other experts, the large majority of the 26B total, sit idle for that token.
That's true of every MoE model, and it's exactly the property vLLM, llama.cpp, and MLX already exploit for GPU and unified-memory offloading of bigger MoE models like DeepSeek V4 and GLM-5.2, keeping inactive experts in slower memory tiers and paging in what the router asks for. turbo-fieldfare takes the same idea and pushes the slow tier all the way to SSD instead of stopping at system RAM, which is what makes an 8GB memory ceiling survivable for a 26B-parameter model in the first place.
On the quantization side, weights use MLX-style affine 4-bit quantization with a group size of 64 for embeddings, attention projections, and expert weights, with the router itself kept at 8-bit precision, presumably because routing decisions are more sensitive to precision loss than the expert weights they select. That combination is what gets the on-disk footprint down to roughly 14.3GB and the resident core down to roughly 1.35GB.
Benchmarked Throughput: M2 vs M5 Pro, and vs Fully-Resident Loading
The published numbers, drawn from the project's own README, are the clearest evidence that this is a real tradeoff and not a free lunch:
Two things stand out. First, the roughly 5-6x jump from M2 to M5 Pro is bigger than the raw chip generation gap alone would predict, which suggests the extra unified memory on the M5 Pro configuration lets more of the working expert set stay warm in the LFU cache and the OS page cache, cutting the number of cold pread calls per token. Second, neither number is close to what you'd expect from a model this size served on GPU infrastructure with continuous batching, and that's by design: this is a single-user, single-request local tool, not a throughput-optimized server.
turbo-fieldfare's own documentation is explicit that these benchmark numbers are reference points, not performance ceilings, and that results vary with prompt length, output length, page-cache state, and hardware configuration. It doesn't publish a side-by-side number for the same weights loaded fully resident on hardware with enough memory to skip streaming altogether, so treat the gap between "streamed" and "fully resident" as directional rather than an exact multiplier: disk streaming is reliably slower than RAM-resident inference, the published numbers just don't pin down by how much on identical hardware.
| Hardware | RAM | Tokens/second |
|---|---|---|
| M2 MacBook Air | 8GB | 5.1-6.3 |
| M5 Pro | 24GB | 31-35 |
Where This Breaks: Sustained Throughput, Disk I/O, and SSD Wear
The honest limitations list matters as much as the headline number, and turbo-fieldfare's own documentation states several of them plainly.
Cold start and page-cache dependence. Performance depends on page-cache state and SSD speed. The first prompt after a fresh launch pays a colder-cache cost than the tenth prompt in the same session, and that variance is real, not a bug to file.
No multi-tenant story. The project states that only one model-owning product should run at a time, and its OpenAI-compatible server supports single-prompt prefix reuse only, not the multi-user prefix caching a production serving stack needs. This is a local single-user tool, not a drop-in replacement for a serving layer built for concurrency.
Text-only, and unverified correctness. There's no image, audio, or video support, and the project states plainly that model repeatability and correctness aren't guaranteed, so results that matter should be independently verified rather than trusted from a single run.
Read pressure, not write pressure. SSD wear is driven by write cycles, and disk-streamed inference is almost entirely reads once the model is installed, so the wear risk here is lower than it sounds. The realistic cost is performance, not drive lifespan: sustained heavy read traffic competes with page cache and any other disk-bound process running at the same time, and a busy disk degrades your tokens-per-second before it threatens your SSD's rated endurance.
The upfront tax. Getting to a first prompt costs a roughly 15GB download and about 14.3GB of local storage before generation even starts, which is a real cost on a MacBook Air's smaller default drive sizes.
Setting Up turbo-fieldfare on an M-series Mac
The listed requirements are an Apple Silicon Mac with at least 8GB of RAM, macOS 26 with Metal 4, and Swift 6.2 or later; the build is arm64-only, so older Intel Macs are out regardless of RAM.
git clone https://github.com/drumih/turbo-fieldfare.git cd turbo-fieldfare swift build -c release # launch the built app .build/release/TurboFieldfareMac
From there, the flow is: select Download to pull the roughly 15GB of model data via Hugging Face range requests directly into the .gturbo layout, wait for the manifest and file-hash validation to confirm the install completed cleanly, select Load Model, then enter a prompt and generate. Because it's under two weeks old and still moving fast, confirm the exact build commands and any new flags against the current README before you run this, rather than treating the steps above as permanently fixed.
turbo-fieldfare vs Quantization vs Cloud Inference: When to Use Which
Three different tools solve three different constraints, and conflating them is the most common mistake in local-inference decisions.
If your Mac has enough unified memory to hold a quantized MoE model resident, standard offloading through a tool like MLX or llama.cpp will outrun disk streaming, because the whole reason to stream from SSD is that you don't have that memory to spare. Our guide to running GLM-5.2 locally covers that fully-resident MoE offloading path in detail, including the quantization tradeoffs on a much bigger 744B-parameter model. If you're deciding between any local setup and a hosted endpoint, the current DeepSeek V4 and GLM-5.1 hosting price spread shows what you're comparing local throughput against in dollar terms, and our self-host versus API break-even analysis walks through when owning hardware actually pays off. And if the workload is a production, multi-user endpoint rather than a single local session, that's a different problem with a different toolset, covered in our vLLM versus Ollama versus TensorRT-LLM comparison.
| Approach | Memory needed | Typical throughput | Best fit |
|---|---|---|---|
| turbo-fieldfare (disk streaming) | 8GB min, Apple Silicon | 5-6 tok/s (M2), 31-35 tok/s (M5 Pro) | Running a model that doesn't otherwise fit on hardware you already own |
| Quantized, fully resident (MLX, llama.cpp, Ollama) | Full quantized weight size in RAM/VRAM | Faster than streaming once it fits | Hardware with enough unified memory or VRAM to hold the weights outright |
| Production inference server (vLLM) | Multi-GPU VRAM | Continuous batching, many concurrent users | Multi-user endpoints, not single-machine local use |
| Cloud API | None locally | Provider-dependent, elastic | Production concurrency and uptime without owning hardware |
Disk-Streaming MoE Inference Is Becoming a Pattern
turbo-fieldfare is not the only project to land on disk streaming for Mixture-of-Experts models in the past few weeks. A separate open-source engine released roughly two weeks earlier applies the same core idea at a much larger scale: it keeps a small set of dense layers resident and streams the routed experts of a 744-billion-parameter MoE model from disk, fitting the whole thing into around 25GB of RAM on consumer hardware rather than the roughly 1.5TB a fully-resident deployment of a model that size would otherwise need.
The pattern underneath both projects is the same insight applied at different scales: sparse activation means most of a MoE model's weights sit idle on any given token, so the traditional rule that inference requires the full model resident in fast memory was always more conservative than the architecture demanded. What's changed in the past few weeks is that independent engineers are now shipping production-quality engines that treat SSD as a legitimate third tier in the memory hierarchy, not just a research curiosity. For local, single-user inference on memory-constrained hardware, that's a genuinely new capability. For anyone weighing this against a production deployment, it's worth remembering that the throughput gap between disk-streamed and fully-resident inference doesn't close as the model gets bigger, it's a fixed cost of every cache miss, so the calculus of when to use this pattern versus provisioning real memory or renting GPU capacity is one worth running against your actual traffic, not a Hacker News benchmark. For more on sizing that decision, see our LLM models pillar page.
FAQ
Quick answers to the questions this post tends to raise.
