Self-Hosted vs API Embeddings: Qwen3, Voyage 2026

The 2026 Shift: Open-Weight Embeddings Caught the Closed APIs

The headline number is Qwen3-Embedding-8B at 70.58 on MTEB multilingual. What makes that significant is not the score in isolation but who holds it. An Apache-licensed, downloadable, self-hostable model is leading a benchmark that closed commercial APIs dominated for years. You can pull the weights, run them on your own hardware, and get retrieval quality that matches what you would otherwise rent per token.

The 32K context window matters as much as the rank. Long-context embedding models let you embed larger chunks without aggressive splitting, which reduces the number of vectors you store and the number of fragments your retriever has to stitch back together. If you have been fighting chunk-boundary problems where the relevant sentence lands at the edge of a 512-token window, a 32K context model changes the chunking calculus entirely.

The strategic point is that self-hosting is no longer the consolation prize. Two years ago, choosing an open model meant accepting a measurable retrieval-quality hit in exchange for control and cost. In 2026 you can have the top multilingual model and host it yourself. The decision shifts from "quality versus control" to a pure operations-and-economics question, which is a much healthier place to be making it.

Reading the Leaderboards Right: Multilingual vs English, Narrowing Margins

The most common mistake I see is treating MTEB as a single ranked list. It is not. The multilingual and English boards reward different things, and the model that tops one will not top the other.

Qwen3-Embedding-8B leads multilingual at 70.58. Gemini Embedding 001 leads English at 68.32. If your corpus is English-only, the multilingual champion is not automatically your pick, and vice versa. Match the board to your actual language distribution before you read a single rank.

The second mistake is treating small gaps as meaningful. Gemini at 68.32 versus Voyage in the high 67s is under a point. That margin is well inside the noise of how any given model performs on your specific domain. Across the RAG systems we have audited at Particula Tech, two models within a point on MTEB routinely swap places by 10 to 20 percent on a client's own test set, because the benchmark's task mix has little to do with their invoices, support tickets, or legal contracts.

Use the leaderboard the way it is actually useful: as a filter that produces a shortlist of three or four credible candidates. Then decide with your own data.

Scores reflect the April 2026 MTEB snapshots cited below; treat them as a shortlist signal, not a final verdict.

Qwen3-Embedding, EmbeddingGemma, Gemini, and Voyage Head to Head

These four cover the meaningful design space, so it helps to be opinionated about where each one wins.

Cost Math: Self-Hosting on an A100 vs Per-Token API Pricing

Here is the number that reframes the whole debate. Deploying Qwen3-Embedding-8B on a single A100 yields batched throughput on par with commercial APIs at roughly 1/20th the cost per million tokens once the hardware is amortized.

One-twentieth is the kind of multiple that flips decisions. But it comes with a sharp condition buried in the phrase "once amortized." A GPU costs the same whether it runs at 5 percent utilization or 95 percent. The 1/20th figure assumes you keep the card busy. An idle A100 is the most expensive embedding service you can buy.

So the cost question is not "how many tokens do I embed?" It is "can I keep a GPU utilized?" Batch your embedding jobs, embed your whole corpus during indexing, and feed a steady stream of new documents, and the A100 stays warm and the math holds. Embed a few hundred documents a day in unpredictable bursts, and you are renting an expensive idle card while an API would have charged you cents.

A simple way to frame the break-even:

API monthly cost      = tokens_per_month × api_price_per_token
Self-host monthly cost = gpu_hourly_rate × 730 hours
                         (fixed, regardless of utilization)

Self-host wins when:
  tokens_per_month × api_price_per_token  >  gpu_hourly_rate × 730

Equivalently, you need enough sustained volume to keep one
A100 above roughly 30-40% utilization. Below that, the API
is both cheaper and zero-ops.

This is the same shape as the self-host LLM versus API break-even math for generation models: the variable cost of an API beats the fixed cost of hardware until volume crosses over, and the crossover is governed by utilization, not headline token counts. The 1/20th multiplier just means the crossover for embeddings arrives at lower volume than people expect, because embedding throughput on a single card is high once you batch properly.

Two costs people forget on the self-host side: storage and re-indexing. Embedding dimensions drive your vector-database footprint, and a model that produces larger vectors costs more to store and search regardless of how cheap the inference was. Size that before you commit; our note on embedding dimensions and vector search walks through the storage-versus-recall trade.

On-Device and Local-First RAG: The EmbeddingGemma Niche

EmbeddingGemma-300M exists for a use case the API providers structurally cannot serve. It is built for on-device inference, running on mobile and edge hardware while delivering retrieval quality strong enough for real local-first RAG.

The value is not cost or even latency, though both are good. It is that the data never leaves the device. A note-taking app that indexes your private notes, a clinical tool that searches patient records on a tablet, a field application that has to work with no connectivity: none of these can ship text to a remote embedding API without either breaking a privacy promise or breaking when the network drops. EmbeddingGemma lets the entire retrieval path, embedding included, run locally.

This is why the self-host-versus-API framing is sometimes a false binary. The right architecture for some products is both: EmbeddingGemma on the client for private, offline retrieval, and a server-class model like Qwen3-8B in the cloud for the heavier indexing and cross-document search. The two models do not compete; they cover different tiers of the same system.

One constraint to respect: a small on-device model and a large server model produce different vector spaces. You cannot mix vectors from EmbeddingGemma and Qwen3 in the same index and expect coherent similarity. If you run both, run two indexes, or accept that on-device retrieval and server retrieval are separate retrieval paths with separate stores.

Decision Framework: When Self-Hosting Actually Pays Off

Strip away the leaderboard noise and the decision comes down to a few gates.

Default to an API when your volume is low or bursty, your corpus is English-heavy, and you have no data-residency requirement. Gemini Embedding 001 or Voyage will give you frontier quality with zero operational burden, and below the break-even they are genuinely cheaper than an idle GPU. Do not self-host to save money you are not actually spending.

Self-host Qwen3-8B when at least one of these holds: you have sustained volume that keeps an A100 utilized (the 1/20th cost advantage is real but only at utilization), you have a hard data-residency or privacy constraint that rules out third-party APIs, or your workload is meaningfully multilingual and you want the top multilingual model. Any one of these justifies the operational cost; volume alone has to clear the utilization bar.

Go on-device with EmbeddingGemma when the data cannot leave the device or the app must work offline. This is not an optimization, it is a requirement that only a local model satisfies.

If you are standing up or re-architecting a retrieval pipeline and want this scored against your real token volume and language mix, that is exactly the work our RAG engineering practice does before a single line of indexing code gets written. The cost math and the retrieval evals come first; the model choice falls out of them.

Operational Reality: Versioning, Re-Indexing, and Lock-In

The model comparison gets the attention, but the operational layer is where embedding decisions actually hurt later. Three things deserve explicit planning.

Re-indexing is the hidden cost of switching. Your stored vectors are only comparable to other vectors from the same model and version. The moment you change embedding models, every document in your index has to be re-embedded before search works correctly again. At scale that is a full corpus pass, GPU time, and a careful cutover. This is why the model choice is sticky and why getting it right up front matters more than for most infrastructure decisions. Our guide on when to re-embed documents covers running old and new indexes in parallel during the migration so search never goes dark.

API model versions move under you. A managed embedding API can update or deprecate a model version on its own schedule. If a provider silently changes the model behind an endpoint, your stored vectors and your freshly embedded queries can drift out of alignment, and retrieval quality degrades without any code change on your side. Pin to explicitly versioned model names where the provider offers them, and monitor retrieval quality continuously so drift surfaces as a metric and not as a user complaint.

Lock-in is asymmetric. With an API you avoid hosting but accept dependence on a vendor's pricing, availability, and version policy. With self-hosting you take on the operations but own the model artifact outright; nobody deprecates your local copy of Qwen3-8B. For some teams that ownership is the entire point, especially under compliance regimes where you must be able to reproduce exactly which model produced a given vector months later. Open weights make that auditability trivial. A closed API makes it a vendor support ticket.

The throughline: pick the embedding model as if you will be living with it for a year, because re-indexing cost means you probably will. Run the cost math at your real volume, test the shortlist on your own queries, and let data residency and operations, not a one-point MTEB delta, make the final call.

Sources