Self-Host LLM vs API: When the Break-Even Math Flips in 2026

The Naïve Break-Even (And What It Hides)

The number you'll see on Twitter, on most "self-host vs OpenAI" blog posts, and in roughly half the LinkedIn slide decks circulating among AI eng leads in 2026 is some variant of this:

Below those, stay on API. Above, self-host. The math comes from dividing premium token cost (input + output, weighted) by H100 sticker pricing at full utilization, then assuming the rest of the stack is free.

It's not a useless number. It's a starting point. But it leaves out everything that turns a Friday slide into a Monday operating reality:

The naïve break-even number assumes all five of those line items are zero. They are not zero. They are roughly half of the spreadsheet.

The Full TCO Model — Line by Line

The cost worksheet we use looks like this. The right column is what gets left off the napkin.

The TCO column is the one that determines whether the project is profitable. A common shape across self-host migrations we've audited: the team sizes the project on column 1, ships it on column 2, and is back to API for the long-tail workloads inside 90 days because the headcount math didn't survive contact with hiring.

The instinct is to pick on $/hr. Across the stacks we've reviewed, the right pick is almost always Modal or Baseten until your monthly premium spend justifies a dedicated platform engineer — at which point the bare-metal number starts to matter.

Token Throughput on H100s — A Sanity-Check Baseline

Before sizing reserved capacity, you need a defensible throughput number for the model class you're hosting. The rule of thumb across the open-weight defaults that matter in 2026:

Two cautions. First, treat published vLLM/SGLang and vendor numbers as ceilings, not budgets — real throughput depends on your prompt length distribution, batch size, KV cache config, and decoding strategy. Our vLLM vs SGLang inference engine comparison covers the engine-level decision. Second, the only number that matters is the one you measure on your own prompt distribution. Synthetic benchmarks lie, especially at long context.

Premium API Price Trajectory — The Moving Target

The break-even number is not stable. Over the last 12 months:

For the self-host decision, this matters in two directions. On simple workloads, the API is winning faster than your spreadsheet predicted, and the break-even threshold keeps climbing. On reasoning- and tool-heavy workloads, the threshold keeps falling. Run the TCO model every six months — the number that justified self-hosting in November doesn't necessarily justify it in May.

The Hybrid Pattern (Where Most Production Systems Land)

Most production AI systems we've reviewed at the $25-100K/mo premium spend band converge on the same shape, even when the team starts out aiming for pure self-host or pure API:

The savings come from sizing reserved GPU capacity for the median, not the P95 — the spike traffic gets routed to API, where elastic capacity is the vendor's problem. That alone is the difference between a 40-50% utilization stack and a 70-85% one, and it's where the break-even flips down to roughly $25K/mo.

# litellm config.yaml — hybrid router with self-hosted vLLM primary
model_list:
  # primary: self-hosted Qwen3.6-35B on internal vLLM endpoint
  - model_name: chat-default
    litellm_params:
      model: openai/qwen-3.6-35b
      api_base: https://vllm-internal.svc.cluster.local/v1
      api_key: os.environ/INTERNAL_API_KEY
      timeout: 8

  # frontier tier: premium API for the long-context / reasoning tail
  - model_name: chat-frontier
    litellm_params:
      model: anthropic/claude-opus-4-7
      api_key: os.environ/ANTHROPIC_API_KEY
      timeout: 30

router_settings:
  routing_strategy: simple-shuffle
  num_retries: 1
  allowed_fails: 2
  cooldown_time: 30

  # self-host outage → spill to frontier automatically
  fallbacks:
    - chat-default: ["chat-frontier"]

  # any request beyond self-host context budget escapes to frontier
  context_window_fallbacks:
    - chat-default: ["chat-frontier"]

Worked Scenarios — Three Shapes We See Converge

Hedged plural scenarios drawn from the deployments we've audited. Numbers are illustrative ranges, not single-case figures.

Decision Matrix — Which Shape Are You?

If you're scoring B on most rows but feel pulled toward C, the bias to resist is the founder/engineer instinct that "we can run our own GPUs." You probably can. You probably shouldn't, until the spend justifies the hire. If you're scoring A but feel pulled toward B, the bias to resist is "open weights are cheaper" — they're cheaper per token, not per delivered request, and the difference is the whole post.

The "Don't Self-Host" Checklist

Before committing, run the negative case. Self-host is the wrong answer if any of these are true:

Hit any two and the spreadsheet has already decided for you.

Practical Migration Path (Without a Day-One Infra Hire)

For teams in Shape B who don't yet have the platform muscle for full self-host, the phased rollout that minimizes risk:

Each step is reversible. The mistake is to skip steps 1-2 and cut over directly because "the math works on paper." It does — until the first sev-1 outage at 3 a.m. burns six months of compute savings in a single weekend.

When the Math Actually Flips

The honest answer to "should we self-host?" in 2026 is: probably not yet, and definitely not on the napkin. The break-even number you can defend in front of finance is the one that includes idle GPU, tail traffic, headcount, eval, churn, and incident cost — and that number is closer to $50-80K/mo for pure self-host, or $25K/mo for the hybrid pattern that most production systems converge on anyway.

The teams that get this right share a habit: they re-run the TCO model every six months, against current vendor pricing and current open-weight throughput. The teams that get it wrong run the model once, build the slide deck, and treat the number as static. In a year where premium input pricing dropped 30-50% and open-weight throughput climbed sharply on the same hardware, static numbers age in weeks, not quarters.

If you're using this framework to bring a recommendation to your CFO, the part that matters is showing both columns of the worksheet — sticker math and TCO math — and being explicit about which line items are assumed away in each. That's the conversation our build vs buy decision framework walks through, and it's the one that holds up in front of finance whether the answer is API, hybrid, or self-host. For deeper architectural patterns across the AI for Business cluster, the gateway, attribution, and routing posts linked above are the rest of the toolkit.