Promptfoo Alternatives for Self-Hosted Red Teaming in 2026

Promptfoo's blog post announcing that it is joining OpenAI is dated 9 March 2026, and its excerpt says "The open-source project will continue." The record since then supports that. The GitHub repository is not archived, the license in package.json is still MIT, and the npm package kept shipping: 0.124.1 was published on 8 October 2026.

So Promptfoo has not exited, gone closed-source or changed license, whatever some alternatives listicles imply. The open-source CLI you installed last year is the same kind of tool today. The docs also describe Promptfoo Enterprise On-Prem as "our on-premises solution that includes a dedicated runner for deployments behind your firewall", which matters later.

What did change is who runs the hosted service the CLI talks to. That service is where the regulated-industry question lives, and it is not a question about the license at all.

The red-team configuration docs at 0.124.1 state the default plainly: "By default, Promptfoo uses your local OpenAI key for redteam attack generation and grading. If you do not have a key, Promptfoo will automatically proxy requests to our API for generation and grading." Add a local redteam.provider and the disable flags, and four setups result:

Telemetry is a separate stream on top of that: a PostHog client sends events to a.promptfoo.app, and every event is also posted to r.promptfoo.app. A version check calls api.promptfoo.dev unless PROMPTFOO_DISABLE_UPDATE is set.

Read the second row twice. Setting an OpenAI key does not fully take Promptfoo's own API out of the loop: the remote-only plugins and unaligned harm categories still call it, and everything else defaults to OpenAI, which agreed in March to acquire Promptfoo. That is still external egress.

This is the finding most worth handing to your engineer. The red-team docs at 0.124.1 say: "If you need to use a custom provider for generation, you can still benefit from our remote service by leaving PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION set to false (the default)." The troubleshooting page on grading results describes configuring redteam.provider for local grading while leaving remote generation at its default. A custom provider and remote generation coexist by design.

The source confirms it. shouldGenerateRemote() in src/redteam/remoteGeneration.ts returns false when a disable flag is set. Otherwise it returns true when PROMPTFOO_REMOTE_GENERATION_URL is set, when you are logged into Promptfoo Cloud, when OPENAI_API_KEY is absent, or when --remote is passed. It never reads redteam.provider. The paths that branch on it include:

What a local provider does move: LLM-rubric grading skips the remote path when redteam.provider is set, and the crescendo, iterative and iterativeTree multi-turn providers treat "remote generation enabled and no red-team provider" as their remote condition, so their attacker calls go to your model. Grading in-network is a real gain. It is not the same as a local run.

Promptfoo's docs warn that "Disabling remote generation may result in lower quality adversarial inputs", and that local generation quality "is generally low for most models." Those are vendor statements, and none of the sources behind this post measures the gap independently. The bigger cost is visible in source, and it is coverage.

REMOTE_ONLY_PLUGIN_IDS in src/redteam/constants/plugins.ts carries the comment "These have no local implementation and always call the remote API". When remote generation is disabled, the function that builds those plugins logs an error and returns an empty list. The unaligned harm categories behave the same way.

The first four rows add up to 28 regulated-domain plugins. For a bank or a hospital, these are the plugins written for your domain, and they exist only as hosted generation. Promptfoo's default plugin set includes all the harm plugins too, so a default red-team config run fully local is quietly a smaller test than the same config run online.

The docs themselves illustrate the trap. The "Using Local Providers for Grading" example on the grading-results troubleshooting page sets redteam.provider to an Ollama model, lists two plugins, harmful:hate and excessive-agency, and says it runs entirely locally when combined with PROMPTFOO_DISABLE_REMOTE_GENERATION=true. Read against the 0.124.1 source, harmful:hate is an unaligned harm category that returns zero tests in that mode, so only excessive-agency generates. The documented local example drops one of its two plugins.

The practical rule: after any local run, compare the plugin list in the generated report with the list in your config. A plugin missing from the report was never tested. And once the harness is running, the metric that matters is attack success rate per category; see what prompt injection benchmarks show about attack success for how to read it.

The three Python options are not three versions of Promptfoo. Compare what each is for before comparing egress.

Every cell below comes from the source or docs at the pinned version.

None of these variables is a firewall, and Promptfoo's own FAQ and SECURITY.md say so about theirs. The guarantee an auditor accepts is a default-deny egress policy on the runner host, with only your in-network model endpoints allow-listed, and a staging run with the egress log open to show nothing else was attempted. Env vars reduce what tries to leave. The network rule proves what did.

In a bank, insurer or hospital, the confidential part of a red-team run is not the attack strings. It is what they carry and what comes back: the application purpose, the system prompt, the tool schemas, and target responses that can contain customer or patient data the model was grounded on. That changes the tool choice.

If you keep Promptfoo's hosted generation, ask three questions in writing: what the hosted service retains and for how long, whether its data-handling terms changed after March 2026, and whether the generation endpoint can run inside your network. The docs say Enterprise On-Prem includes a runner behind your firewall and that PROMPTFOO_REMOTE_GENERATION_URL can point the CLI at a different generation endpoint; what the on-prem product ships beyond that is a sales conversation, not a docs fact.

Every fully local path has the same hidden cost: the attacker and the judge are now models you host. A weaker local judge produces scores that look as confident as a strong one's, so calibrate it against a human-labelled sample before trusting its pass rate; our post on regression testing with LLM-as-judge covers that. Red teaming also finds gaps without closing them; the runtime control is a separate layer, compared in NeMo Guardrails vs Llama Guard vs Guardrails AI. And red-team runs belong in the same loop as your functional evals, as described in our evals-driven development workflow. More tooling comparisons sit in the AI development tools guide.

Our recommendation for a regulated team that cannot send prompts or outputs out: use garak as the baseline scanner and PyRIT for the multi-turn campaigns specific to your application, not Promptfoo in local mode, unless you have checked that your test plan survives the plugin loss. If your plan is built around the finance or healthcare packs, the honest choice is between hosted generation under contract and the on-prem product, not a local flag that silently removes them.

Take the red-team config you run today, whichever harness it uses. Put the runner in staging behind a default-deny egress rule that allows only your in-network model endpoints, run it once, and write down every blocked destination from the egress log. Then open the generated report and check its plugin or probe list against what your test plan says you are testing. Anything blocked is traffic your current setup sends out; anything missing from the report was never tested.