Langfuse v3 to v4 Upgrade: What Breaks When Self-Hosted

Two new ClickHouse tables replace traces and observations as the source for all reads. events_full is the immutable, full-fidelity table: every LLM call, tool execution and agent step is one row, with the trace-level attributes (user, session, tags, release, version) on the row instead of joined at read time. events_core is a lightweight projection with truncated input, output and metadata, populated by a materialized view, serving the table and chart queries.

Traces stop being a separate entity. Existing traces convert into virtual root spans of type SPAN with the span ID t-<trace_id>, so historic data stays browsable, and the UI collapses into a single Observations view that opens filtered to root observations only, reproducing the old one-row-per-trace table.

Initial table loads for large amounts of data go from seconds to milliseconds, and dashboard load times for large projects improve by 10x or more over longer time ranges. The new model is also what brings full-text search across inputs, outputs and metadata, the filter search bar, alerts, and the Observations API v2 and Metrics API v2 to self-hosted deployments.

The v4 schema migrations are not purely additive. Alongside creating the new tables, they drop seven tables superseded during the v3 line, whose live copies already moved to ClickHouse back then: traces, observations, scores and dataset_run_items in PostgreSQL, plus event_log, project_environments and the legacy dataset_run_items table in ClickHouse. Provided every v3 background migration completed, those drops remove only data that exists elsewhere.

If one did not complete, they do not. A migration that failed and was never retried, or that never ran because the deployment jumped forward from an older v3 release, leaves rows that were never copied into ClickHouse, and those are exactly the rows the v4 migrations delete permanently. This is the one step with no recovery path other than the backup.

Go to the latest v3 release first, let the background migrations drain, and verify. The UI shows this under the Langfuse version tag, in Background Migrations, where every entry has to show a finished state. The runbook version is one query against PostgreSQL:

-- Must return zero rows before upgrading to v4:
SELECT name, failed_at, failed_reason
FROM background_migrations
WHERE finished_at IS NULL;

Then back up both databases immediately before the server upgrade. There is no automatic downgrade once the v4 schema migrations have applied.

The ClickHouse traces, observations and scores tables are not among the seven that get dropped. They keep receiving every incoming event while you run legacy or dual, and the historic backfill only reads them.

ClickHouse carries the real requirement: 25.12 is what provides the lightweight updates, the JSON type and the full-text search the new model depends on. Sequencing matters as much as the number. Do the ClickHouse upgrade while you are still running Langfuse v3, which stays compatible with current ClickHouse versions, so you can complete the storage upgrade, run on it for a month, and only then touch the application image.

Those numbers are release lines, not pinned builds: choose the current patch inside the 25.12 or 26.4 line from the Langfuse ClickHouse deployment guidance.

-- Schema migrations: the v4 migrations add columns and indexes, replace views,
-- and change materialized view queries
GRANT DROP VIEW ON default.* TO 'user';
GRANT ALTER ADD COLUMN, ALTER MODIFY COLUMN, ALTER VIEW MODIFY QUERY ON default.* TO 'user';
GRANT ALTER ADD INDEX, ALTER DROP INDEX, ALTER MATERIALIZE INDEX ON default.* TO 'user';

-- System table reads for event propagation, the historic backfill, and the
-- v4 transition usage detection
GRANT SELECT(database, table, name, partition, partition_id, active, rows) ON system.parts TO 'user';
GRANT SELECT(database, table, is_done) ON system.mutations TO 'user';
GRANT SELECT(database, name, engine) ON system.tables TO 'user';
GRANT SELECT ON system.processes TO 'user';
GRANT SELECT ON system.query_log* TO 'user';

-- Merge control on the intermediate table used by the historic backfill
GRANT SYSTEM SYNC REPLICA, SYSTEM MERGES, ALTER SETTINGS ON default.observations_pid_tid_sorting TO 'user';

-- Clustered deployments (CLICKHOUSE_CLUSTER_ENABLED=true) only
GRANT READ ON REMOTE TO 'user';
GRANT CLUSTER ON *.* TO 'user';

# Chart v2.0.0 ships Langfuse v4 as its default appVersion.
# Pin the image so the chart migration and the v4 upgrade stay two separate changes.
langfuse:
  image:
    tag: "<your-current-v3-tag>"   # not the chart default

clickhouse:
  deploy: true       # this is what makes the chart v1 to v2 move a prerequisite

cd examples/upgrade-v1-to-v2/scripts

# Required: kubectl, helm >= 3.17, jq, python3
# YAML parsing also needs one of: yq (mikefarah), PyYAML, or Ruby
./migrate-v1-to-v2.sh --values /path/to/your-v1-values.yaml

# Skip prompts (still prints the plan)
./migrate-v1-to-v2.sh --values /path/to/your-v1-values.yaml --yes

# Preflight + generate values only
./migrate-v1-to-v2.sh --values /path/to/your-v1-values.yaml --dry-run

legacy retains the full v3 ingestion and read behaviour on the v4 release, which makes the image change an ordinary deploy:

LANGFUSE_MIGRATION_V4_WRITE_MODE=legacy
LANGFUSE_MIGRATION_V4_NATIVE_OTEL_BEHAVIOUR=dual_write
# Keep the v4 read paths (UI toggle, v2 APIs) off; the events tables
# are not written in legacy mode:
LANGFUSE_MIGRATION_V4_ALLOW_PREVIEW_OPT_IN=false
LANGFUSE_BACKGROUND_MIGRATION_V4_ENABLE_HISTORIC_BACKFILL=false

dual is where the migration actually happens, and the stated cost is plain: it doubles the write load and the storage cost. The configuration below keeps the v4 read paths off while you prove the write path, so the UI toggle and the v2 APIs stay hidden until you set LANGFUSE_MIGRATION_V4_ALLOW_PREVIEW_OPT_IN=true, which is worth doing only after a few days of healthy dual writing:

LANGFUSE_MIGRATION_V4_WRITE_MODE=dual
# Only if native OTel producers rely on server-side attribute propagation:
LANGFUSE_MIGRATION_V4_NATIVE_OTEL_BEHAVIOUR=dual_write
# Recommended until dual_write behaviour is confirmed to be working
# (see step 3):
LANGFUSE_BACKGROUND_MIGRATION_V4_ENABLE_HISTORIC_BACKFILL=false
# Optional: set false to keep the v3 read experience for all users while
# dual-writing (hides the toggle, disables the v2 APIs); defaults to true.
# Flip to true or remove overwrite, if you want to allow users to opt-in to v4 experience.
LANGFUSE_MIGRATION_V4_ALLOW_PREVIEW_OPT_IN=false

The OpenTelemetry setting decides who propagates the shared attributes. direct, the v4 default, writes every OTel span straight into the new tables and assumes the producer attaches userId, sessionId and the rest client-side. dual_write keeps the server doing that propagation, routing those spans through the staging pipeline unless they carry the x-langfuse-ingestion-version: 4 header. Which one is right depends on how stable your instrumentation's attribute keys are, and the gen_ai conventions moved enough in 2026 to be worth checking.

The cutover is the smallest diff here and the largest change in behaviour: remove the three overrides, or set them explicitly to the v4 defaults.

# Remove these overrides (or set them explicitly):
# LANGFUSE_MIGRATION_V4_WRITE_MODE=events_only
# LANGFUSE_MIGRATION_V4_NATIVE_OTEL_BEHAVIOUR=direct
# LANGFUSE_MIGRATION_V4_ALLOW_PREVIEW_OPT_IN=true

Two traps sit around those variables. Langfuse validates this configuration on startup and exits on invalid combinations, specifically events_only with dual_write OTel behaviour and events_only with LANGFUSE_MIGRATION_V4_ALLOW_PREVIEW_OPT_IN=false, so a half-removed override becomes a container that will not boot. And the names carry an uppercase V4 that is easy to lowercase in a templated values file, where a typo reads as unset and hands you the default, which is the cutover.

Neither legacy nor dual is a destination. Both will be removed in an upcoming major version, with no version number or date attached. The only dated commitment nearby is that Langfuse v3 receives security patches until the end of January 2027, which makes this a scheduled change rather than an optional one.

Those five producer classes are why the SDK inventory comes before the server upgrade: a rejected producer loses data outright at the cutover, a delayed one loses nothing and arrives late.

The mechanism behind the delay bounds your recovery window. Events from older SDKs go to traces and observations as before, and additionally into a staging table, observations_batch_staging, partitioned into three-minute windows. A worker job picks up completed partitions, joins them against traces to propagate the trace-level attributes, and inserts the result into events_full. Staging partitions are kept for 48 hours, so a propagation job that dies over a weekend is recoverable, provided you notice inside two days.

That is the argument for wiring one health check before you switch to dual:

# Worker container, port 3030. Returns 503 when event propagation is stuck.
curl -sf "http://<worker-host>:3030/api/health?failIfEventPropagationStuck=true"

# Threshold is configurable; default is 15 minutes:
# LANGFUSE_EVENT_PROPAGATION_STUCK_THRESHOLD_MINUTES=15

The check passes on deployments where the dual write does not run, so leave it configured after the cutover. This is a platform-health probe and nothing more; what to alert on inside the application is a separate set of signals.

The producer side can be done first. Python SDK 4.7.0 and JS/TS SDK 5.4.0 both shipped in late May 2026, roughly two months before Langfuse v4.0.0 arrived in July 2026, and both lines run against v3 servers. Two caveats: client.api.observations.* and client.api.metrics.* point at v2 endpoints a v3 server does not serve, so use client.api.legacy.* meanwhile, and trace input and output have to be set explicitly if you depart from the default root-span behaviour. Framework integrations are not enumerated, so canary one service rather than assuming blanket compatibility.

Build that inventory from the network, not from memory. If calls route through an LLM gateway, the version that matters is the gateway's: one upgrade there clears a dozen services or blocks them all.

Data ingested before the dual write became active exists only in the old tables. The automated backfill brings it across as a chain of background migrations on the worker, each step starting only after its predecessor finished, so a failure halts the chain instead of running on partial data.

The ordering rule is what costs data. Enable the backfill only after the dual write is active and confirmed healthy: it runs exactly once, so anything ingested after its cutoff but before the dual write started would be permanently missing from the new tables. That is why LANGFUSE_BACKGROUND_MIGRATION_V4_ENABLE_HISTORIC_BACKFILL=false belongs in the legacy configuration from the first deploy: on legacy the new tables are not written at all, so a backfill that fires there leaves a gap nothing later fills.

Step 3 carries a detail for anyone whose dashboards filter on trace metadata. Trace properties (name, user, session, version, release, tags, and the public and bookmarked flags) are propagated onto the backfilled child events, but trace metadata is deliberately not copied, to keep that join inexpensive. A saved view filtering on metadata needs rebuilding, not debugging.

Historic data appears gradually and newest first, so for a period only recent dual-written data is visible. There is no downtime and no impact on new ingestion, the migrations are resumable across restarts, and because the new tables use a ReplacingMergeTree, re-runs and overlaps with the dual write are de-duplicated with the most recent version winning.

Langfuse publishes no rate for this: the only stated commitment is that it can take from minutes to days depending on data volume. If a duration has to go into a change ticket, restore a ClickHouse snapshot into staging, run the backfill there and measure it.

Twenty-two endpoints across five of those seven groups return 404, and two more change behaviour without returning it. Replacements exist for all of them: the Observations API v2, the Scores API v3, the Metrics API v2, and the experiments API for dataset runs.

Two non-endpoint casualties land at the same moment. Trace-based and legacy-dataset-based LLM-as-a-judge evaluators look up the old tables, are marked Legacy in the UI, and stop running after the cutover, so recreate them as observation-level evaluators first. And the blob storage, PostHog and Mixpanel integrations read through a per-integration export source: the legacy traces and observations source (LEGACY_TRACES_OBSERVATIONS) stops producing data at the cutover, so switch each one to the enriched observations source before the cutover.

The consumer inventory is harder, because consumers are not in your dependency file. Grep the estate for /api/public/traces, /api/public/observations, /api/public/metrics and /api/public/scores across dashboards, scheduled jobs, notebooks and the ad hoc scripts people reach for when reading traces during an incident.

Rollback is where expectations break. A plain image swap does not work once the v4 schema migrations have applied: the v3 web container exits on startup with no migration found for version 46 while the worker container starts normally, so a half-green deployment reads as a web-container problem, not a schema problem. The rewind uses the migrate binary shipped inside the v4 web image:

# Run inside a v4 web container of the SAME version that is deployed,
# so the migration files match the recorded schema version:
cd /app/packages/shared

# Single-node ClickHouse (CLICKHOUSE_CLUSTER_ENABLED=false):
migrate -source file://clickhouse/migrations/unclustered \
  -database "${CLICKHOUSE_MIGRATION_URL}?username=${CLICKHOUSE_USER}&password=${CLICKHOUSE_PASSWORD}&database=${CLICKHOUSE_DB:-default}&x-multi-statement=true&x-migrations-table-engine=MergeTree" \
  goto 37

# Clustered ClickHouse:
migrate -source file://clickhouse/migrations/clustered \
  -database "${CLICKHOUSE_MIGRATION_URL}?username=${CLICKHOUSE_USER}&password=${CLICKHOUSE_PASSWORD}&database=${CLICKHOUSE_DB:-default}&x-multi-statement=true&x-cluster-name=${CLICKHOUSE_CLUSTER_NAME:-default}&x-migrations-table-engine=ReplicatedMergeTree" \
  goto 37

Version 37 is the schema version of the latest v3 releases. The rewind removes the events tables and recreates the ones v4 dropped. PostgreSQL needs no changes, because the latest v3 releases run against the v4 PostgreSQL schema, with the v4 backfill entries dormant unless LANGFUSE_BACKGROUND_MIGRATION_V4_ENABLE_HISTORIC_BACKFILL is true. Append &secure=true&skip_verify=true to the URL if your deployment sets CLICKHOUSE_MIGRATION_SSL=true, then deploy the latest v3 images.

Prefer rolling forward. While you run legacy or dual, every v3 surface still works on the v4 release, so staying on v4 in legacy mode while you debug retains the complete v3 behaviour. After the cutover that stops being true: new data lands only in the new tables, and a rollback to a v3 read path would miss everything written since. That is the point of commitment.

The cleanup afterwards is optional and irreversible. Set LANGFUSE_BACKGROUND_MIGRATION_V4_DROP_PID_TID_SORTING_TABLES=true once the backfill result is confirmed, then reclaim the old tables once every record you need is visible:

TRUNCATE TABLE traces;
TRUNCATE TABLE observations;

-- On clustered ClickHouse deployments; substitute your CLICKHOUSE_CLUSTER_NAME
-- for `default` if it differs:
TRUNCATE TABLE traces ON CLUSTER default;
TRUNCATE TABLE observations ON CLUSTER default;

TRUNCATE rather than DROP, because Langfuse expects the tables to exist. And not before you are sure.

Run the background-migrations query against the PostgreSQL behind the Langfuse v3 release you are already on. Any rows in that result are the longest lead time in this migration, because clearing them means getting onto the latest v3 release first.

Inventory both sides: producers against the Python 4.7.0 and JS/TS 5.4.0 floors and the v2 and v3 rejection lines, consumers by grepping for the deprecated endpoint paths rather than asking teams what they call.

Then decide whether your historic data comes across through the backfill or through a retention window. That determines whether you are buying 3x ClickHouse headroom or scheduling a longer dual-write phase, and it is cheaper to settle before the image changes. The rest of the platform decisions sit in our AI development tools pillar.