Multilingual RAG Pipeline: One Index or One Per Language

All three return k results at normal latency with the wrong documents. From the log they are one bug. They are three.

The query language differs from the document language. A claims handler asks in German about a clause that exists only in French. This is what the LAReQA benchmark isolates: strong alignment, defined there as requiring semantically related cross-language pairs to be closer in representation space than unrelated same-language pairs. Without it, an unrelated German passage outranks the correct French one.

One index holds several languages at once. A study published in April 2026 on multilingual RAG reports that current systems suffer from a language bias during reranking, systematically favouring English and the query's native language, and suppress answer-critical documents in other languages.

A single document code-switches inside a page. A bilingual contract, or a German memo with a French annex. A document-level tag is right for part of the file and wrong for the rest, and the wrong half gets the wrong analyzer and token accounting.

The default is one index with the language stored as a chunk-level payload field, right for two to five languages. Filter on that field when the query language is known, drop it when it is not.

A language predicate is the friendly case for filtered vector search: a German query is asking about German documents, so the filter correlates with the query rather than cutting across it, which is what separates a harmless filter from one that guts recall on an HNSW graph. The shared-versus-split question underneath is the tenant argument in the silo, pool and bridge tradeoff, with language substituted for tenant.

The tenant analogy stops at the encoder. LAReQA found that the baseline best on its strong-alignment task falls short of competing baselines on zero-shot variants targeting only weak alignment, so a model chosen on bitext-style scores has not been shown to rank a cross-language answer above a same-language distractor.

A shared index does not force one analyzer on the sparse leg. Qdrant attaches a text index per field:

PUT /collections/policies/index
{
  "field_name": "body_de",
  "field_schema": {
    "type": "text",
    "tokenizer": "multilingual",
    "lowercase": true,
    "min_token_len": 2,
    "max_token_len": 30,
    "stopwords": { "languages": ["german"] },
    "stemmer": { "type": "snowball", "language": "german" }
  }
}

Swap german for polish in both the stopwords and the stemmer and the stemmer language does not exist. That gap is what pushes a shared index into a split one.

Splitting by language buys per-language analyzers with nothing to configure and evaluation slices for free, because the index boundary is already the slice boundary. The price is a language-detection step in front of every query, and it is the weakest component in the design.

Detection at ingest is easy, because you have a whole document. fastText publishes two language-identification models, lid.176.bin at 126MB and lid.176.ftz at 917kB, both recognising 176 languages under a Creative Commons Attribution-Share-Alike 3.0 licence. Elasticsearch ships lang_ident_model_1, called from an ingest pipeline:

PUT _ingest/pipeline/detect-language
{
  "processors": [
    {
      "inference": {
        "model_id": "lang_ident_model_1",
        "inference_config": {
          "classification": { "num_top_classes": 5 }
        },
        "field_map": {}
      }
    }
  ]
}

num_top_classes at 5 is the part worth keeping: one label says nothing about confidence, while a five-class distribution shows when a page is mixed, because the runner-up then sits close to the leader.

Query time is a different problem behind the same API. A character-ngram classifier looking at a two-word query has very little signal, and the failure is silent: a wrong route sends the query to an index holding nothing relevant, and the pipeline returns k results at normal latency with no error. Fan-out plus a fused rank restores recall, and gives up the routing saving the split was supposed to buy.

Translating everything into a pivot language is the cheapest way to reuse an English-tuned stack, and the one layout that damages something you cannot repair downstream.

Retrieval quality points the same way. A November 2025 evaluation of cross-lingual ranking tested four intervention types across three benchmark datasets and concluded that dense retrieval models trained for cross-language information retrieval consistently outperform lexical matching and derive little benefit from document translation. Machine translation has a documented place upstream of retrieval: LAReQA found that augmenting training data via machine translation improved significantly over using mBERT out of the box. Translating training pairs is not translating a corpus.

The citation cost is mechanical rather than measured, and it ends the argument in a regulated deployment. If you index translations, the span the generator quotes lives in a derivative nobody signed, and a regulator asking for the clause behind a decision cannot be handed one. Each chunk then carries two artefacts, the text it ranked on and the original span it must display, and the audit trail records which translation model version produced the derivative. That is citing sources correctly, one level harder.

The fusion machinery, normalisation, weighting and reciprocal rank fusion, is covered in combining dense and sparse retrieval. This section is about the input to the sparse leg, where language decides whether that leg contributes at all.

"invertedIndexConfig": {
  "bm25": { "k1": 1.2, "b": 0.75 },
  "stopwords": {
    "preset": "none",
    "additions": ["der", "die", "das", "und", "oder", "nicht"],
    "removals": []
  }
}

PUT /policies-de
{
  "settings": {
    "analysis": {
      "filter": {
        "de_stop": { "type": "stop", "stopwords": "_german_" },
        "de_stemmer": { "type": "stemmer", "language": "light_german" },
        "de_decompound": {
          "type": "hyphenation_decompounder",
          "hyphenation_patterns_path": "analysis/de_hyph.xml",
          "word_list_path": "analysis/de_word_list.txt",
          "min_subword_size": 4,
          "only_longest_match": true
        }
      },
      "analyzer": {
        "policy_de": {
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "de_decompound",
            "de_stop",
            "german_normalization",
            "de_stemmer"
          ]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "body":     { "type": "text", "analyzer": "policy_de" },
      "language": { "type": "keyword" }
    }
  }
}

Nine rows, and Polish carries the argument: Elasticsearch needs a plugin, PostgreSQL has no dictionary, Qdrant has no stemmer. If Polish, Japanese, Korean or Chinese is in your corpus, that language's sparse leg is a platform decision, not a tuning task.

intfloat/multilingual-e5-large is an XLMRobertaModel with max_position_embeddings 514 and vocab_size 250002, and its tokenizer config sets model_max_length to 512. The card states that long texts are truncated to at most 512 tokens, and that every input needs an English query: or passage: prefix even for non-English text. BAAI/bge-m3 is the same architecture on the same vocabulary, with 8,192 usable input tokens; specs are in the embedding model dimension table.

A chunking rule written in characters or words does not produce equal token counts across languages, because the subword vocabulary was not built to cover them equally. The same clause in German, French and Italian yields three different counts from one rule, and the longest is the one silently truncated. Swapping models does not fix it: bge-m3 raises the ceiling but shares the vocabulary. Do not take a multiplier off a blog post, this one included. Measure it with the retriever's own tokenizer:

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("intfloat/multilingual-e5-large")

# same clause, three language versions, from your own corpus
samples = {
    "de": open("clause_4_2.de.txt").read(),
    "fr": open("clause_4_2.fr.txt").read(),
    "it": open("clause_4_2.it.txt").read(),
}

for lang, text in samples.items():
    ids = tok(f"passage: {text}", add_special_tokens=True)["input_ids"]
    print(lang, len(text), len(ids), len(ids) > 512)

Set chunk size from the worst language, not the average. Where a language has no stemmer, one model property helps: bge-m3 returns dense, multi-vector and sparse representations in one pass, through BGEM3FlagModel with model.encode(..., return_dense=True, return_sparse=True, return_colbert_vecs=True), so Polish gets a sparse leg that does not wait on a Snowball dictionary.

An averaged retrieval number hides a language that has collapsed. Four languages at recall 0.82 and a fifth at 0.31 still average 0.72. Report recall and nDCG per language, against an evaluation set drawn from your own corpus.

Public benchmarks cannot stand in for that set. MIRACL is an 18-language challenge from the WSDM 2023 Cup with around 77,000 queries and over 700,000 relevance judgments, all assessed by native speakers hired by the project. Corpora are public for 16 of the languages, the other two are held back, and German is not among the 16. Those Wikipedia-derived corpora range from 131,924 passages for Swahili to 32,893,221 for English, roughly a 249-fold spread, so an unweighted average follows the biggest language.

MMTEB, published in February 2025, is broader: over 500 quality-controlled evaluation tasks across more than 250 languages, assembled by 85 authors. Its leaderboard result is a snapshot of that date, and the paper named a 560-million-parameter instruct checkpoint as the best-performing publicly available model at publication. The board has moved since, and how the multilingual leaderboard has moved covers where it stands now. Shortlist with these benchmarks, decide with your own set: Wikipedia prose is not policy prose.

Three slices belong in that set from the first week: monolingual retrieval in each language, cross-language retrieval where query and answer languages differ, and code-switched documents if you have any. Those slices are the three failure shapes at the top of this post, turned into measurements.

The on-prem constraint does not decorate this topic. It deletes an option. In a private perimeter, Option C is the one layout you cannot buy as a service: sending a French policy clause or a German claims note to a hosted translation endpoint is a cross-border transfer of the exact text under review, and that text is what a regulator or a policyholder will later ask you to produce. Running the translation model inside the perimeter removes the transfer problem and leaves the citation problem exactly as it was.

The per-language sparse gap is worse inside a perimeter, because the fix for a missing language is a plugin install, and on an air-gapped cluster installing analysis-stempel or analysis-kuromoji is a change-controlled event rather than a command someone runs on a Thursday. The plugin set is part of the cluster build artifact and part of upgrade testing.

Three things to do this week, in order. Run the tokenizer script above over the same clause in each of your languages and check whether your chunk size truncates the longest one. List what your engine ships for every language in the corpus, with \dF in psql or the equivalent check on your Elasticsearch nodes, and write down which languages have no stemmer. Then split your retrieval evaluation by language and read the worst row instead of the mean. If that row is a language your engine has no analyzer for, you have found the bug, and it was never the embedding model. The rest of the stack sits across our RAG systems work.