Under the July 2025 consultation draft of EU GMP Annex 22, generative AI and large language models should not be used in critical GMP applications, meaning anything with direct impact on patient safety, product quality or data integrity, and may be used in non-critical ones with a qualified person responsible for every output. The draft is not in force: as of November 2026 the EudraLex Volume 4 index lists no Annex 22, the consultation drew about 1,300 comments, and EMA's June 2026 network minutes put publication in early 2027. The text is also moving. EMA's report on the 30 June 2026 expert workshop says the drafting group has discussed widening the scope to dynamic, probabilistic and generative models, provided they meet the annex and sit under a fully documented, risk-based control strategy, while regulators pressed on hallucination, detectability and supplier evidence. Either way the evidence a qualified model needs barely changes: a signed intended use, an independent and access-controlled test set, logged confidence per decision, change control over frozen weights. A hosted endpoint makes several of those artefacts impossible to produce, which the workshop itself flagged. Start by inventorying every model that touches a GMP decision and sorting it with the three-impact test.
Under the draft EU GMP Annex 22, large language models should not be used in critical GMP applications, and may be used in non-critical ones if a qualified person is responsible for every output. That is the July 2025 consultation text, not a final rule: as of November 2026 no Annex 22 has been adopted, and EMA's own workshop report shows the drafting group discussing a risk-based route for generative AI even in critical use.
So the practical answer for a pharmaceutical manufacturer has two parts. Today nothing in EU GMP specifically bans an LLM, and the existing obligations (Annex 11, Chapter 4, your quality system) already require you to validate and own anything that touches a GMP decision. Tomorrow, whichever version of Annex 22 lands, the evidence a qualified model needs looks the same, and most of it is impossible to produce for a model you cannot freeze, hash and rerun.
This post covers what the draft says, where the revision is heading, how to classify an application, and the evidence set worth building now. Whether the same system is also regulated under the EU AI Act is a separate question from GMP, which the draft never mentions.
What the draft Annex 22 says about LLMs
The draft runs six pages, ten numbered sections plus a glossary. Its scope narrows in four steps before the LLM sentence arrives, and the sentence opens with "Following the above", so it is a consequence of those filters rather than a free-standing rule.
The fifth paragraph continues with the permission most summaries drop: in non-critical applications, "personnel with adequate qualification and training should always be responsible for ensuring that the outputs from such models are suitable for the intended use, i.e. a human-in-the-loop (HITL)".
Two details change how hard this reads. The modal verb throughout is should: 62 occurrences, with must and shall at zero, so a table that renders an Annex 22 clause as mandatory is the summariser talking. And the probabilistic filter catches systems nobody calls generative AI, which is why the exclusion is better read as bookkeeping than as a verdict on transformers.
| Model type | Critical application | Non-critical application | Deciding text |
|---|---|---|---|
| Static ML, deterministic output | In scope, annex applies in full | Principles optional | Scope paras 2 to 4 |
| Dynamic, learns during use | Should not be used | Not addressed | Scope para 3 |
| Probabilistic output, any architecture | Should not be used | Not addressed | Scope para 4 |
| Generative AI and LLMs | Should not be used | Permitted, qualified human responsible | Scope para 5 |
Is Annex 22 in force? Status as of November 2026
No. The European Commission ran the consultation from 7 July to 7 October 2025, alongside a revised Chapter 4 on Documentation and a revised Annex 11 on Computerised Systems. Check the primary index rather than a summary: the EudraLex Volume 4 page lists annexes up to 21, with no Annex 22, and Annex 11 still at its January 2011 revision. It does carry adoption dates where they exist, such as the revised Annex 19 applicable from 24 September 2026.
EU GMP annexes normally come with a transition period between publication and application, so the date your inspector can cite it is later again. Treat any adoption timetable that does not trace to one of the documents above as unsourced.
| Date | Event | Source |
|---|---|---|
| 7 Jul to 7 Oct 2025 | Public consultation on draft Annex 22, revised Annex 11 and Chapter 4 | European Commission consultation page |
| 4 Feb 2026 | About 1,300 comments received, annex under revision, final expected by end of 2026 | EMA/40804/2026, HMA-EMA AI group summary notes |
| 16 Jun 2026 | Annex 22 listed with workshop in July, publication early 2027 | EMA/143308/2026, Network Data Steering Group minutes |
| 30 Jun to 1 Jul 2026 | Multistakeholder workshop on Annex 22, report published in October | EMA/156789/2026 |
| November 2026 | No Annex 22 on the EudraLex Volume 4 index | EudraLex Volume 4 |
Where the revision is heading: the June 2026 workshop
EMA's report on the workshop is the most direct signal of what the final text may say. Its introduction states that consultation comments "indicated support for allowing manufacturers to use Large Language Models and generative AI systems in GMP applications, whether critical or non-critical", and that "the drafting group has therefore discussed the scope of Annex 22 to allow dynamic or adaptive models, probabilistic models, and Generative AI/LLMs in GMP applications, provided they comply with the requirements of the annex and are supported by a fully documented, robust, risk-based control strategy." It adds that concerns remain about what such a control strategy needs.
The industry position, presented jointly by the associations, was that technology labels should not decide acceptability: intended use, impact, uncertainty, complexity and control strategy should, under ICH Q9(R1) quality risk management. Regulators pushed back on three points that matter for anyone planning an LLM in a GMP workflow.
The report also records broad agreement that a valid risk-management outcome can still be "do not use", and that accountability stays with the regulated user even when a cloud provider supplies the model. Read together: a wider scope is plausible, but the price of entry for a generative model in a critical application would be a documented control strategy with layered prevention, detection and containment, plus evidence a hosted endpoint may not give you.
How to decide whether your application is critical
The test is the first scope sentence: direct impact on patient safety, product quality or data integrity. One hit is enough. Clause 3.1 says what the classification costs: an intended use with "a comprehensive characterisation of the data the model is intended to use as input and all common and rare variations", known limitations and erroneous inputs, approved by a process subject matter expert "before the start of acceptance testing". Clause 3.2 splits the input space into subgroups by decision output, site or equipment, material characteristics and defect type.
The trap is the SOP row: a drafted document signed by a human is non-critical only while the human actually reads it. Computer vision defect detection on the line inherits the whole evidence set. Predictive maintenance does not, unless its output feeds a product-quality decision.
| Application | Impact on | Class | Evidence that follows |
|---|---|---|---|
| Visual inspection accept or reject | Product quality, patient safety | Critical | Sections 3 to 10 in full, static model only under the draft |
| Parameter prediction driving a control action | Product quality | Critical | Same, plus change control over physical inputs |
| Deviation triage that closes records automatically | Data integrity | Critical | Non-critical only if a reviewer decides each case |
| Draft SOPs, translations, meeting notes | None once a qualified person owns them | Non-critical | Named operator, clause 10.5 review records |
| Generative model producing test labels | The qualification evidence itself | Not recommended, clause 5.6 | Use must be fully justified |
The determinism test an LLM cannot pass
Scope paragraph four is a claim about your serving stack, not about model architecture: identical inputs, identical outputs. A language model fails it in three ways, and only the first is sampling. Temperature 0 removes the sampler, not the variance: identical inputs still do not give identical outputs at temperature 0, because batching changes the floating-point arithmetic under a request.
The third failure is the one a validation lead should worry about. The glossary defines a static model as "A model where all parameters have been finally set, not allowing further adaption to new data." A provider-managed endpoint gives you a model id, not evidence that the parameters behind it were identical for the batch you qualified and the batch you released last week. You cannot hash it, diff it or stop it changing on the provider's calendar.
Passing would take weights you hold and can hash, a pinned runtime, a fixed batch shape, recorded seeds and a record of identical outputs across reruns. That is achievable for a small classifier on your own hardware. If the final text opens a risk-based route for generative models, the same reproducibility evidence becomes part of the control strategy rather than a scope filter, so it is worth building either way.
Test data independency: the clause a hosted model fails
Section 6 turns a hosted model from a procurement question into an evidence question. Clause 6.1 wants test data "not used during development, training or validation of the model". Clause 6.2 says staff involved in development and training "have never had access to the test data", that the test data sits behind access control and an audit trail, and that "There should be no copies of test data outside this repository." Clause 6.5 asks for staff separation or, where impossible, a 4-eyes check by someone without that access. Clause 6.4 adds that physical objects used for the final test should not have been used in training, which for vision systems means an inventoried, segregated set of vials or tablets.
The "nobody" rows are where a private perimeter changes the answer, for a reason unrelated to data residency. Annex 11 supplies the lever: its section 7 keeps the regulated user fully responsible for a vendor-supplied system and requires documentation to be accessible and explainable from your own facility, and the workshop report confirms that position applies to AI suppliers. For the critical-path classifier, keep the weights, the test repository and the inference runtime inside your perimeter and record the frozen model's hash in change control (cloud vs on-premise AI security and cost covers the general trade). For a permitted non-critical LLM, a self-hosted endpoint can log prompt, output, model hash and reviewer against the batch record, which is exactly what clause 10.5 asks for. Building that evidence layer on client infrastructure is the work our AI development team does for regulated manufacturers.
| Clause | What it asks for | Artefact for an inspector | Who can hold it when the model is hosted |
|---|---|---|---|
| 3.1 to 4.3 | Intended use, sample space, subgroups, criteria set before testing | Signed intended use and acceptance criteria | You |
| 5.1 to 5.6 | Stratified test set, verified labels, justified exclusions | Characterisation and label verification | You |
| 6.1 to 6.3 | No test data in training, access control, audit trail, no copies outside | Repository access log and audit trail | The supplier, if your test set was uploaded |
| 6.4, 6.5 | Final-test objects unused in training, staff separation or 4-eyes | Object inventory, role separation record | Nobody, if the supplier trained the model |
| 7 to 8.1 | Approved test plan, retained results, feature attribution | Plan, results, SHAP or LIME output | You, if you hold the weights |
| 9 to 10.5 | Confidence logged, change and configuration control, drift metrics | Confidence log, change records, configuration hash | Nobody, for the configuration hash |
What regulators have already done without an AI annex
FDA has already written AI into an enforcement document. Warning Letter 320-26-58, dated 2 April 2026 and issued by CDER after an October 2025 inspection, carries a section headed "Inappropriate Use of Artificial Intelligence in Pharmaceutical Manufacturing". It cites the firm under 21 CFR 211.22(c) for using AI agents to create specifications, procedures and master production or control records without adequate review, and under 21 CFR 211.100 for distributing without process validation, which the firm explained by saying the AI agent never told it validation was required. The remedy: AI output must be reviewed and cleared by an authorised person in the quality unit.
Keep it in proportion. The recipient was a small homeopathic manufacturer, and the same letter cites insanitary conditions and untested components. The AI heading is new, the severity is not. Mapped onto the draft annex, the first finding lands on clause 10.5 review records, the second on clause 3.1 and its demand that the intended use state what the tool is not qualified to tell you.
The US framework draws its line around a decision and its consequence. The EU draft drew it around a category of model, and the workshop suggests it may move toward the US shape.
| Text | Date | Status | Position on generative or probabilistic models |
|---|---|---|---|
| Draft EU GMP Annex 22 | Consultation Jul to Oct 2025 | Under revision, publication expected early 2027 | Excluded from critical use in the draft, risk-based route under discussion |
| FDA draft guidance, docket FDA-2024-D-4689 | January 2025 | Draft | No model-class exclusion, risk-based credibility assessment |
| EMA and FDA guiding principles of good AI practice | January 2026 | Non-binding principles | No model-class exclusion, validation proportionate to use |
| EMA reflection paper EMA/CHMP/CVMP/83833/2023 | September 2024 | Adopted reflection paper | Close human oversight, flags erroneous output from generative models |
The evidence record that survives either final text
Clause numbers will move; section names are less likely to. Key the evidence record on names. The template below is ours, not a schema any regulator publishes, and every field maps to a draft clause cited above.
model_release:
intended_use: # clause 3.1
task: "classify filled vial images as accept or reject"
input_sample_space: "line 3 camera, 4 formats, rare variations"
limitations: "no amber vials, no wet-label surfaces"
sme_approver: "process SME, before acceptance testing"
subgroups: # clause 3.2
decision_output: [accept, reject]
defect: [particulate, crack, fill_low]
acceptance_criteria: # clauses 4.1 to 4.3
metrics: [sensitivity, specificity, precision, f1] # per subgroup
baseline_process_performance: "measured manual inspection rate"
test_data: # clauses 5.1 to 5.6
selection: "stratified over all subgroups, rare variations included"
labelling_verification: "two independent experts, adjudicated"
generated_by_generative_ai: false
test_data_independency: # clauses 6.1 to 6.5
repository_access_control: true
audit_trail: true
copies_outside_repository: none
physical_objects_separated: true
staff_separation_or_4_eyes: "reviewer had no test data access"
explainability: {method: shap, feature_review_approved_by: "QA"} # clause 8
confidence: # clause 9
score_logged_per_decision: true
undecided_band: "below threshold, route to manual"
operation: # clause 10
change_control_id: "CC-2026-0431"
configuration_hash: "sha256 of frozen weights and runtime image"
input_drift_metrics: defined
human_review_records: retainedIf the final annex opens a route for generative models, the same record grows a control-strategy block (input checks, output constraints, escalation and fallback), which is what the workshop discussed as layered prevention, detection and containment. The base record does not shrink. The discipline of building a hold-out set helps with section 5, and monitoring drift in the input data covers clause 10. For the approval point itself, see designing fallback and human escalation.
One drafting detail to re-read when the final text lands: clause 4.3 cross-references "Annex 11 2.7" for the no-decrease principle, but in the co-published Annex 11 draft 2.7 is Security and the risk clause is 2.8. That is why the record keys on names.
Five moves are safe whatever the final wording says:
The decision worth making this week is smaller still. For each model on your critical list, can you freeze it, hash it and reproduce an output twice? If not, because inference runs behind someone else's endpoint, that is the gap to close first. For the wider strategic frame, see our AI for business pillar.
FAQ
Quick answers to the questions this post tends to raise.



