Pharma AI glossary
Explainability (XAI)
Techniques that expose *why* a model produced an output: SHAP and LIME attribute a prediction to input features; heat maps show which image regions drove a vision model's call.
Definition
Techniques that expose *why* a model produced an output: SHAP and LIME attribute a prediction to input features; heat maps show which image regions drove a vision model's call. Regulators want explainability not for its own sake but to make human challenge possible: an operator shown *which* region of the vial triggered "defect" can meaningfully agree or disagree. Annex 22 names these techniques explicitly and expects feature-attribution review before test approval; FDA is more outcome-focused but expects sponsors to justify model logic appropriate to risk.
Why regulators care
Black-box models dominate deep learning; explainability methods (SHAP, LIME, saliency maps) show which inputs drove an output. FDA often accepts strong performance evidence without full interpretability if risk matches; EU Annex 22 draft pushes explainability artifacts for critical GMP models.
Practical literacy
Teams should know explainability is not a substitute for performance validation, can be gamed or unstable, and must be stored if required, as another record type with retention and review. The question in audit is whether a human can challenge the output, not whether the chart looks plausible.
Frequently asked questions
Do we need SHAP for every model?
Only where rulebooks or your risk assessment require it, typically higher-impact manufacturing models such as visual inspection or process monitoring, not every Excel macro labeled AI.
Is explainability the same as interpretability?
Colloquially merged; in practice both aim at understandable decisions, but regulatory packages expect defined methods, limitations, and human review of explanations.
Related terms
Benchmark your AI literacy
Free 20-question assessment, eight modules, and instant scoring, built for pharma quality and manufacturing teams.
Open the Literacy CenterBrowse all terms
See the full 71-term glossary index or search interactively in the Literacy Center.
Back to glossary hub →Educational content only, not legal or regulatory advice. Regulatory guidance cited here includes drafts (FDA AI credibility guidance; EU GMP Annex 22) as of August 2026; verify against final texts before relying on them in submissions. Company-reported figures (Merck CSR timings, Sanofi results) are labeled where used. MIT's ~95% pilot figure carries its own caveat (~150 interviews, contested definitions, not peer-reviewed). Re-check sources on module finalization.