Validation & Uncertainty Quantification

Fairness

Data & digital tools: 

Assess bias and fairness of system e.g. using techniques described in section 8.2 of ISO 24027, to identify and mitigate potential unwanted bias prior to deployment. (Ref. ISO 24027).

Personnel Training:

Appropriate data and model bias developer training: sandbox environments used as safe space.​

Links:

ICH Q7: https://database.ich.org/sites/default/files/Q7%20Guideline.pdf

ICH Q8(R2): https://database.ich.org/sites/default/files/Q8%28R2%29%20Guideline.pdf

ICH Q9(R1): https://database.ich.org/sites/default/files/ICH_Q9%28R1%29_Guideline_Step4_2025_0115_0.pdf

ICH Q10: https://database.ich.org/sites/default/files/Q10%20Guideline.pdf

ICH Q11: https://database.ich.org/sites/default/files/Q11%20Guideline.pdf

ICH Q12: https://database.ich.org/sites/default/files/Q12_Guideline_Step4_2019_1119.pdf

ICH M15: https://database.ich.org/sites/default/files/ICH_M15_EWG_Step2_DraftGuideline_2024_1031.pdf

ISO/IEC 5259:2024-2025 (AI data quality management bundle): https://www.iso.org/publication/PUB200525.html

ISO/IEC TS 12791:2024 (Information technology — Artificial intelligence — Treatment of unwanted bias in classification and regression machine learning tasks): https://www.iso.org/standard/84110.html

ISO/IEC 17025:2017 (General requirements for the competence of testing and calibration laboratories): https://www.iso.org/ISO-IEC-17025-testing-and-calibration-laboratories.html

ISPE AI GAMP: Artificial Intelligence GUIDE: https://ispe.org/publications/guidance-documents/gamp-guide-artificial-intelligence

PIC/S Good Practices For Data Management And Integrity In Regulated GMP/GDP Environments: https://picscheme.org/docview/4234;

ASME VV40: https://www.asme.org/codes-standards/find-codes-standards/assessing-credibility-of-computational-modeling-through-verification-and-validation-application-to-medical-devices

FDA Artificial Intelligence in Drug Manufacturing: https://www.fda.gov/media/165743/download?attachment

FDA Using Artificial Intelligence & Machine Learning in the Development of Drug & Biological Products: https://www.fda.gov/media/167973/download

FDA Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions - Guidance for Industry and Food and Drug Administration Staff: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/assessing-credibility-computational-modeling-and-simulation-medical-device-submissions

FDA Data Integrity and Compliance With Drug CGMP Questions and Answers Guidance for Industry: https://www.fda.gov/media/119267/download

FDA Considerations for the Use of Artificial Intelligence To Support Regulatory Decision-Making for Drug and Biological Products:

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-use-artificial-intelligence-support-regulatory-decision-making-drug-and-biological

FDA Guidance on GxP data integrity: https://www.gov.uk/government/publications/guidance-on-gxp-data-integrity

EMA and FDA set common principles for AI in medicine development: https://www.ema.europa.eu/en/news/ema-fda-set-common-principles-ai-medicine-development-0

EMA Reflection paper on the use of Artificial Intelligence (AI) in the medicinal product lifecycle: https://www.ema.europa.eu/system/files/documents/scientific-guideline/reflection-paper-use-artificial-intelligence-ai-medicinal-product-lifecycle-en.pdf

MHRA ‘GXP’ Data Integrity Guidance and Definitions: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/687246/MHRA_GxP_data_integrity_guide_March_edited_Final.pdf

UK Gov Transparency for machine learning-enabled medical devices: guiding principles: https://www.gov.uk/government/publications/machine-learning-medical-devices-transparency-principles/transparency-for-machine-learning-enabled-medical-devices-guiding-principles