This is the final Foundations module. It does not introduce new concepts. Instead, it integrates everything from Modules 1-7 into a single scenario that tests whether you can apply the full toolkit: data quality (M1-M2), model training (M3-M4), evaluation (M5), architecture selection (M6), and responsible AI (M7). After this module, you are ready for the Applied stage, beginning with transformers and attention.
By the end of this module you will be able to:
Evaluate an end-to-end AI system by applying concepts from all seven preceding modules
Identify data quality, evaluation, architecture, and fairness risks in a healthcare AI deployment
Construct a deployment checklist that addresses technical, ethical, and operational requirements
This scenario is composite but realistic. Early warning scores have been used in UK hospitals since 2012 (the National Early Warning Score, NEWS). AI-augmented versions are now in active trials. The questions you need to ask draw on every module so far. Work through each section as though you are writing the evaluation report.
With the learning outcomes established, this module begins by examining data quality: what was the model trained on? in depth.
8.1 Data quality: what was the model trained on?
The first set of questions concerns the training data. Drawing on Modules 1 and 2 (what data is, data representation):
Population match. Was the model trained on a population similar to the trust's patients? A model trained on a tertiary referral centre (sicker patients, more complex conditions) may not transfer to a district general hospital. A model trained in the US may not generalise to UK populations with different demographics, comorbidity profiles, and healthcare pathways.
Feature availability. Does the trust collect the same vital signs and lab results at the same frequency as the training data? If the model expects continuous pulse oximetry but the trust takes spot readings every 4 hours, the input distribution will differ from training.
Label quality. How was "deterioration" defined in the training data? Was it transfer to ICU, activation of a rapid response team, or death within 24 hours? Different definitions produce different models with different performance characteristics.
Missing data. Hospital data is notoriously incomplete. How did the vendor handle missing vitals? Imputation methods matter: mean imputation can mask the very signals the model needs to detect.
Common misconception
“If the vendor's validation metrics are good, the model will work in our hospital.”
Validation metrics are only valid for the population and data collection process they were measured on. A model with AUC 0.89 in its development setting may perform at 0.72 in your hospital if the patient population, monitoring frequency, or clinical workflows differ. This is called dataset shift, and it is the single most common reason AI systems fail in new clinical environments. The only way to know is to evaluate the model on your own data.
With an understanding of data quality: what was the model trained on? in place, the discussion can now turn to evaluation: are the reported metrics trustworthy?, which builds directly on these foundations.
8.2 Evaluation: are the reported metrics trustworthy?
The vendor reports AUC 0.89 and F1 0.78. Drawing on Module 5 (evaluating AI):
AUC is a threshold-independent metric. It summarises performance across all possible decision thresholds. But the trust needs to operate at a specific threshold: how sensitive should the system be? What false positive rate is clinically acceptable? AUC does not answer these questions. You need the full precision-recall curve and the receiver operating characteristic (ROC) curve, not just a single number.
Class imbalance. Deterioration events are rare (typically 2-5% of hospital admissions). On imbalanced data, high AUC can coexist with clinically unacceptable precision. If the model fires 100 alerts per day and only 5 are genuine, staff will stop responding. You need the positive predictive value (PPV) at the operating threshold.
Temporal validation. Was the model validated on data from the same time period as training (random split) or from a later period (temporal split)? Random splits inflate performance because they leak temporal patterns. Temporal validation is the only acceptable approach for time-series clinical data.
Cross-validation. Was performance reported from a single split or from k-fold cross-validation with confidence intervals? A single split gives you a point estimate with no measure of reliability.
“The move from algorithm development to clinical implementation requires a fundamental shift in evaluation approach: from retrospective validation on curated datasets to prospective evaluation in real clinical workflows.”
Kelly, C.J. et al., 'Key challenges for delivering clinical impact with artificial intelligence', BMC Medicine (2019) - Section 3, Evaluation challenges
This paper from DeepMind researchers argues that the evaluation standards used in ML benchmarks are insufficient for clinical deployment. Retrospective metrics do not predict real-world performance because they cannot capture the effects of clinical workflow integration, alert fatigue, and dataset shift.
With an understanding of evaluation: are the reported metrics trustworthy? in place, the discussion can now turn to architecture: is the model design appropriate?, which builds directly on these foundations.
Clinical AI must work within existing workflows. A model with excellent offline metrics can fail in practice if it generates too many false alarms, requires data that is not routinely collected, or does not integrate with the hospital's electronic health record system.
8.3 Architecture: is the model design appropriate?
Drawing on Module 6 (deep learning architectures):
Temporal data requires temporal architectures. Vital signs are time series. An architecture that treats each measurement independently (e.g. a simple feedforward network on the latest vital signs) discards temporal trends that are often the earliest signal of deterioration. LSTMs, temporal convolutional networks (TCNs), or transformers are more appropriate because they model how vital signs evolve over time.
Irregularly sampled data. Hospital vital signs are not measured at fixed intervals. A patient in a general ward might have observations every 4 hours; a patient causing concern might have them every 30 minutes. The architecture must handle irregular time intervals, either through explicit time-aware mechanisms or through appropriate preprocessing.
Interpretability versus performance. A gradient-boosted tree model may sacrifice 1-2 points of AUC compared to a deep neural network but produce feature importance rankings that clinicians can understand and trust. In healthcare, interpretability is not optional: clinicians need to know why the system flagged a patient to decide whether to act.
With an understanding of architecture: is the model design appropriate? in place, the discussion can now turn to fairness and accountability: who is affected?, which builds directly on these foundations.
8.4 Fairness and accountability: who is affected?
Drawing on Module 7 (responsible AI basics):
Disaggregated performance. Does the model perform equally well across age groups, ethnicities, and sexes? Elderly patients, who are most likely to deteriorate, may also be the group most poorly represented in training data. A model that performs well on average but poorly for patients over 80 is actively dangerous.
Alert equity. If the model's false positive rate differs by demographic group, some patients receive unnecessary interventions while others receive insufficient attention. This is the equalized odds problem from Module 7 applied to clinical care.
Model card. Has the vendor published a model card? Does it include disaggregated metrics, intended use constraints, and known limitations? If not, the trust is deploying a system without the minimum documentation needed for accountability.
Override policy. Can clinicians override the model? What happens when they do? An AI system that generates alerts but provides no mechanism for clinical judgment to prevail is unsafe. Conversely, if clinicians override every alert, the system adds cost without value.
“An AI/ML-based Software as a Medical Device should include considerations for the identification and mitigation of biases in the training dataset.”
FDA, 'Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan' (2021) - Section II, Total Product Lifecycle Approach
The US FDA has recognised that bias in medical AI is a patient safety issue. Their regulatory framework requires pre-market evaluation of bias and post-market monitoring for performance degradation. The UK MHRA has adopted a similar position. These are not guidelines; they are regulatory requirements.
Common misconception
“If the model helps some patients, it is worth deploying even if it is imperfect.”
A model that improves outcomes for one group while worsening them for another is not a net positive. If the deterioration prediction system has a 40% false positive rate for elderly patients, the resulting alert fatigue may cause staff to ignore genuine warnings for that population. The harm is not hypothetical: alert fatigue is a documented cause of patient safety incidents. Deployment decisions must consider disaggregated performance, not just average improvement.
Loading interactive component...
Loading interactive component...
Loading interactive component...
Key takeaways
AI deployment evaluation is not a single-dimension assessment. It requires simultaneously evaluating data quality (population match, feature availability, label definitions), model metrics (at the operating threshold, with confidence intervals, on local data), architecture appropriateness, and fairness across all affected subgroups.
Vendor-reported metrics are necessary but not sufficient. They were measured on a specific population, with a specific data collection process, at a specific point in time. Local validation on your own data is the minimum standard before deployment in any high-stakes setting.
In healthcare, false positives cause alert fatigue and false negatives cause missed deterioration. The acceptable balance depends on clinical context, patient population, and nursing workflow. A single AUC number cannot capture these trade-offs.
Model cards, disaggregated metrics, interpretability mechanisms, and clinical override policies are not optional extras. They are the minimum infrastructure for accountable AI deployment. If the vendor cannot provide them, the system is not ready.
Continuous post-deployment monitoring is essential because performance degrades over time as patient populations, clinical practices, and data collection processes change. A model that is fair and accurate at launch may become neither within months.
Written by DeepMind Health researchers. Identifies the gap between retrospective ML benchmarks and real clinical deployment. Establishes that evaluation must extend beyond AUC to include workflow integration, prospective validation, and monitoring.
The US FDA regulatory framework for clinical AI. Requires pre-market bias evaluation, a predetermined change control plan, and real-world performance monitoring. The UK MHRA has adopted a compatible approach. Used for the regulatory context in Section 8.4.
Google Health study demonstrating both the potential and challenges of EHR-based predictive models. Achieved strong results on mortality, readmission, and length-of-stay prediction while highlighting data quality, missing data, and temporal validation challenges.
Practical guide for NHS organisations evaluating AI systems. Provides a checklist-based framework covering clinical evidence, technical validation, usability, data governance, and monitoring. Directly applicable to the scenario in this module.
The clinical scoring system that the AI vendor's product would supplement or replace. NEWS2 is the current standard for detecting patient deterioration in UK hospitals. Understanding the baseline system is essential for evaluating whether AI adds genuine value.
The Foundations stage is complete. You can now reason about data quality, train and evaluate models, choose appropriate architectures, and assess fairness and accountability. The Applied stage begins with the architecture that changed everything after 2017: transformers and the attention mechanism. Transformers replaced both CNNs and RNNs/LSTMs for many tasks and are the foundation of every large language model you interact with today. The concepts from Foundations, particularly evaluation and responsible AI, remain essential: transformers amplify both capability and risk.
