Model cards that hold up in a regulatory review.

Intended use & indications

Anchor every downstream claim. Reviewers compare every metric and limitation against the words you put here.

• Clinical task in plain language
• Target patient population (age, sex, comorbidities)
• Care setting and user profile
• Explicit out-of-scope uses & contraindications

Task

Triage of non-contrast head CT for suspected intracranial haemorrhage

Population

Adults ≥18 presenting to ED with suspected acute stroke

User

Board-certified radiologist; not for use without expert review

Out of scope

Paediatric, post-operative, or contrast-enhanced studies

Training data provenance

Reviewers want to know who is represented, who isn't, and why. Gaps disclosed up front are mitigations; gaps discovered later are findings.

• Sources, institutions, geographies, years
• Sample size + class balance
• Demographic + device distribution
• Inclusion / exclusion criteria
• Labelling protocol + inter-rater agreement
• Known representational gaps

Sources

4 US academic centres + 1 EU teaching hospital, 2018–2023

N

12,840 studies; 18.4% positive for ICH

Scanners

GE, Siemens, Canon, 64-slice and above

Gap

Under-representation of patients <30 and non-contrast scanners <16-slice

Performance, overall and by subgroup

Headline metrics are not enough. Stratified results are now an explicit expectation in FDA, MHRA and Health Canada review.

• Independent test set definition (site / patient / time split)
• Primary metrics with 95% CIs
• Subgroup table: sex, age band, race/ethnicity, device, site
• Calibration plot or ECE
• External validation cohort

Sensitivity

94.1% (92.7–95.3)

Specificity

89.6% (88.4–90.7)

Sens. (female ≥65)

91.0% (87.8–93.6), flagged for monitoring

External cohort

n=2,104, EU site, sens. 92.4% / spec. 88.1%

Limitations & failure modes

Documenting where the model breaks is a transparency obligation under both the FDA guiding principles and EU AI Act Article 13.

• Documented failure modes
• Populations / settings where performance is degraded
• Open bugs and CAPAs disclosed to deployers
• Behaviour on out-of-distribution inputs

Failure mode

Motion-degraded studies → confidence < 0.6 returned with abstention

Degraded setting

Non-contrast scanners <16-slice not validated; device blocks inference

OOD

Paediatric input → input-validation error, no score returned

Cybersecurity posture

AI inherits every classical software threat and adds adversarial inputs, prompt injection, and model extraction. Both FDA and EU MDR Annex I §17 expect explicit handling.

• SBOM including model weights + inference runtime
• Threat model + risk register reference
• Adversarial-robustness testing for the modality in scope
• Coordinated vulnerability disclosure contact

SBOM

SPDX 2.3 generated per release; weights pinned by SHA-256

Adversarial

FGSM + PGD evaluated; degradation < 3% at ε=2/255

CVD

security@vendor.example, 90-day disclosure window

Lifecycle & change control (PCCP)

FDA's August 2025 final PCCP guidance defines what can change without a new submission. Anything not in the PCCP triggers a new authorisation.

• PCCP on file? Reference document ID
• Modification protocol: what can change, how, with what limits
• Performance monitoring metrics + alert thresholds
• Drift detection method
• Rollback path

PCCP scope

Re-training on +20% data per quarter; threshold tuning ±0.05

Monitoring

Weekly sensitivity by site; alert at −2σ from baseline

Rollback

Previous model retained for 24 months; rollback ≤ 4h