Performance Claims and Ongoing Monitoring for AI Software

Written by HATEM RABEH, MD, MSc Ing

Your Clinical Evaluation Expert And Partner

in
S

Performance claims are the backbone of your clinical evaluation. They define what you are proving. For AI software, claims must be specific enough to verify, realistic enough to achieve, and robust enough to survive post-market reality.

Writing Claims That Reviewers Accept

Every claim should contain these elements:

• Who uses it – the intended user population
• In what setting – the clinical environment
• For which population – the patient population
• For what decision – the clinical use case
• With what metric – the measurable endpoint
• At what threshold – the acceptance criterion

Strong claims use precise language. Instead of the software detects disease accurately, write: In radiologists using the software for screening mammography, the software achieves sensitivity of at least 90% for detecting malignant lesions in women aged 40-74, on external validation data with defined case mix.

Claim Categories for AI Software

Clinical Association Claims. The model output is associated with the targeted condition as defined by the accepted reference standard. Primary endpoint is area under the ROC curve meeting target and floor from the CEP.

Analytical Performance Claims. The software processes input data and produces outputs within specified limits for accuracy and repeatability. Time to result is within workflow limits. Performance under degraded conditions remains within tolerance.

Clinical Performance Claims. In the intended users and setting, the device achieves sensitivity and specificity at or above targets on external validation data. Subgroup performance meets floors for predefined groups.

Safety and Oversight Claims. Outputs are accompanied by information needed for user understanding. Human oversight procedures prevent or minimize risks. Post-market monitoring detects drift and new risks.

Mapping Claims to Evidence

For every claim, your CER must include:

1. GSPR item from Annex I that the claim supports
2. Acceptance criteria from CEP including target and floor
3. Datasets and studies that test the claim
4. Risk controls that depend on the claim

This creates traceability. Reviewers can follow any claim from requirement through evidence to conclusion.

Ongoing Monitoring Requirements

AI software can drift. Performance validated pre-market may degrade as patient populations change, clinical practice evolves, or data quality varies. MDR and MDCG require post-market monitoring that addresses these risks.

Performance Logging. Maintain records of algorithm inputs, outputs, and outcomes where available. This data feeds drift detection.

Drift Monitoring. Define metrics and thresholds for detecting performance degradation. When sensitivity drops below 85% on rolling monitoring data, trigger investigation.

Update Validation. When the algorithm is updated, revalidate against acceptance criteria. Document that new versions maintain or improve performance.

PMCF Integration. Connect post-market clinical follow-up objectives to specific gaps and uncertainties from your clinical evaluation. Each gap should map to a monitoring activity.

Closing the Loop

Clinical evaluation for AI software is not a one-time exercise. It is a continuous process. Your CER represents your understanding at a point in time. Post-market activities either confirm that understanding or reveal where it needs updating.

Build this lifecycle thinking into your clinical evaluation from the start. Identify what you know, what you do not know, and how you will learn. This demonstrates maturity that reviewers recognize and respect.

This concludes our series on AI Medical Device Clinical Evaluation. The principles are straightforward: understand the three-pillar framework, plan systematically, set defensible criteria, appraise evidence rigorously, structure your CER clearly, and monitor continuously. Do these consistently, and your AI software CERs will pass review.

Peace,
Hatem
Your Clinical Evaluation Partner