Why Your Literature Scores Keep Getting Challenged by Reviewers

Written by HATEM RABEH, MD, MSc Ing

Your Clinical Evaluation Expert And Partner

in
S

This happens more often than it should. Manufacturers appraise literature. They extract data. They summarize findings. But when the Notified Body reviews the CER, the question comes back: on what basis did you assign evidence weight?

The problem is not the effort. It is the structure. Clinical evaluation under the MDR requires transparent, methodical appraisal of evidence quality. This is not optional, and it is not about opinion. It is about demonstrating that your conclusions rest on solid ground.

When evidence levels are unclear or inconsistently applied, your entire clinical evaluation weakens. Reviewers see it immediately.

What the MDR Actually Requires

Article 61 of MDR 2017/745 mandates that clinical evaluation follow a defined and methodologically sound procedure. Annex XIV Part A lays out that appraisal of clinical data must be systematic, objective, and rationally justified.

MDCG 2020-6 on sufficient clinical evidence further reinforces that the evaluation must consider the quality and relevance of the data. You cannot treat all studies equally. A case report does not carry the same weight as a randomized controlled trial. This is not about preference. It is about evidence hierarchy.

But here is what I see in real submissions: manufacturers acknowledge the concept of evidence levels, then fail to apply it consistently. The appraisal becomes a checkbox exercise instead of a structured analysis.

Evidence Hierarchies Are Not Arbitrary

Evidence-based medicine uses hierarchies for a reason. Stronger study designs reduce bias, control confounding, and produce more reliable conclusions. Clinical evaluation for medical devices follows the same logic.

At the top of the hierarchy: systematic reviews of randomized controlled trials, then individual RCTs with adequate power and rigorous methodology. Below that: cohort studies, case-control studies, case series, case reports, expert opinion.

This is not about dismissing lower-level evidence. In many device categories, RCTs are rare or impractical. But that does not eliminate the need to classify evidence systematically. It means you must be transparent about what level of evidence you are relying on, and why it is appropriate for your device and intended purpose.

The issue is when manufacturers blend evidence levels without distinction. A single case report is cited alongside a multi-center cohort study, and both are treated as if they carry equal weight. Reviewers notice this. They question whether the conclusions are actually supported.

How Evidence Levels Should Shape Your Appraisal

Every piece of literature in your CER should be appraised for both relevance and quality. Relevance asks: does this study apply to my device, my population, my intended use? Quality asks: how reliable is this evidence?

Evidence level is a core component of quality. It reflects study design, risk of bias, sample size, follow-up duration, and methodological rigor. When you appraise a study, you are not just summarizing its findings. You are assessing whether those findings are trustworthy enough to support your clinical evaluation.

This means your appraisal should explicitly state the evidence level for each study. Use a recognized framework. Many manufacturers apply the Oxford Centre for Evidence-Based Medicine levels, or adapt GRADE methodology. The specific framework matters less than consistency and transparency.

What Happens When Evidence Levels Are Missing

Reviewers read clinical evaluations critically. They look for logical consistency. If your document claims strong clinical evidence but relies heavily on case reports and expert opinion, the gap is obvious.

When evidence levels are not clearly assigned, reviewers cannot verify your reasoning. They cannot see whether your conclusions are proportional to the strength of the data. This creates doubt, and doubt leads to additional questions, clarifications, and delays.

I have seen CERs returned because the literature appraisal lacked a structured quality assessment. The manufacturer had extracted data, summarized outcomes, and written a clinical evaluation narrative. But when asked how they weighted evidence, there was no clear answer.

The Notified Body did not reject the literature itself. They rejected the appraisal process. The studies may have been appropriate, but the evaluation did not demonstrate systematic quality assessment.

How to Appraise Evidence Levels in Practice

Start by selecting a recognized evidence hierarchy framework. Document this in your clinical evaluation plan. The framework should align with the type of clinical data you expect to find for your device category.

When you appraise each study, assign an evidence level based on study design and methodological quality. Document this in your literature table. Do not leave it implicit. A column for evidence level should be visible, consistent, and justified.

Then, in your appraisal narrative, explain how evidence levels influenced your conclusions. If you are relying on lower-level evidence, justify why. Explain what higher-level evidence exists or does not exist, and why the available data is still sufficient.

If you have a mix of evidence levels, describe how you weighted them. Did you prioritize RCTs over case series? Did you use case reports only to identify rare adverse events? Make the logic explicit.

When Lower-Level Evidence Is All You Have

Not every device category has abundant high-level evidence. For some devices, case series and retrospective cohorts are the norm. This does not disqualify your clinical evaluation.

What matters is transparency. Acknowledge the evidence level. Explain why higher-level studies are absent. Justify why the available evidence is still appropriate for demonstrating safety and performance.

For example, if your device is used in a rare condition, large RCTs may not exist. You rely on case series from specialized centers. Document this. Explain that the evidence base is limited by the clinical context, not by insufficient literature search.

Reviewers accept this reasoning when it is clearly stated. What they do not accept is treating low-level evidence as if it were high-level, or ignoring evidence quality altogether.

How This Connects to Clinical Conclusions

Evidence levels do not just influence literature appraisal. They shape the strength of your clinical conclusions. When you conclude that your device is safe and performs as intended, that conclusion must be proportional to the evidence supporting it.

If your conclusion is based on high-level evidence—multiple well-designed studies with consistent findings—you can state it with confidence. If your conclusion relies on lower-level evidence, your language should reflect that. You may conclude that available evidence suggests safety and performance, while acknowledging gaps to be addressed through PMCF.

This is not about weakening your conclusions. It is about aligning them with the evidence base. Reviewers respect measured, evidence-based statements. They question overconfident conclusions that are not supported by the data.

What Reviewers Look For

When a Notified Body reviews your literature appraisal, they look for structure, transparency, and consistency. They want to see that you applied a defined method, that you assessed quality systematically, and that your conclusions follow logically from the evidence.

They check whether evidence levels are assigned. They check whether the appraisal narrative explains how evidence quality influenced your evaluation. They check whether your clinical conclusions are proportional to the strength of the data.

If these elements are missing or inconsistent, they raise questions. Not because they doubt your competence, but because the evaluation does not meet the methodological standard required by the MDR.

How to Fix This Before Submission

Before submitting your CER, review your literature appraisal critically. Ask: can a reviewer see how I assessed evidence quality? Can they trace my reasoning from appraisal to conclusions?

Check your literature tables. Are evidence levels clearly assigned? Check your appraisal narrative. Does it explain how evidence quality influenced your evaluation? Check your clinical conclusions. Are they proportional to the strength of the data?

If the answer to any of these questions is unclear, revise. Add the missing elements. Make your reasoning explicit.

This is not about adding length. It is about adding structure. A few clear statements about evidence levels can prevent multiple rounds of clarifications later.

Final Thought

Appraising literature quality is not about following a formula. It is about making your reasoning visible. When you assign evidence levels systematically, weight studies accordingly, and align your conclusions with the strength of the data, your clinical evaluation becomes more credible and more defensible.

Reviewers do not expect perfection. They expect rigor. They expect transparency. They expect that you can justify why your conclusions are supported by the evidence you cited.

If your literature appraisal lacks explicit evidence levels, the entire evaluation weakens. Not because the studies are wrong, but because the reasoning is hidden.

Make it visible. Make it systematic. Make it defensible.

Peace,
Hatem
Clinical Evaluation Expert for Medical Devices
Follow me for more insights and practical advice.