What reviewers actually check in your literature search protocol

Written by HATEM RABEH, MD, MSc Ing

Your Clinical Evaluation Expert And Partner

in
S

The literature search is the foundation of every clinical evaluation. Yet it remains one of the most frequently cited deficiencies in MDR submissions. The reason is simple: manufacturers often treat the search as a task to complete, not as a protocol to defend.

Under MDR Article 61 and Annex XIV Part A, the clinical evaluation must be based on a comprehensive review of clinical data. That comprehensiveness is proven through the literature search strategy. The documentation must show that the search was designed to identify all relevant clinical evidence, not just the studies that support a favorable conclusion.

What reviewers verify is not whether you found studies. They verify whether your method was capable of finding all studies that should have been considered.

The search protocol as a legal document

A literature search strategy is not a narrative description. It is a reproducible protocol that another evaluator could execute and arrive at the same set of results.

This is where most documentation fails. Manufacturers describe what they did in general terms, but they do not document it with the precision required for verification.

Reviewers need to see the exact search strings used in each database. They need to see how those strings were constructed. They need to understand why certain terms were included and others excluded.

Without this, the search cannot be reproduced. And if it cannot be reproduced, it cannot be verified.

Database selection and justification

The choice of databases is not arbitrary. It must be justified based on the type of device, the clinical field, and the expected location of relevant evidence.

PubMed and Embase are standard. But for certain devices, other databases are necessary. If your device is used in a specific regional context, additional databases may be required. If it relates to rehabilitation or quality of life, sources like CINAHL or PsycINFO may be relevant.

What reviewers check is whether the manufacturer considered the scope of available evidence and selected databases accordingly.

The same applies to grey literature. Clinical trial registries, regulatory databases, and manufacturer repositories are part of the evidence base. Reviewers verify whether these sources were considered and, if not, why they were deemed unnecessary.

Search string construction and logic

The search string is the technical core of the strategy. It determines what is captured and what is missed.

Reviewers verify whether the search string was built logically. They check the combination of Boolean operators, the use of truncation and wildcards, and the inclusion of MeSH terms or Emtree descriptors where applicable.

A common deficiency is the use of overly narrow search strings that exclude relevant variations. For example, searching only for the proprietary device name without including generic terms, synonyms, or related procedures.

Reviewers also check for balance. If the search includes only terms related to efficacy and benefits, but excludes terms related to complications or adverse events, that is a red flag. The search must be designed to capture both favorable and unfavorable evidence.

This is not about creating a biased search. It is about demonstrating that the manufacturer did not create a biased search.

Filters, limits, and their justification

Filters and limits are necessary. But every restriction must be justified.

If the search is limited by publication date, the CER must explain why. If it is limited by language, the justification must be clear. If certain study types are excluded, the rationale must be provided.

Reviewers verify whether these restrictions are reasonable or whether they were applied to reduce the volume of inconvenient evidence.

A common issue is the exclusion of older literature without justification. Manufacturers assume that recent studies are more relevant, but this assumption must be defended. If the device has been on the market for years, older studies may contain critical safety data. Excluding them without explanation creates a gap in the evidence base.

Reviewers do not expect every possible study to be included. They expect the exclusions to be transparent and defensible.

Documentation of search execution

The protocol describes what should be done. The execution documentation proves that it was done.

This includes the exact search strings entered into each database, the date of the search, the number of results retrieved, and any alerts or error messages encountered during execution.

Many CERs skip this step. They document the protocol but not the execution. When reviewers ask for verification, the manufacturer cannot provide it because the searches were not logged.

Best practice is to document each search with screenshots or exported search histories. Include the number of hits for each string. Include the date and database version. This creates a complete audit trail.

It also protects the manufacturer. If the Notified Body questions whether certain studies were missed, the documented search proves what was retrievable at the time of execution.

The selection and screening process

Once the search is executed, the next step is screening. Reviewers verify that the screening process was systematic and transparent.

The CER should document how many records were retrieved, how many were duplicates, how many were excluded at title and abstract screening, and how many proceeded to full-text review.

Each exclusion decision should be based on predefined criteria. If studies were excluded because they did not meet the population, intervention, comparator, or outcome criteria, those criteria must be stated upfront in the protocol.

Reviewers check for consistency. If the protocol states that case reports will be excluded, but several case reports appear in the final reference list, that is a discrepancy. If the protocol states that only studies on the exact device will be included, but the CER relies on studies of similar devices, that is a contradiction.

Many deficiencies arise because the search strategy and the clinical evaluation plan were developed in isolation. They do not match. Reviewers notice.

Updates and ongoing surveillance

The initial search is not the end. Under MDR Article 61(11) and Annex XIV Part B, clinical evaluation is a continuous process. The literature search must be updated regularly.

Reviewers verify whether the CER includes a plan for updating the search. They check whether previous updates were performed as planned. They verify whether newly identified studies were incorporated into the evaluation.

A common issue is that manufacturers perform an initial search during the CER preparation, but they do not update it before submission. If six months have passed, significant new evidence may have been published. The absence of an update raises questions about the completeness of the data.

Best practice is to perform a final update search immediately before submission. Document it. Include the results in the CER. If no new relevant studies are found, state that. If new studies are found, incorporate them.

Transparency as the standard

What reviewers verify is transparency. They verify that the manufacturer documented the search in enough detail that it could be reproduced, challenged, and defended.

This is not about perfection. No search strategy is perfect. But it must be transparent.

When deficiencies arise, they rarely result from genuine gaps in the literature. They result from gaps in the documentation. The manufacturer performed a reasonable search but did not document it with the rigor required for regulatory verification.

The solution is not to perform more searches. The solution is to document the searches as if they will be audited. Because they will be.

This approach eliminates most deficiencies. It shifts the conversation from whether the search was adequate to whether the evidence supports the conclusions. That is where the conversation should be.

The literature search is not an administrative step. It is the proof that your clinical evaluation is based on all available evidence, not just the evidence you wanted to find.

Peace,
Hatem
Clinical Evaluation Expert for Medical Devices
Follow me for more insights and practical advice.