ACADEMIC WRITING GUIDE

How to Handle Missing Study Data in a Systematic Review Dissertation (With Worked Examples)

By Acadelyra Editorial TeamPublished

A systematic review can have a carefully planned search, sensible eligibility criteria and a well-organized data extraction table—yet still run into a stubborn problem: an eligible study does not report the information needed to answer the review question. A paper mentions improvement but gives no usable outcome data. Two reports of the same trial describe different follow-up periods. A study reports an outcome only for some participants. Or the investigator never responds to a request for clarification.

These gaps matter. They can change which studies contribute to a synthesis, introduce bias and make apparently decisive findings less trustworthy. They also tempt researchers into shortcuts: silently dropping studies, filling cells with invented values, describing an unreported result as “no effect,” or claiming a meta-analysis was impossible without explaining what was missing.

This guide explains how to handle missing study data in a systematic review dissertation, from the first uncertain extraction field through author contact, bias assessment, synthesis decisions and transparent dissertation reporting. The emphasis is not on manufacturing a complete spreadsheet; it is on maintaining an accurate record of what the evidence can and cannot establish. It draws on Cochrane's data-collection guidance and its analysis guidance on missing data.

Teaching-example disclaimer: Every study label, report excerpt, response and research decision in the worked examples below is fictional. The tables are illustrations, not evidence from an actual systematic review. Do not copy the fictional entries into a real dissertation as though they were observed findings.

What counts as missing data in a systematic review?

“Missing data” is not one problem with one remedy. At least four different situations need to be distinguished before selecting a response.

An eligible study may be missing entirely. It may never have been published, may have been indexed inadequately or may be known only through a registry or another report. A review cannot assume the studies it found are necessarily all the studies that exist. This is particularly important where the probability of a study being reported is related to its findings.

A study may be found, but a relevant outcome or result is not reported. The methods might say that course completion was measured, while the results section gives only student satisfaction. Even where the report mentions completion, it may omit the group-level statistics, effect estimate or measure of uncertainty needed for analysis.

A study may report a result, but some individual participants lack outcome measurements. For example, some participants may have withdrawn before final follow-up. That is a risk-of-bias concern within the primary study, not automatically a missing study-level effect estimate. The study might report a result despite having incomplete participant data.

A value may be unclear rather than truly unavailable. The report may use inconsistent outcome labels, show only a graph, contain an apparent transcription error or report adjusted estimates without identifying their basis. Checking other reports and contacting authors may resolve the uncertainty.

Types of missing information and the first question to ask
SituationExample in an eligible studyFirst questionWhy the distinction matters
Missing studyAn identified trial has a registration record but no located results reportIs there another source of results?Potential missing-evidence or reporting bias
Missing outcome/resultCompletion was measured but not reported numericallyWas the result reported elsewhere or obtainable?May affect which studies contribute to an outcome synthesis
Missing summary statisticsMeans are given without an appropriate measure of dispersionCan the required quantity be obtained or validly derived?A calculation is not justified without suitable inputs and assumptions
Missing participant outcomesFollow-up outcomes are unavailable for some enrolled studentsCould missingness depend on participants' outcomes?Can bias the study's reported result
Ambiguous extractionTwo reports give incompatible labels or time pointsAre these different analyses or an error?Prevents extracting the wrong value

The foundational rule is to label the problem before treating it. The Cochrane Handbook Chapter 13 also distinguishes missing results of known studies from potentially missing, unknown studies. These different forms of missing evidence require different reasoning.

Start with your protocol, not the empty spreadsheet cell

A reviewer should decide how to handle unavailable information as far as possible before discovering which findings are favorable, unfavorable or statistically significant. A systematic review protocol can predefine the outcomes, time points, preferred measures, hierarchy for multiple reports and plans for unavailable summary data.

A useful decision rule might read:

When an eligible study lacks the statistics needed for a planned effect estimate, we will search supplementary and linked reports, examine available registry information and request clarification from corresponding authors where feasible. We will retain the study in the review if it meets eligibility criteria, indicate whether its result contributes to each synthesis, record any defensible conversions or assumptions and assess the consequences of missing evidence.

This is an illustrative rule, not a claim that every review must contact every author or make the same choices. Adapt it to the question, research design, resources and approved protocol. If a dissertation project is already underway, document changes explicitly rather than backdating the protocol.

Our guide to writing a systematic review protocol for a dissertation explains where such plans belong. The related guide on inclusion and exclusion criteria helps distinguish whether a study is eligible from whether its data are usable for one particular outcome.

Step 1: Establish what the study actually reports

Before marking a value missing, check whether it appears under another label or in another source. Researchers sometimes extract from the first PDF they find, even when the study has a protocol, supplementary tables, registry entry, correction or follow-up publication.

Start by assigning one study ID and linking every report that belongs to it. The unit of interest is generally the study, not the number of articles written about it. Mixing separate reports as if they were separate studies can double-count participants; choosing one convenient report without comparing the others can produce inaccurate extraction.

Next, record each outcome precisely: construct, measurement instrument, metric, comparison, analysis population and time point. “Academic achievement” is too vague if a study measured exam score at one month and course completion at the end of term. Both may be relevant but they are not interchangeable.

Keep a clear extraction status for every critical field:

  • Reported: a clearly identified value appears in the source.

  • Calculated or converted: a value was derived with a documented, methodologically valid rule and adequate source inputs.

  • Author-confirmed: clarification or data came from identifiable study investigators, with correspondence retained securely.

  • Unavailable: appropriate checks were made but the information could not be obtained.

  • Not applicable: the field does not apply to that design or outcome.

  • Unclear: ambiguity remains and should not be silently resolved.

Do not treat “not reported,” “not measured,” “not applicable,” “zero,” and “no effect” as synonyms. Zero is a numerical value; “no effect” is an inferential claim; “not reported” is a documentation status. If these become one code in your spreadsheet, later synthesis may become misleading.

For an end-to-end extraction workflow, see Acadelyra's data extraction form guide. Cochrane's Chapter 5 discusses data sources, correspondence and consistent extraction across multiple reports.

Step 2: Search alternative sources before contacting investigators

A missing value in a journal article does not mean the value does not exist. In a reproducible workflow, inspect likely alternative sources before escalating the issue:

First, look for supplementary files associated with the article, including supplementary outcome tables and pre-specified analyses. Verify that each supplementary file actually belongs to the study and version being reviewed.

Second, check the study's trial or project registry entry where applicable. A registry may identify intended outcomes, analysis time points or linked result records. Be careful: a protocol's planned outcome is not itself a measured result, and a registry field does not automatically resolve contradictory published findings.

Third, search for linked publications, conference abstracts, dissertations or subsequent follow-up reports. A conference abstract may provide preliminary results, but those results may differ from a final report. Document the source and do not combine non-equivalent versions.

Fourth, inspect corrections or retractions. A corrected table may supersede an earlier figure. A retraction requires a deliberate eligibility and trustworthiness assessment; it should not be treated as an ordinary missing cell.

If needed, contact an institutional librarian for help locating hard-to-find reports. Record the source searched and the result. A note saying “searched everywhere” is not auditable; a short dated log with named sources is.

Step 3: Contact study authors professionally

Investigator correspondence can resolve missing definitions, unclear analyses or unavailable statistics. However, no dissertation should claim that authors supplied information unless the correspondence actually occurred and is documented.

A concise request should specify the study, explain the systematic review, identify the exact outcome and time point, and ask for the smallest information needed. Avoid requesting identifiable participant-level data unless there is a specific authorized, secure research arrangement; summary-level clarification may suffice.

Illustrative author-contact email (fictional; adapt before sending):

Subject: Clarification request regarding [study title] for a systematic review

Dear Dr [Surname],

I am conducting a systematic review for a dissertation on structured written formative feedback and first-term course completion in undergraduate education. Your study, [full citation], appears eligible. I can find the completion outcome in the Methods section, but cannot locate the group-specific first-term results in the published article or supplement.

Could you please confirm whether these results were collected and, if available, direct me to a public report or provide the relevant aggregate summary statistics and the corresponding analysis population? I would acknowledge the source of any clarification in the review.

Thank you for considering this request.

Kind regards,

[Researcher's name and institutional affiliation]

Before sending, check university and supervisory policies on external correspondence. Where co-reviewers share responsibilities, agree who contacts whom; duplicates can confuse investigators. Keep an audit trail of requests and replies, but do not post private contact details or confidential responses publicly without permission.

There is no universal response schedule that fits every project. A protocol might specify one initial email and one follow-up after a reasonable interval, subject to dissertation deadlines and institutional guidance. If no reply arrives, record no response by the extraction cutoff, not “the author has no data.”

Fictional author-contact and clarification log
Study IDQuestionSource checkContact statusExtraction decision
F01Which cohort generated the completion result?Article and supplement checkedHypothetical response clarifies first-year cohortExtract only the clarified eligible cohort if documented
F02Is a first-term completion result available?Article, supplement and registry checkedHypothetical request sent; no response by cutoffRecord result unavailable; retain eligibility assessment
F03Which follow-up period does the graph show?Two related reports checkedHypothetical correction notice located; no contact neededUse corrected report and document provenance
F04Were absent follow-up outcomes analysed?Methods and trial report checkedHypothetical response clarifies analysis populationRecord missing participant data and assess study-level bias

All log entries above are invented teaching scenarios. They are not real investigator communications.

Step 4: Separate a missing result from a missing study

Suppose a paper meets your inclusion criteria but omits the course-completion outcome you planned to synthesise. It may still contribute study characteristics, risk-of-bias information or results for other outcomes. Removing the entire study solely because the desired result is not available can conceal the very reporting problem the review should disclose.

Cochrane's reporting guidance cautions against treating lack of reported outcomes as an automatic reason to exclude an otherwise eligible study. Eligibility for the review and contribution to an individual synthesis should be recorded separately.

For an intervention review, a useful outcome-specific map may therefore show the eligible studies, those with an available result for each outcome, and those with results missing or unusable. Explain whether non-reporting could be related to effect size or statistical significance, without claiming to know the missing result.

This is also where a PRISMA 2020 flow diagram and a synthesis-contribution table answer different questions. The flow diagram records how records and reports were screened and studies included; it does not by itself show which results contributed to every outcome analysis. See our PRISMA 2020 flow-diagram guide.

Step 5: Decide whether a reported result can legitimately be used

A value may be partially reported rather than entirely absent. The study may give an estimate without precision, provide a figure but not the underlying table, or report a measure different from your planned effect measure.

The correct response depends on the design, outcome scale and planned synthesis. Some summary statistics can sometimes be derived from other reported information; some effects can be converted with explicit assumptions. But there is no universally valid conversion from an unspecified p-value, graph or qualitative claim to the numerical estimate a meta-analysis requires.

Before calculating or imputing anything, ask:

  1. Is the relevant outcome and time point unambiguously identified?

  2. Are the reported quantities sufficient for the specific mathematical operation?

  3. Does the proposed derivation preserve the correct comparison, effect direction and analysis population?

  4. Is the method consistent with the protocol and relevant methodology?

  5. Could a different defensible assumption materially change the conclusion?

  6. Can a reader reproduce the derivation from the reported inputs?

Document the inputs, formula, assumptions, rounding and source. Never invent a standard deviation, participant denominator, event count or confidence interval merely to make an analysis run. If the necessary conditions are not met, label the effect unavailable for that synthesis and explain why.

For reviews that do not pool effects, a structured narrative account may still describe what the paper reports, provided its limitations remain visible. Acadelyra's narrative synthesis guide explains how to compare findings without resorting to vote-counting by statistical significance. The Cochrane Handbook Chapter 10 explains the broader analysis choices.

Worked example: Handling four kinds of extraction gaps

Imagine a fictional dissertation review assessing structured written formative feedback and first-term course completion among first-year university students. Four invented studies meet the eligibility criteria. The reviewer is preparing an outcome-specific synthesis but discovers gaps that are superficially similar and methodologically different.

F01 reports a completion outcome, but the cohort is ambiguous. The article mixes first-year and later-year students in one summary. Another study report may clarify whether the relevant first-year subgroup can be identified. Until then, the reviewer records the subgroup result as unclear, not eligible by assumption. In this fictional scenario, a hypothetical author reply confirms a separately reported first-year group. The review then records the correspondence and uses the group-specific information if its quality and definitions are adequate.

F02 mentions completion in the Methods but reports only satisfaction scores. The reviewer checks the linked registry and supplement and sends a clarification request. No response is available by the planned extraction cutoff. F02 stays in the included-study inventory, but its completion effect is marked unavailable. The dissertation states explicitly that completion could have been measured but was not reported in an extractable form; it does not state that the intervention had no effect.

F03 reports follow-up results in two versions. An earlier report appears to measure completion at the end of term, while a correction identifies a later follow-up. The reviewer must determine which result corresponds to the protocol's predefined outcome and time point. The corrected version takes precedence for the corrected field, with the source and decision recorded. The other time point is not casually substituted merely because it produces a cleaner result.

F04 reports a completion result but has missing participant outcomes. This is not simply a missing cell in the reviewer's extraction workbook. The reviewer extracts the study's reported result while separately assessing whether attrition could bias it, using a suitable design-specific risk-of-bias approach. Any concerns should inform interpretation rather than being erased by the presence of a numerical result.

Fictional outcome-level disposition for four eligible studies
Fictional studyWhat is missing or uncertain?Review inclusionContribution to first-term completion synthesisRequired disclosure
F01Cohort definition originally unclear; hypothetical clarification obtainedIncludedEligible only if clarified first-year result and comparison meet protocol rulesClarification source and cohort definition
F02Outcome mentioned but usable result unavailableIncludedNo usable completion estimateChecks performed, contact status, potential missing-result bias
F03Conflicting report versions and time pointsIncludedUse only the version matching approved outcome definitionsCorrection provenance and chosen time point
F04Outcome reported but participant-level follow-up incompleteIncludedResult may contribute if methodologically appropriateAttrition, analysis population and study-level risk-of-bias judgment

The point of this example is not that the reviewer must include every available estimate in one pooled analysis. Depending on how comparable the studies are, the evidence may support a meta-analysis, another structured synthesis, or a transparent description of why quantitative pooling is inappropriate. The decision follows the question, methods and data quality—not a desire to produce a forest plot.

Step 6: Assess the right kind of bias

The phrase “risk of bias from missing data” can refer to different levels of the evidence. Avoid substituting one assessment for another.

Missing participant outcome data within a trial may produce biased effect estimates if missingness depends on the true unobserved outcome or is otherwise related to the intervention and outcome. Cochrane's RoB 2 guidance addresses this within-trial domain. A high follow-up percentage does not automatically guarantee low risk; context and reasons for missingness matter.

Selective non-reporting of an outcome or result can affect a particular review synthesis when a study's result is missing because of its magnitude, direction or statistical significance. Cochrane's Chapter 13 describes approaches to assessing missing-evidence bias in meta-analysis, including ROB-ME for relevant intervention comparisons.

Non-publication of entire studies is another form of missing evidence. A review may lack knowledge of trials that were conducted but never fully reported. Searches of registries and other sources may help, but an exhaustive search cannot prove there are no missing studies.

Do not conflate risk-of-bias tools: study-level appraisal, a synthesis-level missing-evidence assessment and certainty-of-evidence assessment are related but different activities. For more on selecting a tool and documenting study-level judgments, see Acadelyra's risk-of-bias assessment guide. To communicate the implications for an outcome's body of evidence, see our GRADE and Summary of Findings guide.

Step 7: Make defensible synthesis decisions

Missing information does not always require cancelling a synthesis, and it never justifies creating substitute observations. The appropriate response depends on how important the missing results are, whether missingness might be selective, and whether the remaining information answers the review question.

Where the planned effect estimates are available and comparable, a synthesis may proceed with a clear account of which studies contribute and why. Where an eligible study lacks usable data, retain it in the study inventory and transparently identify the affected synthesis. Where missing evidence raises serious concern about bias, consider whether any summary estimate would be misleading.

A preplanned sensitivity analysis may test justified alternative assumptions when there is adequate data and a legitimate analysis method. It cannot reconstruct unknowable results by wishful thinking. If only narrative synthesis is possible, clearly explain the available effects, study differences, uncertainties and the absence of missing information.

For student researchers, the most important discipline is to document each decision at the time it is made. The choice to exclude an estimate, contact an author or use a conversion should be traceable to a source and methodological rationale. A later dissertation examiner should not have to guess why a cell was left blank or why a study disappeared from a results table.

Decision guide for missing evidence at synthesis stage
Evidence situationPossible defensible responseWhat not to do
Eligibility established, outcome result unavailableRetain study inventory entry; disclose non-contribution to that outcomeExclude the entire study solely to hide the missing result
Missing summary statistic but valid inputs availableApply documented, appropriate derivation if the method permitsInsert an estimated value without inputs or assumptions
Graph-only resultConsider careful extraction only if appropriate and adequately documentedTreat approximate digitization as exact author-supplied data
Missing participant outcomesAssess bias in the reported result and interpret cautiouslyAssume the published estimate is unbiased merely because it is numeric
Missing results potentially related to findingsAssess risk of missing-evidence bias using a suitable approachTreat non-reporting as proof of no effect
Very substantial or consequential missing evidenceReconsider whether a pooled result would be misleading; explain decisionForce a precise-looking meta-analysis regardless of evidence quality

How to report missing data in dissertation Methods and Results

Good reporting distinguishes the planned procedure from the observed outcome of following that procedure. Readers need to see what was sought, what became available, which studies contributed and how limitations affected interpretation.

Methods: describe what you planned and did

An honest Methods paragraph should state the data sources, extraction roles, handling of multiple reports, author-contact procedure, decision rule for unclear or missing data and any defensible conversions or sensitivity analyses. If the reviewer worked alone, say so; do not claim independent dual extraction that did not happen.

Illustrative Methods wording (adapt only to actual practice):

We extracted study characteristics, prespecified outcomes, time points and available summary results into a structured form. When a required field was unclear or absent, we checked linked reports and supplementary sources and sought author clarification where feasible. We recorded the provenance and status of each missing or derived field. Studies meeting review eligibility criteria were not automatically excluded because a result for an individual outcome was unavailable. Any permitted data preparation was documented with its assumptions, and missing evidence was considered during risk-of-bias assessment and interpretation.

This model wording is useful only if those procedures were actually performed. If no authors were contacted, do not imply that they were.

Results: report what was found, without filling the gaps

The Results section should identify eligible studies with unavailable outcome results, distinguish them from studies excluded at screening, and explain which studies contributed to each synthesis. If authors provided clarifications, identify the nature of those clarifications and relevant limitations. Report material deviations from the protocol.

Illustrative Results wording based on the fictional example:

Four fictional studies met the prespecified eligibility criteria. One had no extractable first-term completion result after review of the available reports and hypothetical author correspondence; it was retained in the study inventory but did not contribute to the completion synthesis. Another study required clarification of its eligible cohort, while a separate report correction affected time-point selection. The review recorded these decisions in an outcome-level extraction log. Missing participant follow-up outcomes in a further study were addressed through risk-of-bias assessment rather than treated as an absence of a study-level result.

The paragraph above describes fictional events only. In a real dissertation, replace its content with verified study counts, actual correspondence outcomes and documented findings.

Discussion: explain how uncertainty affects conclusions

Do more than add “some data were missing” to the limitations paragraph. Discuss whether non-reporting could plausibly alter the direction, size or confidence of the synthesis; whether the missing information disproportionately affects certain comparisons or populations; and what uncertainty remains despite attempts to obtain data.

A balanced conclusion might note that apparent consistency among available studies is insufficient to establish a robust effect when consequential results from eligible studies are unreported. Conversely, not every missing descriptive variable undermines every outcome conclusion. Identify exactly which inference is affected and how much can be said from the available evidence.

PRISMA 2020 asks review authors to explain data collection, any data preparation such as handling missing summary statistics, and methods to assess bias due to missing results. The official PRISMA 2020 materials provide the reporting framework; they are not a replacement for an appropriate review method.

Common mistakes when handling missing study data

Mistake: deleting rows with blank outcome cells. A missing result can be an important finding about reporting practices. Preserve the study's eligibility and its outcome-specific contribution status separately.

Mistake: interpreting an empty field as zero. Unknown values are not zero observations. This error can create false precision or materially change an effect estimate.

Mistake: treating every missing result as evidence of selective reporting. Selective non-reporting is a concern to assess, not an accusation to make without evidence. Sometimes data were never measured or the report is simply unclear.

Mistake: applying a generic missing-survey-data technique to a review spreadsheet. Multiple imputation for an individual-level dataset is not an automatic remedy for absent published study estimates. The units, assumptions and available data are different. Our separate missing survey data guide concerns primary survey datasets, not missing results across studies.

Mistake: overlooking multiple reports of the same study. A registry, conference abstract and full paper may describe the same cohort. Keep them linked and check whether one updates or corrects another.

Mistake: saying an author “did not respond” without a contact record. Keep a dated, confidential log. Distinguish no address found, email not sent, undeliverable email and no reply before the cutoff.

Mistake: claiming missing results prove a null finding. Non-reporting does not establish absence of an effect. The honest conclusion is that the result is unavailable or insufficiently described.

Mistake: hiding protocol deviations. If you changed the planned outcome time point or extraction rule because a preferred dataset was missing, disclose and justify the change. Do not quietly rewrite the original plan.

Frequently asked questions

Should I exclude a study if it has no usable effect size?

Not automatically. A study may satisfy the review's eligibility criteria but be unable to contribute to a particular quantitative synthesis. Record both decisions, explain what is missing and assess the possible implications of non-reporting. Always apply the review's justified, prespecified criteria rather than inventing eligibility rules after seeing results.

Is “not reported” the same as “not measured”?

No. An outcome may have been measured but not published, or it may never have been measured. When the report does not establish which, classify the status as unclear and seek confirmation where proportionate. Avoid presenting an assumption as a fact.

Do I have to contact every study author?

There is no one-size-fits-all rule. Contact may be warranted where clarification could materially affect the review and where it is feasible and appropriate. Define a proportionate procedure, document what you actually did, and report unresolved gaps transparently.

Can I use software to fill missing review data?

Software can organize extraction, perform justified conversions and help maintain an audit trail. It cannot manufacture unavailable research evidence or make unsupported assumptions valid. Every derived value needs adequate source inputs, an appropriate method and clear documentation.

What if an outcome is mentioned in the protocol but absent from the publication?

Record that discrepancy, search relevant linked sources and consider whether selective reporting might bias the planned synthesis. Do not automatically conclude that the missing result was statistically non-significant or unfavorable. Assess the available evidence and report uncertainty.

Can I write narrative synthesis when some effect estimates are missing?

Yes, when the available evidence supports a transparent synthesis without an appropriate pooled estimate. Explain which studies contribute, what is reported and what remains unavailable. Narrative synthesis is not permission to hide missing outcomes or count statistically significant studies as votes.

Where should the missing-data log go in my dissertation?

Describe your method and key consequences in the Methods and Results chapters. A more detailed extraction-status or author-contact summary can be placed in an appendix where appropriate, with confidential correspondence and personal information protected under institutional policies. Use your department's dissertation requirements to decide the level of detail.

A final quality check before submission

Before submitting a systematic review dissertation, confirm that every missing-result decision has a documented source and disposition; eligible studies have not silently disappeared from the study inventory; any calculations or conversions are reproducible; author contact claims are supported; and the Discussion acknowledges uncertainty without overstating it. Also confirm the selected appraisal framework actually addresses the type of missingness at issue.

A persuasive dissertation does not pretend every study supplied perfect data. It shows that the researcher could distinguish unavailable evidence, uncertain reporting and possible bias, respond proportionately and explain what conclusions remain defensible.

Need support refining your systematic review? Acadelyra offers human-led research and dissertation support focused on methodological clarity, transparent reporting and academic integrity. Explore Acadelyra’s academic support services and share your project brief through the site’s enquiry process so the team can discuss suitable assistance. Support should strengthen your own scholarly work, not invent findings or impersonate a researcher.

References and further reading