ACADEMIC WRITING GUIDE
How to Critically Appraise Research Articles for a Dissertation Literature Review (With Worked Examples)
Finding relevant studies is only the beginning of a good dissertation literature review. Two articles may investigate the same question yet offer very different grounds for trusting their findings. One may measure an outcome carefully and account for competing explanations; another may rely on a self-selected sample or draw causal conclusions from a design that cannot establish causation.
Critical appraisal helps you explain what a study contributes, where uncertainty enters, and how much weight its findings deserve in your own argument. The aim is neither to praise every published paper nor to hunt for flaws. It is to make judgments that are fair, specific, and supported by the methods and results actually reported.
This guide shows how to appraise quantitative, qualitative, and intervention research; how to translate your judgments into dissertation paragraphs; and how to compare studies without repeating one article summary after another. Every worked study and numerical result below is hypothetical and provided for teaching—not a finding to cite in a real dissertation.
What is critical appraisal in research?
Critical appraisal is a structured examination of a study's design, conduct, analysis, reporting, relevance, and interpretation. It asks whether the evidence is suitable for the research question, how it may have been affected by bias or missing information, and what conclusions remain defensible.
It is not the same as checking the journal's reputation, the author's academic title, or the citation count. Those details may help identify a source, but they cannot substitute for reading its methods and results.
| Task | Main question | What you produce |
|---|---|---|
| Summarising | What did the researchers investigate and find? | A faithful account of the study's purpose, methods, and results |
| Critically appraising | How securely do the evidence and methods support those findings? | Specific strengths, risks of bias, limitations, and warranted conclusions |
| Synthesising | How do the findings relate across studies? | A qualified conclusion that considers differences in design, context, and evidence quality |
For example, summarising a survey might mean reporting an association between feedback usefulness and student engagement. Appraising it means asking how both variables were measured, who answered, and whether the authors went beyond what an observational survey can show. Synthesising means comparing that result with evidence from other studies and explaining what their differences imply.
Why the research design changes your appraisal
There is no single checklist suitable for every paper. JBI provides design-specific critical appraisal tools, while the Critical Appraisal Skills Programme (CASP) offers separate checklists for several study types. The point of using a checklist is to prompt careful judgment—not to manufacture a universal percentage score. The JBI critical appraisal tools are a useful starting point.
| Study design | Questions that matter especially | A claim to question |
|---|---|---|
| Cross-sectional survey | Who responded? Were variables measured suitably? Were confounders considered? | “The measured association proves a causal effect.” |
| Randomised controlled trial | Was the allocation sequence random and concealed? Were deviations and missing outcomes handled? | “Randomisation automatically means this result is free of bias.” |
| Cohort or before-and-after study | Were comparison groups or periods comparable? Could confounding, selection, or time trends explain the result? | “The observed change must have been caused by the intervention.” |
| Qualitative interview study | Do sampling, researcher position, interpretation, and supporting extracts fit the approach? | “The interview themes represent how everyone in the population thinks.” |
| Mixed-methods study | Are both strands credible and is their integration meaningful? | “Using interviews and a survey automatically strengthens the findings.” |
| Systematic review | Was the search and study-selection process appropriate? Were included studies appraised and findings synthesised transparently? | “A systematic-review label guarantees reliable conclusions.” |
A well-conducted interview study may illuminate lived experiences but not estimate their population prevalence. A large cross-sectional dataset may establish a precise association while remaining unable to demonstrate which variable came first. Appraise a study against its own question and design, not against a different kind of study you wish the authors had conducted.
If you are still collecting literature, begin with our systematic literature-search guide to document databases, search strings, screening decisions, and inclusion rules. Critical appraisal begins once you have the relevant full texts.
Nine steps to critically appraise a research article
1. Identify the exact question and intended claim
Write a one-sentence description of the study's population, setting, key concepts, exposure or intervention, comparison (if relevant), and outcome. Distinguish describing, exploring, predicting, and estimating an intervention effect. A paper exploring students' accounts of feedback is not trying to prove that feedback caused higher examination scores.
2. Confirm the design from the Methods section
Look past the title and abstract. Determine whether the study is cross-sectional, longitudinal, experimental, qualitative, mixed methods, or a review. Note whether the design was selected and described consistently with the question. For mixed methods, identify the sequence or relationship between strands and where the authors integrated their findings.
Our research methodology guide explains how research questions, designs, evidence, and analyses should align—the same chain you need to inspect when appraising a published study.
3. Examine selection, sampling, and missing participants
Ask who was eligible, how people or records were selected, who did not participate, and whether exclusions or loss to follow-up could affect the findings. Calculate the response rate where the denominator is available, but do not assume that a low rate automatically proves bias. The central question is whether respondents may differ from nonrespondents in ways relevant to the result.
For qualitative work, ask whether recruitment could access the perspectives needed for the research question—not whether its sample is statistically representative.
4. Check the quality of the information collected
In quantitative studies, inspect how each variable was defined and measured. Consider validity, reliability, timing, instrument suitability, and missing-data handling. For interviews and observations, examine whether the data-generation procedures, context, prompts, and relationships between participants and researchers are described sufficiently to judge the findings.
For a practical explanation of why these decisions matter, see our operational definitions guide.
5. Identify the biases that could actually change the answer
Consider selection, confounding, measurement error, recall, social desirability, researcher influence, attrition, and selective reporting where the design makes them relevant. Do not simply list every bias you have heard of. Explain how the specific weakness could affect the outcome or interpretation.
A before-and-after study conducted across two seasons, for example, may confuse an intervention's effect with seasonal changes. A voluntary survey may overrepresent students with unusually strong opinions. Sometimes the size or direction of the resulting bias cannot be determined from the paper; say so.
6. Assess the analysis and uncertainty
Check whether statistical analyses are suited to the data and design, and whether relevant estimates, uncertainty, and missing information are reported. A non-significant p-value does not automatically establish no effect. Qualitative appraisal instead asks how the authors moved from accounts or documents to interpretations, how disagreements or alternative readings were addressed where appropriate, and whether reported extracts genuinely support the claims.
7. Compare the findings with the authors' conclusions
Mark the boundary between what the data show and what the authors say the data mean. Association does not alone imply causation; a descriptive interview theme is not a population prevalence estimate; and a small imprecise trial does not prove equivalence. A strong appraisal often accepts the reported observation while qualifying an overstated inference.
8. Check relevance and applicability to your dissertation
Even a methodologically careful study may differ from your population, location, timeframe, educational level, institutional context, or outcomes. Explain which differences matter to your own question. An evidence gap in your chosen setting is not established merely because a paper comes from another country.
9. Record an evidence-backed judgment, then use it in synthesis
Save a concise appraisal note with the relevant page or section, the specific strength or limitation, and its consequence for your argument. Where your department or systematic-review protocol requires a formal tool, use the relevant current version and document how you applied it. Do not turn a checklist into a single numerical quality percentage unless the tool and protocol explicitly justify that approach.
The end product should answer: What can I legitimately use this study to support, and what remains uncertain?
Worked example 1: Appraising a cross-sectional survey
Hypothetical study question: Is perceived usefulness of written academic feedback related to undergraduate academic engagement?
A university emails 600 undergraduates an online survey. Of these, 240 complete it: a 40% response rate (240 ÷ 600 × 100). Both feedback usefulness and engagement are self-reported in the same questionnaire at one point in time. The article reports a positive Pearson correlation, r = .32, and concludes that better feedback improves engagement.
| Appraisal area | Evidence in this imaginary paper | What it means for your judgment |
|---|---|---|
| Sampling | 240 of 600 invited students respond (40%) | Nonresponse may matter if respondents differ systematically from nonrespondents |
| Measurement | Both variables are self-reported on one survey | Shared response tendencies and measurement error may influence the association |
| Design | Cross-sectional, with no time ordering established | The study cannot establish whether feedback usefulness preceded engagement |
| Result | Positive association, r = .32 | A relationship is observed; its cause and practical implications need separate evidence |
| Interpretation | Authors say feedback “improves” engagement | The language is stronger than the evidence supports |
Weak dissertation sentence
This was a strong study because it surveyed many students and showed that feedback improves engagement.
Stronger dissertation-style paragraph
The hypothetical survey suggests a positive association between perceived feedback usefulness and self-reported engagement among respondents (r = .32). However, only 240 of the 600 invited students participated, and the study provides no evidence here that nonrespondents resembled respondents on the relevant characteristics. The two constructs were also measured by self-report at one time point, raising possible shared-method influences and leaving temporal order unresolved. The finding is useful for identifying a possible association in this university context, but it does not establish that improving written feedback would cause greater engagement.
Why the revision works: It reports the observation faithfully, uses the actual response-rate calculation, identifies design-specific uncertainties, and adjusts the causal claim. It does not call r = .32 “proof” of a strong relationship or assert that nonresponse bias has definitely occurred. The authors would need stronger design and supporting evidence to justify an intervention-effect claim.
Useful synthesis role: This study could support the proposition that perceptions of feedback and engagement are associated in some respondents, while a longitudinal or experimental study might be needed to investigate direction or causation.
Worked example 2: Appraising qualitative interviews
Hypothetical study question: How do first-year students describe the development of a sense of belonging at university?
Researchers interview 18 first-year students, recruited through a university wellbeing office. They identify three themes: navigating social networks, understanding academic expectations, and seeking institutional support. The report provides participant quotations but offers limited explanation of recruitment refusals, the interviewer's institutional role, or how interpretive decisions were developed.
| Appraisal question | Grounded judgment for this example |
|---|---|
| Were relevant experiences accessible? | Potentially, but recruiting only through a wellbeing office may miss students not engaged with support services |
| Do excerpts support the themes? | The quotations help, although a few extracts do not establish that every interpretation is well grounded |
| Is the analytic pathway sufficiently visible? | Limited reporting of theme development restricts the reader's ability to examine alternative interpretations |
| Is researcher position considered? | An unexplained institutional relationship may shape what participants feel comfortable saying |
| What can the study contribute? | Context-rich accounts of some participants' experiences, not population-wide prevalence estimates |
Weak dissertation sentence
The interviews are unreliable because the sample is too small and the researchers did not prove that all coders agreed.
Stronger dissertation-style paragraph
The hypothetical interview study offers useful insight into how some first-year students experience belonging and institutional support. Participant excerpts illustrate the reported themes, but recruitment through a wellbeing office may privilege accounts from students already willing to seek assistance. The limited description of researcher positioning and theme-development decisions also makes it difficult to assess how alternative readings were considered. Consequently, the findings are best used as context-sensitive evidence about possible experiences, rather than as a representative description of all first-year students.
Why the revision works: It assesses the study against its qualitative purpose, explains why recruitment and analytic transparency matter, and identifies the reasonable use of the evidence without rejecting it because it lacks statistical representativeness.
There is an important methodological caution. Member checking, multiple coders, and inter-coder agreement are not universal requirements for every qualitative approach. Braun and Clarke explain why coding-reliability expectations should not be imposed automatically on reflexive thematic analysis. The appropriate question is whether the researcher's choices are coherent with the declared approach and sufficiently explained. Our qualitative trustworthiness guide expands on that distinction.
Useful synthesis role: The interviews may help explain how belonging is experienced or constructed, especially alongside studies using different recruitment routes or other qualitative approaches. They do not establish how common each experience is among all students.
Worked example 3: Weighing a trial against a before-and-after study
Hypothetical review question: Do SMS reminders reduce missed clinic appointments?
Study A is a randomised comparison of 400 patients, allocated in equal groups of 200 to SMS reminders or usual care. Suppose 34 of 200 SMS patients and 50 of 200 usual-care patients miss their appointment: 17% versus 25%, an observed difference of 8 percentage points. The paper describes the allocation sequence and reports outcomes for all participants in this illustrative dataset, but a reviewer would still need to check allocation concealment, deviations from the planned intervention, missing outcomes, and the prespecified analysis.
Study B is a before-and-after clinic evaluation. Suppose 120 of 600 appointments are missed before reminders and 84 of 600 after implementation: 20% versus 14%, an observed decline of 6 percentage points. However, staffing and seasonal attendance patterns also changed between periods, and there was no concurrent comparison group.
These figures illustrate arithmetic only. They are not the results of real trials, and the examples provide no confidence intervals or significance tests. The two percentage-point changes should not be pooled or compared as though they were estimates from equivalent designs.
| Appraisal feature | Study A: randomised comparison | Study B: before-and-after evaluation |
|---|---|---|
| Main contribution | Comparison between randomly allocated groups | Observation of change in a routine service setting |
| Key validity questions | Allocation concealment, deviations, missing outcomes, selective reporting | Concurrent changes, baseline differences, seasonal trends, recording consistency |
| Observation | 25% versus 17% missed appointments | 20% before versus 14% after |
| Defensible inference | Potential intervention effect if execution and analysis support it | Observed improvement, but attribution to SMS alone is uncertain |
Weak synthesis
Both studies proved that text reminders prevent missed appointments, so the intervention is effective everywhere.
Stronger dissertation-style synthesis
Both hypothetical studies observed fewer missed appointments in periods or groups receiving SMS reminders, but their designs provide different grounds for attribution. Study A's randomised comparison offers stronger protection against baseline confounding, provided allocation, adherence, outcome measurement, and follow-up were handled appropriately. Study B adds information about routine implementation, but concurrent staffing and seasonal changes could partly explain its before-and-after difference. The combined observations are consistent with a potentially useful reminder strategy, while the strongest case for a causal effect depends on the quality and uncertainty of the randomised evidence. Neither example alone establishes effectiveness across all clinics or patient groups.
Why the synthesis works: It does not automatically discard the weaker design. Instead, it explains what each study contributes, why the causal confidence differs, and what remains unestablished. This is how appraisal should change the weight and wording of a literature-review conclusion.
A reusable critical appraisal matrix
Create an evidence matrix as you read so you do not have to reconstruct your judgments when writing. Record exact page numbers or sections and distinguish not reported from not done. A missing description may prevent judgment; it does not prove the procedure never occurred.
| Field | Record this for each paper |
|---|---|
| Citation, question, context | Author/year; precise question; population and setting |
| Design and evidence | Design; recruitment; data, measures or qualitative materials; analysis |
| What supports the finding | Concrete strengths tied to the paper's methods and reported evidence |
| Main uncertainty | Specific risk of bias, unexplained procedure, imprecision, or applicability limit |
| Evidence location | Page/table/appendix supporting each appraisal judgment |
| Consequence and use | What claim the study supports, what it cannot establish, and how it fits other studies |
Completed sample note (Example 1): Cross-sectional university survey; 240/600 responses (40%); simultaneously reported perceptions and engagement; r = .32; relevant association but possible nonresponse and shared-method influences; do not infer that feedback changes engagement.
That note is useful because it records the finding, the supporting and limiting evidence, and the consequence. “Quality: medium” on its own would not help you write a defensible paragraph.
Turn appraisal notes into a convincing literature-review paragraph
A critical literature review should not become a succession of mini-reviews that begin “Author A found…” and end “More research is needed.” Instead, arrange evidence around a shared question, compare the design features that matter, and finish with an interpretation your sources can support.
Try this sequence:
- State the evidence question or pattern. What is consistent or disputed across the sources?
- Compare findings and methods. Which studies measure the same outcome, in similar populations, with what designs?
- Explain consequences. Why do sampling, measurement, context, analysis, or bias risks affect the conclusions?
- Give a calibrated conclusion. What is reasonably supported, and which specific uncertainty remains?
Reusable paragraph model
Studies A and B both reported [a specific observation], but they differed in [the design feature that matters]. Study A's [strength] supports [limited inference], whereas Study B's [specific constraint] leaves [precise alternative explanation] unresolved. The available evidence therefore supports [qualified conclusion] in [defined context] rather than [overstated conclusion]. Further investigation is needed specifically to clarify [unanswered question], not merely because the papers are imperfect.
Replace every bracket with information checked in the real studies; never use a model example as a source of empirical findings. For help connecting studies into a meaningful argument, see Acadelyra's academic-source synthesis guide.
What if the article is mixed methods or a systematic review?
For mixed-methods research, examine the appropriateness of each strand and then the integration. If the survey claims broad prevalence but uses a severely selective sample, interviews cannot automatically repair that problem. Similarly, a well-run survey does not validate weak interpretation of qualitative interviews. Ask whether the combination answers a question that the strands could not answer as well independently. The Mixed Methods Appraisal Tool (MMAT) offers a recognised appraisal framework for mixed-studies reviews.
For a systematic review, assess whether the authors defined eligibility criteria, searched relevant sources transparently, handled screening appropriately, appraised included evidence, and chose defensible synthesis methods. A systematic review of low-quality or poorly selected studies does not become reliable simply because many papers were combined. Follow the tool and reporting expectations suited to the review type; a dissertation's general literature review does not automatically require a full formal systematic-review protocol.
Common critical-appraisal mistakes
- Mistaking publication for proof of quality. Peer review and citation counts are not substitutes for examining actual study methods.
- Using the same checklist for everything. Match appraisal questions to the design and, for formal reviews, your approved protocol.
- Automatically rejecting small qualitative samples. Judge fitness for the qualitative aim, recruitment, depth, context, and transparency instead.
- Treating a response rate as a complete bias assessment. Consider who may be missing and whether their absence could alter the result.
- Reporting a non-significant finding as proof of no effect. Precision, study design, and the interval of plausible effects matter.
- Converting association into causation. Respect temporal order and alternative explanations.
- Awarding unexplained “quality scores.” Give reasons and consequences, not unsupported percentages.
- Equating “not reported” with “not done.” Where methods are unclear, describe the reporting gap accurately.
- Listing every imaginable weakness. Prioritise problems that could change the answer to your research question.
- Failing to use appraisal in synthesis. The literature-review conclusion must reflect the strengths and uncertainties you identified.
Frequently asked questions
Must every dissertation literature review use a formal appraisal checklist?
Not always. Requirements vary by university, discipline, and review type. A systematic review commonly needs a documented appraisal process, while a conventional dissertation literature review may require reasoned critical analysis without formal ratings. Follow your supervisor, handbook, and any approved protocol.
Should I exclude a study whenever I find a limitation?
Not automatically. If you have predefined exclusion rules, apply them consistently. Otherwise, explain whether the limitation changes your confidence, restricts applicability, or qualifies the conclusion. A study may still contribute valuable evidence despite uncertainty.
Is critical appraisal the same as risk-of-bias assessment?
They overlap. A formal risk-of-bias tool focuses on particular threats to the validity of a result, while broader appraisal may also consider reporting, applicability, and relevance. The Cochrane RoB 2 guidance explains domain-based judgments for results from randomised trials.
Can I use one checklist for mixed-methods articles?
Use a tool designed for that purpose when appropriate, and check the qualitative and quantitative components and their integration. Avoid forcing mixed research into an exclusively quantitative or exclusively qualitative template.
How much critical appraisal belongs in my final chapter?
Enough to explain the evidence behind your interpretation. Brief, precise observations are usually more valuable than a long list of generic strengths and weaknesses. More detailed appraisal matrices may belong in appendices if your course or protocol requires them.
Can AI do critical appraisal for me?
AI can help organise questions or draft a note-taking table, but it may invent unreported methods, confuse outcomes, or overlook important details. Read the full source, verify every judgment and citation, and follow your institution's AI-use rules.
Final critical-appraisal checklist
- [ ] I identified the research question and actual study design correctly.
- [ ] My appraisal questions fit that design and my review's purpose.
- [ ] I examined how participants, records, or qualitative materials were selected.
- [ ] I considered the suitability of measures, data generation, and analysis.
- [ ] I identified consequential bias risks rather than merely listing possible problems.
- [ ] I distinguished what the study reports from what its authors infer.
- [ ] I qualified causal, population-wide, and certainty claims appropriately.
- [ ] I considered whether the study applies to my dissertation context.
- [ ] I recorded specific page or section evidence for important judgments.
- [ ] I distinguished missing reporting from proven methodological failures.
- [ ] I did not automatically discard every study with limitations.
- [ ] I used the appraisal to shape cross-study synthesis.
- [ ] My final paragraph cites actual papers, not these illustrative examples.
Final takeaway
Good critical appraisal does not make a literature review more negative. It makes the argument more trustworthy. Identify what each study contributes, examine the design and evidence, and explain how your judgments affect the conclusions you draw from the literature. Credit useful findings without granting them more certainty than they deserve.
If you need help organising the literature, structuring your argument, or presenting research methods accurately, Acadelyra offers academic writing guidance and support tailored to your brief. Request academic help when you need it.
References and methodological guidance
- JBI. Critical Appraisal Tools. https://jbi.global/critical-appraisal-tools
- Critical Appraisal Skills Programme (CASP). Checklist Archive. https://casp-uk.net/casp-tools-checklists/checklist-archive/
- Cochrane. Handbook, Chapter 8: Assessing risk of bias in a randomized trial. https://training.cochrane.org/handbook/current/chapter-08
- Cochrane. Handbook, Chapter 25: Assessing risk of bias in a non-randomized study. https://training.cochrane.org/handbook/current/chapter-25
- Braun, V., & Clarke, V. One size fits all? What counts as quality practice in (reflexive) thematic analysis? https://doi.org/10.1080/14780887.2020.1769238
- National Collaborating Centre for Methods and Tools. Mixed Methods Appraisal Tool (MMAT), Version 2018 User Guide. https://www.nccmt.ca/knowledge-repositories/search/232
Editorial note: All numerical studies, examples, quotations and model appraisal paragraphs in this article are hypothetical teaching illustrations, not findings from identified published studies. Appraise and cite the actual research you use. Consult your institution's literature-review requirements and the appropriate current appraisal tool before submission.