Appraising Qualitative Research
33 unread replies.44 replies.
Download the Johns Hopkins Research Evidence Appraisal Tool and the Johns Hopkins Individual Evidence Summary Tool (ARE ATTACHED).
Select one of the eight practice problems that burden the overall health of the United States. Conduct a library search for one qualitative research study addressing this problem. Appraise the qualitative research study using the Johns Hopkins Research Evidence Appraisal Tool.
(These eight practice problems include obesity; diabetes; heart disease; chronic obstructive pulmonary disease; mental illness; cancer; addiction, which includes alcohol and opioids; and patient safety, which includes any event in which an individual is harmed under our care). *I like addiction, but you can use any other.
After appraising and determining the Level of Evidence and Grade of Quality for your selected qualitative study, summarize your findings. Transfer your findings to the Johns Hopkins Individual Evidence Summary Tool. Complete each column including specific details about the qualitative study.
Include your completed Johns Hopkins Individual Evidence Summary Tool and your qualitative research study as an attachment with your initial post. Also, include a permalink for your selected qualitative research study with your initial discussion post. Confirm the link allows access to the full-text study article. Our faculty team will review both your research study and Johns Hopkins Individual Evidence Summary Tool and provide guidance.
Analyze the evidence summary of the selected qualitative research study to address the following.
Does the research design answer the research question? Explain your rationale.
Were the study sample participants representative? Why or why not?
Compare and contrast the study limitations in this study.
Based on this evidence summary, would you consider this qualitative research study as support for your selected practice problem? Explain your rationale.
Helpful Hints
when appraising your selected research study, begin with the John Hopkins Research Appraisal Tool. This tool guides you in a thorough critique and appraisal that leads to determining the level of evidence and grade of evidence for each individual study.
Once you are finished with the Johns Hopkins Individual Research Appraisal Tool, transfer your findings to the Johns Hopkins Evidence Summary Tool. Please note that you will add to this tool each week and this document is REQUIRED as part of your Week 6 paper. IMPORTANT: Do NOT alter this tool in any way whatsoever. For example, do not change this tool into portrait mode (keep it in landscape). Changing the design of this tool will cause you to lose unnecessary points in your weekly scores and also will affect your Week 6 paper grade.
Some tips for completing the Johns Hopkins Evidence Summary Tool include:
in column #2 insert the selected research study in APA format;
in Column #3 state the specific type of qualitative research design used (in week 1 you want to state whether this qualitative research study is grounded theory, phenomenology, descriptive, etc.);
describe in detail the sample and study findings in the appropriate columns, and
in the last column please insert both the level of evidence and grade of evidence.
Articles selected for appraisal in this course must be current – within the past five years.
If you are unsure if an article meets the research design requirement (qualitative research for week 1), please feel welcome to run it by me to make sure. I will be happy to help!
At the end of your appraisal, if you do not think the research study will inform your selected practice problem, you will need to substitute the publication with another study that does before Week 6. This is required for the Week 6 paper.
When you are ready to upload your initial post, please upload your completed Johns Hopkins Evidence Summary Tool along with a permalink to your two selected qualitative research studies.
A permalink is a full URL. If a permalink is not familiar to you here is a link to a tip sheet in Chamberlain’s Library: https://library.chamberlain.edu/HowToSearch/Permalinks (Links to an external site.) The other option is to upload your qualitative research study as a PDF. I will carefully read each of your studies along with reviewing your complete Johns Hopkins Evidence Summary Tool.
Finding Permalinks – How to Search the Library – Home at Chamberlain University – Home – Chamberlain Library Home – Home at Chamberlain University (Links to an external site.)
Step by Step Instructions. Permalinks in the OVID database are a little bit hidden, but you can find them by doing the following: 1. Go to the article in OVID, and look for Tools on the right-hand side of the page.. 2.
library.chamberlain.edu
Please Upload the JH Evidence Summary Tool.
Students are expected to follow proper APA formatting in each discussion and the Week 6 written assignment. You required to use APA 7th edition for all in-text citations and references, no exceptions.
Practice Question:
Date:
Article Number
Author and Date
Evidence Type
Sample, Sample Size, Setting
Findings That Help Answer the EBP Question
Observable Measures
Limitations
Evidence Level, Quality
· N/A
· N/A
· N/A
· N/A
· N/A
· N/A
· N/A
Attach a reference list with full citations of articles reviewed for this Practice question.
Johns Hopkins Nursing Evidence-Based Practice
Appendix G: Individual Evidence Summary Tool
The Johns Hopkins Hospital/ The Johns Hopkins University
1
Directions for Use of the Individual Evidence Summary Tool
Purpose
This form is used to document the results of evidence appraisal in preparation for evidence synthesis. The form provides the EBP team with documentation of the sources of evidence used, the year the evidence was published or otherwise communicated, the information gathered from each evidence source that helps the team answer the EBP question, and the level and quality of each source of evidence.
Article Number
Assign a number to each reviewed source of evidence. This organizes the individual evidence summary and provides an easy way to reference articles.
Author and Date
Indicate the last name of the first author or the evidence source and the publication/communication date. List both author/evidence source and date.
Evidence Type
Indicate the type of evidence reviewed (for example: RCT, meta-analysis, mixed methods, quaLitative, systematic review, case study, narrative literature review).
Sample, Sample Size, and Setting
Provide a quick view of the population, number of participants, and study location.
Findings That Help Answer the EBP Question
Although the reviewer may find many points of interest, list only findings that directly apply to the EBP question.
Observable Measures
QuaNtitative measures or variables are used to answer a research question, test a hypothesis, describe characteristics, or determine the effect, impact, or influence. QuaLitative evidence uses cases, context, opinions, experiences, and thoughts to represent the phenomenon of study.
Limitations
Include information that may or may not be within the text of the article regarding drawbacks of the piece of evidence. The evidence may list limitations, or it may be evident to you, as you review the evidence, that an important point is missed or the sample does not apply to the population of interest.
Evidence Level and Quality
Using information from the individual appraisal tools, transfer the evidence level and quality rating into this column.
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
©2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
Evidence level and quality rating:
Article title: Number:
Author(s): Publication date:
Journal:
Setting: Sample (composition and size):
Does this evidence address my EBP question?
Yes
No-Do not proceed with appraisal of this evidence
Is this study:
QuaNtitative (collection, analysis, and reporting of numerical data)
Measurable data (how many; how much; or how often) used to formulate facts, uncover patterns in
research, and generalize results from a larger sample population; provides observed effects of a
program, problem, or condition, measured precisely, rather than through researcher interpretation of
data. Common methods are surveys, face-to-face structured interviews, observations, and reviews of
records or documents. Statistical tests are used in data analysis.
Go to Section I: QuaNtitative
QuaLitative (collection, analysis, and reporting of narrative data)
Rich narrative documents are used for uncovering themes; describes a problem or condition from the
point of view of those experiencing it. Common methods are focus groups, individual interviews
(unstructured or semi structured), and participation/observations. Sample sizes are small and are
determined when data saturation is achieved. Data saturation is reached when the researcher identifies
that no new themes emerge and redundancy is occurring. Synthesis is used in data analysis. Often a
starting point for studies when little research exists; may use results to design empirical studies. The
researcher describes, analyzes, and interprets reports, descriptions, and observations from participants.
Go to Section II: QuaLitative
Mixed methods (results reported both numerically and narratively)
Both quaNtitative and quaLitative methods are used in the study design. Using both approaches, in
combination, provides a better understanding of research problems than using either approach alone.
Sample sizes vary based on methods used. Data collection involves collecting and analyzing both
quaNtitative and quaLitative data in a single study or series of studies. Interpretation is continual and
can influence stages in the research process.
Go to Section III: Mixed Methods
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
©2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
Section I: QuaNtitative
Level of Evidence (Study Design)
Is this a report of a single research study? Yes No
Go to B
1. Was there manipulation of an independent variable?
Yes No
2. Was there a control group?
Yes No
3. Were study participants randomly assigned to the intervention
and control groups? Yes No
If Yes to questions 1, 2, and 3, this is a randomized controlled trial (RCT) or
experimental study.
LEVEL I
If Yes to questions 1 and 2 and No to question 3 or Yes to question 1 and
No to questions 2 and 3, this is quasi-experimental.
(Some degree of investigator control, some manipulation of an independent variable,
lacks random assignment to groups, and may have a control group).
LEVEL II
If No to questions 1, 2, and 3, this is nonexperimental.
(No manipulation of independent variable; can be descriptive, comparative, or
correlational; often uses secondary data).
LEVEL III
Study Findings That Help Answer the EBP Question
Skip to the Appraisal of QuaNtitative Research Studies section
A
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
©2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
Section I: QuaNtitative (continued)
Is this a summary of multiple sources of research
evidence?
Yes
Continue
No
Use Appendix F
1. Does it employ a comprehensive search strategy and rigorous
appraisal method?
If this study includes research, nonresearch, and experiential
evidence, it is an integrative review (see Appendix F).
Yes
Continue
No
Use Appendix F
2. For systematic reviews and systematic reviews with meta-analysis
(see descriptions below):
a. Are all studies included RCTs? LEVEL I
b. Are the studies a combination of RCTs and quasi-experimental,
or quasi-experimental only?
LEVEL II
c. Are the studies a combination of RCTs, quasi-experimental, and
nonexperimental, or non- experimental only?
LEVEL III
A systematic review employs a search strategy and a rigorous appraisal method, but does not
generate an effect size.
A meta-analysis, or systematic review with meta-analysis, combines and analyzes results from
studies to generate a new statistic: the effect size.
Study Findings That Help Answer the EBP Question
Skip to the Appraisal of Systematic Review (With or Without a Meta-Analysis) section
B
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
©2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
Appraisal of QuaNtitative Research Studies
Does the researcher identify what is known and not known
about the problem and how the study will address any gaps in
knowledge?
Yes No
Was the purpose of the study clearly presented?
Yes No
Was the literature review current (most sources within the past
five years or a seminal study)? Yes No
Was sample size sufficient based on study design and rationale?
Yes No
If there is a control group:
Were the characteristics and/or demographics similar in
both the control and intervention groups?
Yes No
N/A
If multiple settings were used, were the settings
similar? Yes No
N/A
Were all groups equally treated except for the
intervention group(s)? Yes No
N/A
Are data collection methods described clearly?
Yes No
Were the instruments reliable (Cronbach’s [alpha] > 0.70)?
Yes No N/A
Was instrument validity discussed?
Yes No N/A
If surveys or questionnaires were used, was the response
rate > 25%? Yes No
N/A
Were the results presented clearly?
Yes No
If tables were presented, was the narrative consistent with the
table content? Yes No
N/A
Were study limitations identified and addressed?
Yes No
Were conclusions based on results?
Yes No
Complete the Quality Rating for QuaNtitative Studies section
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
©2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
Appraisal of Systematic Review (With or Without Meta-Analysis)
Were the variables of interest clearly identified?
Yes No
Was the search comprehensive and reproducible?
Key search terms stated Yes No
Multiple databases searched and identified
Yes No
Inclusion and exclusion criteria stated
Yes No
Was there a flow diagram that included the number of studies eliminated
at each level of review? Yes No
Were details of included studies presented (design, sample, methods,
results, outcomes, strengths, and limitations)? Yes No
Were methods for appraising the strength of evidence (level and quality)
described? Yes No
Were conclusions based on results?
Yes No
Results were interpreted
Yes No
Conclusions flowed logically from the interpretation and systematic
review question Yes No
Did the systematic review include a section addressing limitations and
how they were addressed? Yes No
Complete the Quality Rating for QuaNtitative Studies section (below)
Quality Rating for QuaNtitative Studies
Circle the appropriate quality rating below:
A High quality: Consistent, generalizable results; sufficient sample size for the study design; adequate
control; definitive conclusions; consistent recommendations based on comprehensive literature review
that includes thorough reference to scientific evidence.
B Good quality: Reasonably consistent results; sufficient sample size for the study design; some control,
and fairly definitive conclusions; reasonably consistent recommendations based on fairly comprehensive
literature review that includes some reference to scientific evidence.
C Low quality or major flaws: Little evidence with inconsistent results; insufficient sample size for the
study design; conclusions cannot be drawn.
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
©2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
Section II: QuaLitative
Level of Evidence (Study Design)
Is this a report of a single research study?
Yes
this is
Level III
No
go to II B
Study Findings That Help Answer the EBP Question
Complete the Appraisal of Single QuaLitative Research Study section (below)
Appraisal of a Single QuaLitative Research Study
Was there a clearly identifiable and articulated:
Purpose?
❑ Yes ❑ No
Research question? ❑ Yes ❑ No
Justification for method(s) used? ❑ Yes ❑ No
Phenomenon that is the focus of the research? ❑ Yes ❑ No
Were study sample participants representative? ❑ Yes ❑ No
Did they have knowledge of or experience with the research area? ❑ Yes ❑ No
Were participant characteristics described? ❑ Yes ❑ No
Was sampling adequate, as evidenced by achieving saturation of data? ❑ Yes ❑ No
Data analysis:
Was a verification process used in every step by checking and confirming
with participants the trustworthiness of analysis and interpretation?
❑ Yes
❑ No
Was there a description of how data were analyzed (i.e., method), by
computer or manually?
❑ Yes ❑ No
Do findings support the narrative data (quotes)? ❑ Yes ❑ No
Do findings flow from research question to data collected to analysis undertaken? ❑ Yes ❑ No
Are conclusions clearly explained? ❑ Yes ❑ No
A
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
©2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
Skip to the Quality Rating for QuaLitative Studies section
For summaries of multiple quaLitative research studies
(meta-synthesis), was a comprehensive search strategy and
rigorous appraisal method used?
Yes
Level III
No
go to Appendix F
Study Findings That Help Answer the EBP Question
Complete the Appraisal of Meta-Synthesis Studies section (below)
Appraisal of Meta-Synthesis Studies
Were the search strategy and criteria for selecting primary studies clearly defined? ❑ Yes ❑ No
Were findings appropriate and convincing? ❑ Yes ❑ No
Was a description of methods used to:
Compare findings from each study?
❑ Yes ❑ No
Interpret data? ❑ Yes ❑ No
Did synthesis reflect: ❑ Yes ❑ No
New insights? ❑ Yes ❑ No
Discovery of essential features of phenomena? ❑ Yes ❑ No
A fuller understanding of the phenomena? ❑ Yes ❑ No
Was sufficient data presented to support the interpretations? ❑ Yes ❑ No
Complete the Quality Rating for QuaLititative Studies section (below)
B
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
©2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
1 https://www.york.ac.uk/crd/SysRev/!SSL!/WebHelp/6_4_ASSESSMENT_OF_QUALITATIVE_RESEARCH.htm
2 Adapted from Polit & Beck (2017).
Quality Rating for QuaLitative Studies
Circle the appropriate quality rating below:
No commonly agreed-on principles exist for judging the quality of quaLitative studies. It is a subjective
process based on the extent to which study data contributes to synthesis and how much information is
known about the researchers’ efforts to meet the appraisal criteria.
For meta-synthesis, there is preliminary agreement that quality assessments should be made before
synthesis to screen out poor-quality studies1.
A/B High/Good quality is used for single studies and meta-syntheses2.
The report discusses efforts to enhance or evaluate the quality of the data and the overall inquiry in
sufficient detail; and it describes the specific techniques used to enhance the quality of the inquiry.
Evidence of some or all of the following is found in the report:
Transparency: Describes how information was documented to justify decisions, how data were
reviewed by others, and how themes and categories were formulated.
Diligence: Reads and rereads data to check interpretations; seeks opportunity to find multiple
sources to corroborate evidence.
Verification: The process of checking, confirming, and ensuring methodologic coherence.
Self-reflection and self-scrutiny: Being continuously aware of how a researcher’s experiences,
background, or prejudices might shape and bias analysis and interpretations.
Participant-driven inquiry: Participants shape the scope and breadth of questions; analysis and
interpretation give voice to those who participated.
Insightful interpretation: Data and knowledge are linked in meaningful ways to relevant literature.
C Lower-quality studies contribute little to the overall review of findings and have few, if any, of the
features listed for High/Good quality.
http://www.york.ac.uk/crd/SysRev/!SSL!/WebHelp/6_4_ASSESSMENT_OF_QUALITATIVE_RESEARCH.htm
http://www.york.ac.uk/crd/SysRev/!SSL!/WebHelp/6_4_ASSESSMENT_OF_QUALITATIVE_RESEARCH.htm
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
©2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
Section III: Mixed Methods
Level of Evidence (Study Design)
You will need to appraise both the quaNtitative and quaLitative parts of the study independently, before
appraising the study in its entirety.
1. Evaluate the quaNitative part of the study using Section I. Level Quality
Insert here the level of evidence and overall quality for this part:
2. Evaluate the quaLitative part of the study using Section II. Level Quality
Insert here the level of evidence and overall quality for this part:
3. To determine the level of evidence, circle the appropriate study design:
Explanatory sequential designs collect quaNtitative data first, followed by the quaLitative data; and their
purpose is to explain quaNtitative results using quaLitative findings. The level is determined based on the
level of the quaNtitative part.
Exploratory sequential designs collect quaLitative data first, followed by the quaNtitative data; and their
purpose is to explain quaLitative findings using the quaNtitative results. The level is determined based on
the level of the quaLitative part, and it is always Level III.
Convergent parallel designs collect the quaLitative and quaNtitative data concurrently for the purpose of
providing a more complete understanding of a phenomenon by merging both datasets. These designs are
Level III.
Multiphasic designs collect quaLitative and quaNtitative data over more than one phase, with each
phase informing the next phase. These designs are Level III.
Study Findings That Help Answer the EBP Question
Complete the Appraisal of Mixed Methods Studies section (below)
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
©2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
3 National Collaborating Centre for Methods and Tools. (2015). Appraising Qualitative, Quantitative, and Mixed Methods Studie s included in Mixed Studies Reviews: The MMAT.
Hamilton, ON: McMaster University. (Updated 20 July, 2015) Retrieved from http://www.nccmt.ca/ resources/search/232
Appraisal of Mixed Methods Studies3
Was the mixed-methods research design relevant to address the quaNtitative
and quaLitative research questions (or objectives)?
❑ Yes ❑ No ❑ N/A
Was the research design relevant to address the quaNtitative and quaLitative
aspects of the mixed-methods question (or objective)?
❑ Yes ❑ No ❑ N/A
For convergent parallel designs, was the integration of quaNtitative and
quaLitative data (or results) relevant to address the research question or
objective?
❑ Yes ❑ No ❑ N/A
For convergent parallel designs, were the limitations associated with the
integration (for example, the divergence of quaLitative and quaNtitative data or
results) sufficiently addressed?
❑ Yes ❑ No ❑ N/A
Complete the Quality Rating for Mixed-Method Studies section (below)
Quality Rating for Mixed-Methods Studies
Circle the appropriate quality rating below
A High quality: Contains high-quality quaNtitative and quaLitative study components; highly relevant
study design; relevant integration of data or results; and careful consideration of the limitations of the
chosen approach.
B Good quality: Contains good-quality quaNtitative and quaLitative study components; relevant study
design; moderately relevant integration of data or results; and some discussion of limitations of
integration.
C Low quality or major flaws: Contains low quality quaNtitative and quaLitative study components;
study design not relevant to research questions or objectives; poorly integrated data or results; and no
consideration of limits of integration.
http://www.nccmt.ca/