Evidence-Based Project, Part 4: Critical Appraisal Of Research Assignment

Author:

April 1st, 2022

Evaluation Table

Table of Contents

Use this document to complete the evaluation table requirement of the Module 4 Assessment,Evidence-Based Project, Part 4A: Critical Appraisal of Research

Full *citation* of selected article	Article #1	Article #2	Article #3	Article #4

*Conceptual Framework*Describe the theoretical basis for the study
*Design/Method* Describe the designand how the studywas carried out
*Sample/Setting*The number andcharacteristics ofpatients,attrition rate, etc.
*Major Variables Studied*List and define dependent and independent variables
*Measurement*Identify primary statistics used to answer clinical questions
*Data Analysis*Statistical orqualitativefindings
*Findings and Recommendations*General findings and recommendations of the research
*Appraisal*Describe the general worth of this research to practice. What are the strengths and limitations of study? What are the risks associated with implementation of the suggested practices or processes detailed in the research? What is the feasibility ofuse in your practice?
General Notes/Comments

Levels of Evidence Table

Use this document to complete the levels of evidence table requirement of the Module 4 Assessment,Evidence-Based Project, Part 4A: Critical Appraisal of Research

*Author* and *year* of selected article	Article #1	Article #2	Article #3	Article #4

*Study Design*Theoretical basis for the study
*Sample/Setting*The number andcharacteristics ofpatients
Evidence Level *(I, II, or III)
*Outcomes*
General Notes/Comments

* Evidence Levels:

· Level I

Experimental, randomized controlled trial (RCT), systematic review RTCs with or without meta-analysis

· Level II

Quasi-experimental studies, systematic review of a combination of RCTs and quasi-experimental studies, or quasi-experimental studies only, with or without meta-analysis

· Level III

Nonexperimental, systematic review of RCTs, quasi-experimental with/without meta-analysis, qualitative, qualitative systematic review with/without meta-synthesis

· Level IV

Respected authorities’ opinions, nationally recognized expert committee/consensus panel reports based on scientific evidence

· Level V

Literature reviews, quality improvement, program evaluation, financial evaluation, case reports, nationally recognized expert(s) opinion based on experiential evidence

Outcomes Synthesis Table

Use this document to complete the outcomes synthesis table requirement of the Module 4 Assessment,Evidence-Based Project, Part 4A: Critical Appraisal of Research

*Author* and *year* of selected article	Article #1	Article #2	Article #3	Article #4

*Sample/Setting*The number andcharacteristics ofpatients
*Outcomes*
*Key Findings*
*Appraisal and Study Quality*
General Notes/Comments

Critical Appraisal Tools Worksheet Template

Assignment: Evidence-Based Project, Part 4: Critical Appraisal of Research

Realtors rely on detailed property appraisals—conducted using appraisal tools—to assign market values to houses and other properties. These values are then presented to buyers and sellers to set prices and initiate offers.

Research appraisal is not that different. The critical appraisal process utilizes formal appraisal tools to assess the results of research to determine value to the context at hand. Evidence-based practitioners often present these findings to make the case for specific courses of action.

In this Assignment, you will use appraisal tools to conduct a critical appraisal of published research. You will then present the results of your efforts.

To Prepare:

· Review the Resources and consider the importance of critically appraising research evidence.

· Reflect on the four peer-reviewed articles you selected in Module 2 and analyzed in Module 3.

· 1.Client Retention: Caul,M. (2008). Treatment Improvement Protocol

· 2.Relapse and Continued Substance Use: Irvin,A. (1999). Treatment Improvement Protocol

· 3. Substance Use by Family Members: Johnson,M (1999)

· 4.Group Work Issues: CSAT (2005)

· Review and download the Critical Appraisal Tools document provided in the Resources.

The Assignment (Evidence-Based Project)

Part 4A: Critical Appraisal of Research ATTACHED

Conduct a critical appraisal of the four peer-reviewed articles you selected and analyzed by completing the Critical Appraisal Tools document. Be sure to include:

· An evaluation table

· A levels of evidence table

· An outcomes synthesis table

Part 4B: Critical Appraisal of Research

Based on your appraisal, in a 1-2-page critical appraisal, suggest a best practice that emerges from the research you reviewed. Briefly explain the best practice, justifying your proposal with APA citations of the research.

Resources

Fineout-Overholt, E., Melnyk, B. M., Stillwell, S. B., & Williamson, K. M. (2010a). Evidence-based practice step by step: Critical appraisal of the evidence: Part I. American Journal of Nursing, 110(7), 47–52. doi:10.1097/01.NAJ.0000383935.22721.9c

Fineout-Overholt, E., Melnyk, B. M., Stillwell, S. B., & Williamson, K. M. (2010b). Evidence-based practice, step by step: Critical appraisal of the evidence: Part II: Digging deeper—examining the “keeper” studies. American Journal of Nursing, 110(9), 41–48. doi:10.1097/01.NAJ.0000388264.49427.f9

Fineout-Overholt, E., Melnyk, B. M., Stillwell, S. B., & Williamson, K. M. (2010c). Evidence-based practice, step by step: Critical appraisal of the evidence: Part III: The process of synthesis: Seeing similarities and differences across the body of evidence. American Journal of Nursing, 110(11), 43–51. doi: 10.1097/01.NAJ.0000390523.99066.b5

Williamson, K. M. (2009). Evidence-based practice: Critical appraisal of qualitative evidence. Journal of the American Psychiatric Nurses Association, 15(3), 202–207. doi:10.1177/1078390309338733

· Grid View

· List View

	Excellent
Part 4A: Critical Appraisal of Research Conduct a critical appraisal of the four peer-reviewed articles you selected and analyzed by completing the Critical Appraisal Tools document. Be sure to include: · An evaluation table · A levels of evidence table · An outcomes synthesis table	45 (45%) – 50 (50%)The critical appraisal accurately and clearly provides a detailed evaluation table. The responses provide a detailed, specific, and accurate evaluation of each of the peer-reviewed articles selected. The critical appraisal accurately clearly provides a detailed levels of evidence table. The levels of evidence are accurate and fully aligned to the peer-reviewed articles selected. The critical appraisal accurately and clearly provides a detailed outcomes synthesis table. The outcomes synthesis accurately describes in detail the relevance of the peer-reviewed articles selected and is fully aligned to each of the peer-reviewed articles.
Part 4B: Evidence-Based Best Practices Based on your appraisal, suggest a best practice that emerges from the research you reviewed. Briefly explain the best practice, justifying your proposal with APA citations of the research.	32 (32%) – 35 (35%)The responses accurately and clearly suggest a detailed best practice that is fully aligned to the research reviewed. The responses accurately and clearly explain in detail the best practice, with sufficient justification of why this represents a best practice in the field. The responses provide a complete, detailed, and specific synthesis of two outside resources reviewed on the best practice explained. The response fully integrates at least two outside resources and two or three course-specific resources that fully support the responses provided. Accurate, complete, and full APA citations are provided for the research reviewed.
Written Expression and Formatting—Paragraph Development and Organization: Paragraphs make clear points that support well-developed ideas, flow logically, and demonstrate continuity of ideas. Sentences are carefully focused—neither long and rambling nor short and lacking substance. A clear and comprehensive purpose statement and introduction is provided which delineates all required criteria.	5 (5%) – 5 (5%)Paragraphs and sentences follow writing standards for flow, continuity, and clarity. A clear and comprehensive purpose statement, introduction, and conclusion is provided which delineates all required criteria.
Written Expression and Formatting—English Writing Standards: Correct grammar, mechanics, and proper punctuation.	5 (5%) – 5 (5%)Uses correct grammar, spelling, and punctuation with no errors.
Written Expression and Formatting—The paper follows correct APA format for title page, headings, font, spacing, margins, indentations, page numbers, running head, parenthetical/in-text citations, and reference list.	5 (5%) – 5 (5%)Uses correct APA format with no errors.

Name: NURS_6052_Module04_Week07_Assignment_Rubric

Critical Appraisal of the Evidence: Part I

In May’s evidence-based prac- tice (EBP) article, Rebecca R., our hypothetical staff nurse, and Carlos A., her hospital’s ex- pert EBP mentor, learned how to search for the evidence to answer their clinical question (shown here in PICOT format): “In hos pitalized adults (P), how does a rapid response team (I) compared with no rapid response team (C) affect the number of cardiac ar rests (O) and unplanned admis sions to the ICU (O) during a threemonth period (T)?” With the help of Lynne Z., the hospi- tal librarian, Rebecca and Car- los searched three databases, PubMed, the Cumulative Index of Nursing and Allied Health Literature (CINAHL), and the Cochrane Database of Systematic Reviews. They used keywords from their clinical question, in- cluding ICU, rapid response team, cardiac arrest, and un planned ICU admissions, as well as the following synonyms: failure to rescue, never events, medical emergency teams, rapid response systems, and code blue. Whenever terms from a

database’s own indexing lan- guage, or controlled vocabulary, matched the keywords or syn- onyms, those terms were also searched. At the end of the data- base searches, Rebecca and Car- los chose to retain 18 of the 18 studies found in PubMed; six of the 79 studies found in CINAHL; and the one study found in the Cochrane Database of System- atic Reviews, because they best answered the clinical question.

As a final step, at Lynne’s rec- ommendation, Rebecca and Car- los conducted a hand search of the reference lists of each study they retained looking for any rele- vant studies they hadn’t found in their original search; this process is also called the ancestry method. The hand search yielded one ad- ditional study, for a total of 26.

RAPID CRITICAL APPRAISAL

The next time Rebecca and Car- los meet, they discuss the next step in the EBP process—critically appraising the 26 studies. They obtain copies of the studies by printing those that are immedi- ately available as full text through

library subscription or those flagged as “free full text” by a database or journal’s Web site. Others are available through in- terlibrary loan, when another hospital library shares its articles with Rebecca and Carlos’s hospi- tal library.

Carlos explains to Rebecca that the purpose of critical appraisal isn’t solely to find the flaws in a study, but to determine its worth to practice. In this rapid critical appraisal (RCA), they will review each study to determine

• its level of evidence. • how well it was conducted. • how useful it is to practice.

Once they determine which studies are “keepers,” Rebecca and Carlos will move on to the final steps of critical appraisal: evaluation and synthesis (to be discussed in the next two install- ments of the series). These final steps will determine whether overall findings from the evi- dence review can help clinicians improve patient outcomes.

Rebecca is a bit apprehensive because it’s been a few years since she took a research class. She shares her anxiety with Chen M., a fellow staff nurse, who says she never studied research in school but would like to learn; she asks if she can join Carlos and Rebecca’s EBP team. Chen’s spirit of inquiry encourages Re- becca, and they talk about the opportunity to learn that this project affords them. Together they speak with the nurse man- ager on their medical–surgical unit, who agrees to let them use their allotted continuing educa- tion time to work on this project, after they discuss their expecta- tions for the project and how its outcome may benefit the patients, the unit staff, and the hospital.

Learning research terminol- ogy. At the first meeting of the

new EBP team, Carlos provides Rebecca and Chen with a glossary of terms so they can learn basic research terminology, such as sam ple, independent variable, and de pendent variable. The glossary also defines some of the study de- signs the team is likely to come across in doing their RCA, such as systematic review, randomized controlled trial, and cohort, qual- itative, and descriptive studies. (For the definitions of these terms and others, see the glossaries pro- vided by the Center for the Ad- vancement of Evidence-Based Practice at the Arizona State Uni- versity College of Nursing and Health Innovation [http://nursing andhealth.asu.edu/evidence-based- practice/resources/glossary.htm]

and the Boston University Medi- cal Center Alumni Medical Li- brary [http://medlib.bu.edu/ bugms/content.cfm/content/ ebmglossary.cfm#R].)

Determining the level of evi- dence. The team begins to divide the 26 studies into categories ac- cording to study design. To help in this, Carlos provides a list of several different study designs (see Hierarchy of Evidence for Intervention Studies). Rebecca, Carlos, and Chen work together to determine each study’s design by reviewing its abstract. They also create an “I don’t know” pile of studies that don’t appear to fit a specific design. When they find studies that don’t actively answer the clinical question but

Hierarchy of Evidence for Intervention Studies

Type of evidence	Level of evidence	Description
Systematic review or meta-analysis	I	A synthesis of evidence from all relevant randomized controlled trials.
Randomized con- trolled trial	II	An experiment in which subjects are randomized to a treatment group or control group.
Controlled trial with- out randomization	III	An experiment in which subjects are nonrandomly assigned to a treatment group or control group.
Case-control or cohort study	IV	Case-control study: a comparison of subjects with a condition (case) with those who don’t have the condition (control) to determine characteristics that might predict the condition.Cohort study: an observation of a group(s) (cohort[s]) to determine the development of an outcome(s) such as a disease.
Systematic review of qualitative or descrip- tive studies	V	A synthesis of evidence from qualitative or descriptive studies to answer a clinical question.
Qualitative or de- scriptive study	VI	Qualitative study: gathers data on human behavior to understand why and how decisions are made.Descriptive study: provides background information on the what, where, and when of a topic of interest.
Expert opinion or consensus	VII	Authoritative opinion of expert committee.

Adapted with permission from Melnyk BM, Fineout-Overholt E, editors. Evidence-based practice in nursing and healthcare: a guide to best practice [forthcoming]. 2nd ed. Philadelphia: Wolters Kluwer Health/Lippincott Williams and Wilkins.

48 AJN ▼ July 2010 ▼ Vol. 110, No. 7

ajnonline.com

Critical Appraisal Guide for Quantitative Studies

1. Why was the study done? • Was there a clear explanation of the purpose of the study and, if so, what was it?

2. What is the sample size? • Were there enough people in the study to establish that the findings did not occur by chance?

3. Are the instruments of the major variables valid and reliable? • How were variables defined? Were the instruments designed to measure a concept valid (did

they measure what the researchers said they measured)? Were they reliable (did they measure a

concept the same way every time they were used)?

4. How were the data analyzed?

• What statistics were used to determine if the purpose of the study was achieved?

5. Were there any untoward events during the study?

• Did people leave the study and, if so, was there something special about them?

6. How do the results fit with previous research in the area?

• Did the researchers base their work on a thorough literature review?

7. What does this research mean for clinical practice?

• Is the study purpose an important clinical issue?

may inform thinking, such as descriptive research, expert opin- ions, or guidelines, they put them aside. Carlos explains that they’ll be used later to support Rebecca’s case for having a rapid response team (RRT) in her hospital, sh- ould the evidence point in that direction.

After the studies—including those in the “I don’t know” group—are categorized, 15 of the original 26 remain and will be included in the RCA: three systematic reviews that include one meta-analysis (Level I evi- dence), one randomized con- trolled trial (Level II evidence), two cohort studies (Level IV evi- dence), one retrospective pre- post study with historic controls (Level VI evidence), four preex- perimental (pre-post) interven- tion studies (no control group) (Level VI evidence), and four EBP implementation projects (Level VI evidence). Carlos reminds Rebecca and Chen that Level I evidence—a systematic review

of randomized controlled trials

or a meta-analysis—is the most reliable and the best evidence to answer their clinical question.

Using a critical appraisal guide. Carlos recommends that the team use a critical appraisal checklist (see Critical Appraisal Guide for Quantitative Studies) to help evaluate the 15 studies. This checklist is relevant to all studies and contains questions about the essential elements of research (such as, purpose of the study, sample size, and major variables).

The questions in the critical ap- praisal guide seem a little strange to Rebecca and Chen. As they re- view the guide together, Carlos explains and clarifies each ques- tion. He suggests that as they try to figure out which are the essen- tial elements of the studies, they focus on answering the first three questions: Why was the study done? What is the sample size? Are the instruments of the major variables valid and reliable? The remaining questions will be ad- dressed later on in the critical

appraisal process (to appear in future installments of this series).

Creating a study evaluation table. Carlos provides an online template for a table where Re- becca and Chen can put all the data they’ll need for the RCA. Here they’ll record each study’s essential elements that answer the three questions and begin to ap- praise the 15 studies. (To use this template to create your own eval- uation table, download the Eval uation Table Template at http:// links.lww.com/AJN/A10.)

EXTRACTING THE DATA

Starting with level I evidence studies and moving down the hierarchy list, the EBP team takes each study and, one by one, finds and enters its essential elements into the first five columns of

the evaluation table (see Table 1; to see the entire table with all 15 studies, go to http://links. lww.com/AJN/A11). The team discusses each element as they enter it, and tries to determine if it meets the criteria of the critical

ajn@wolterskluwer.com

AJN ▼ July 2010 ▼ Vol. 110, No. 7 49

50 AJN ▼ July 2010 ▼ Vol. 110, No. 7 ajnonline.com Table 1. Evaluation Table, Phase I First Author (Year) Conceptual Design/Method Framework Sample/Setting Major Variables Studied (and Their Definitions) Measure- ment Data Analysis Findings Appraisal: Worth to Practice Chan PS, et al. None SR Purpose: effect of RRT on N = 18 studies IV: RRT DV1: HMR DV2: CR Arch Intern Med HMR and CR • Searched 5 databases Setting: acute care hos- pitals; 13 adult, 5 peds 2010;170(1):18-26. McGaughey J, et al. None SR (Cochrane review) Purpose: effect of RRT N = 2 studies 24 adult hospitals Attrition: NR IV: RRT DV1: HMR Cochrane Database Syst Rev 2007;3: CD005529. on HMR • Searched 6 databases Winters BD, et al. None SR Purpose: effect of RRT on N = 8 studies Average no. beds: 500 Attrition: NR IV: RRT DV1: HMR DV2: CR Crit Care Med HMR and CR • Searched 3 databases 2007;35(5): 1238-43. from 1990-2005 • Included only studies with a control group Hillman K, et al. Lancet 2005; 365(9477): 2091-7. None RCT Purpose: effect of RRT on N = 23 hospitals Average no. beds: 340 • Intervention group IV: RRT protocol for 6 months • 1 AP • 1ICUorEDRN DV1: HMR (unexpected deaths, excluding DNRs) DV2: CR (excluding DNRs) HMR CR rates of UICUA Note: • Criteria for from 1950-2008, and “grey literature” from MD conferences Average no. beds: NR Attrition: NR • Included only studies with a control group from 1990-2006 • Excluded all but 2 RCTs CR, HMR, and UICUA (n=12) • Control group activating RRT (n = 11) Setting: Australia Attrition: none DV3: UICUA Shaded columns indicate where data will be entered in future installments of the series. AP = attending physician; CR = cardiopulmonary arrest or code rates; DNR = do not resuscitate; DV = dependent variable; ED = emergency department; HMR: hospital-wide mor- tality rates; ICU = intensive care unit; IV = independent variable; MD = medical doctor; NR = not reported; Peds = pediatric; RCT = randomized controlled trial; RN = registered nurse; RRT = rapid response team; SR = systematic review; UICUA = unplanned ICU admissions.

appraisal guide. These elements— such as purpose of the study, sam- ple size, and major variables—are typical parts of a research report and should be presented in a pre- dictable fashion in every study so that the reader understands what’s being reported.

suggests they leave the column in. He says they can further discuss this point later on in the process when they synthesize the studies’ findings. As Rebecca and Chen review each study, they enter its citation in a separate reference list so that they won’t have to create

findings, not to compare them with other like studies. Rebecca realizes that she enjoys this kind of conversation, in which she and Chen have a voice and can contribute to a deeper under- standing of how research impacts practice.

As Rebecca and Chen con- tinue to enter data into the table, they begin to see similarities and differences across studies. They mention this to Carlos, who tells them they’ve begun the process of synthesis! Both nurses are en- couraged by the fact that they’re learning this new skill.

The MERIT trial is next in the stack of studies and it’s a good trial to use to illustrate this phase of the RCA process. Set in Aus- tralia, the MERIT trial1 examined whether the introduction of an RRT (called a medical emergency team or MET in the study) would reduce the incidence of cardiac arrest, unplanned admissions to the ICU, and death in the hospi- tals studied. See Table 1 to follow along as the EBP team finds and enters the trial data into the table.

Design/Method. After Rebecca and Chen enter the citation infor- mation and note the lack of a con- ceptual framework, they’re ready to fill in the “Design/Method” column. First they enter RCT for randomized controlled trial, which they find in both the study title and introduction. But MERIT is called a “cluster-randomised controlled trial,” and cluster is a term they haven’t seen before. Carlos explains that it means that hospitals, not individuals or pa- tients, were randomly assigned to the RRT. He says that the likely reason the researchers chose to randomly assign hospitals is that if they had randomly assigned individual patients or units, oth- ers in the hospital might have heard about the RRT and poten- tially influenced the outcome.

Usually the important information in a study can be found in the abstract.

As the EBP team continues to review the studies and fill in the evaluation table, they realize that it’s taking about 10 to 15 minutes per study to locate and enter the information. This may be because when they look for a description of the sample, for example, it’s important that they note how the sample was obtained, how many patients are included, other char- acteristics of the sample, as well as any diagnoses or illnesses the sample might have that could be important to the study outcome. They discuss with Carlos the like- lihood that they’ll need a few ses- sions to enter all the data into the table. Carlos responds that the more studies they do, the less time it will take. He also says that it takes less time to find the information when study reports are clearly written. He adds that usually the important informa- tion can be found in the abstract.

Rebecca and Chen ask if it would be all right to take out the “Conceptual Framework” column, since none of the stud- ies they’re reviewing have con- ceptual frameworks (which help guide researchers as to how a study should proceed). Carlos replies that it’s helpful to know that a study has no framework underpinning the research and

this list at the end of the process. The reference list will be shared with colleagues and placed at the end of any RRT policy that re- sults from this endeavor.

Carlos spends much of his time answering Rebecca’s and Chen’s questions concerning how to phrase the information they’re entering in the table. He suggests that they keep it simple and con- sistent. For example, if a study indicated that it was implement- ing an RRT and hoped to see a change in a certain outcome, the nurses could enter “change in [the outcome] after RRT” as the purpose of the study. For studies examining the effect of an RRT on an outcome, they could say as the purpose, “effect of RRT on [the outcome].” Using the same words to describe the same pur- pose, even though it may not have been stated exactly that way in the study, can help when they compare studies later on.

Rebecca and Chen find it frus- trating that the study data are not always presented in the same way from study to study. They ask Carlos why the authors or journals wouldn’t present similar information in a similar manner. Carlos explains that the purpose of publishing these studies may have been to disseminate the

ajn@wolterskluwer.com

AJN ▼ July 2010 ▼ Vol. 110, No. 7 51

To randomly assign hospitals (instead of units or patients) to the intervention and comparison groups is a cleaner research de- sign.

the RRTs were activated and pro- vided their protocol for calling the RRTs. However, these elements might be helpful to the EBP team later on when they make decisions

continue the work—as long as Carlos is there to help.

In applying these principles for evaluating research studies to your own search for the evi- dence to answer your PICOT question, remember that this se- ries can’t contain all the available information about research meth- odology. Fortunately, there are many good resources available in books and online. For example, to find out more about sample size, which can affect the likeli- hood that researchers’ results oc- cur by chance (a random finding) rather than that the intervention brought about the expected out- come, search the Web using terms that describe what you want to know. If you type sample size findings by chance in a search en- gine, you’ll find several Web sites that can help you better under- stand this study essential.

Be sure to join the EBP team in the next installment of the se- ries, “Critical Appraisal of the Evidence: Part II,” when Rebecca and Chen will use the MERIT trial to illustrate the next steps in the RCA process, complete the rest of the evaluation table, and dig a little deeper into the studies in order to detect the “keepers.” ▼

Ellen FineoutOverholt is clinical profes sor and director of the Center for the Advancement of EvidenceBased Practice at Arizona State University in Phoenix, where Bernadette Mazurek Melnyk

is dean and distinguished foundation professor of nursing, Susan B. Stillwell is clinical associate professor and pro gram coordinator of the Nurse Educator EvidenceBased Practice Mentorship Program, and Kathleen M. Williamson is associate director of the Center for the Advancement of EvidenceBased Practice. Contact author: Ellen FineoutOverholt, ellen.fineoutoverholt@asu.edu.

REFERENCE

1.Hillman K, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised con- trolled trial. Lancet 2005;365(9477): 2091-7.

Keep the data in the table consistent by using simple, inclusive terminology.

To keep the study purposes consistent among the studies in the RCA, the EBP team uses inclu- sive terminology they developed after they noticed that different trials had different ways of de- scribing the same objectives. Now they write that the purpose of the MERIT trial is to see if an RRT can reduce CR, for cardiopulmo- nary arrest or code rates, HMR, for hospital-wide mortality rates, and UICUA for unplanned ICU admissions. They use those same terms consistently throughout the evaluation table.

Sample/Setting. A total of 23 hospitals in Australia with an average of 340 beds per hospi- tal is the study sample. Twelve hospitals had an RRT (the inter- vention group) and 11 hospitals didn’t (the control group).

Major Variables Studied. The independent variable is the vari- able that influences the outcome (in this trial, it’s an RRT for six months). The dependent vari- able is the outcome (in this case, HMR, CR, and UICUA). In this trial, the outcomes didn’t include do-not-resuscitate data. The RRT was made up of an attending phy- sician and an ICU or ED nurse.

While the MERIT trial seems to perfectly answer Rebecca’s PICOT question, it contains ele- ments that aren’t entirely relevant, such as the fact that the research- ers collected information on how

about implementing an RRT in their hospital. So that they can come back to this information, they place it in the last column, “Appraisal: Worth to Practice.”

After reviewing the studies to make sure they’ve captured the essential elements in the evalua- tion table, Rebecca and Chen still feel unsure about whether the in- formation is complete. Carlos reminds them that a system-wide practice change—such as the change Rebecca is exploring, that of implementing an RRT in her hospital—requires careful consid- eration of the evidence and this is only the first step. He cautions them not to worry too much about perfection and to put their efforts into understanding the information in the studies. He re- minds them that as they move on to the next steps in the critical appraisal process, and learn even more about the studies and proj- ects, they can refine any data in the table. Rebecca and Chen feel uncomfortable with this uncer- tainty but decide to trust the pro- cess. They continue extracting data and entering it into the table even though they may not com- pletely understand what they’re entering at present. They both realize that this will be a learn- ing opportunity and, though the learning curve may be steep at times, they value the outcome of improving patient care enough to

52 AJN ▼ July 2010 ▼ Vol. 110, No. 7

ajnonline.com

Critical Appraisal of the Evidence: Part II

Digging deeper—examining the “keeper” studies.

By Ellen Fineout-Overholt, PhD, RN, FNAP, FAAN, Bernadette Mazurek Melnyk, PhD, RN, CPNP/PMHNP,

FNAP, FAAN, Susan B. Stillwell, DNP, RN, CNE, and Kathleen M. Williamson, PhD, RN

This is the sixth article in a series from the Arizona State University College of Nursing and Health Innovation’s Center for the Advancement of Evidence-Based Practice. Evidence-based practice (EBP) is a problem-solving approach to the delivery of health care that integrates the best evidence from studies and patient care data with clinician expertise and patient preferences and values. When delivered in a context of caring and in a supportive organizational culture, the highest quality of care and best patient outcomes can be achieved.

The purpose of this series is to give nurses the knowledge and skills they need to implement EBP consistently, one step at a time. Articles will appear every two months to allow you time to incorporate information as you work toward implementing EBP at your institution. Also, we’ve scheduled “Chat with the Authors” calls every few months to provide a direct line to the experts to help you resolve questions. Details about how to participate in the next call will be pub- lished with November’s Evidence-Based Practice, Step by Step.

In July’s evidence-based prac- tice (EBP) article, Rebecca R., our hypothetical staff nurse, Carlos A., her hospital’s expert EBP mentor, and Chen M., Rebecca’s nurse colleague, collected the evidence to an- swer their clinical question: “In hospitalized adults (P), how does a rapid response team (I) compared with no rapid response team (C) affect the number of cardiac arrests (O) and unplanned admissions to the ICU (O) during a three- month period (T)?” As part of their rapid critical appraisal (RCA) of the 15 potential “keeper” studies, the EBP team found and placed the essential elements of each study (such as its population, study design, and setting) into an evaluation table. In so doing, they began to see similarities and differ- ences between the studies, which Carlos told them is the beginning of synthesis. We now join the team as they continue with their RCA of these studies to determine their worth to practice.

RAPID CRITICAL APPRAISAL

Carlos explains that typically an RCA is conducted along with an RCA checklist that’s specific to the research design of the study being evaluated—and before any data are entered into an evalua- tion table. However, since Rebecca and Chen are new to appraising studies, he felt it would be easier for them to first enter the essen- tials into the table and then eval- uate each study. Carlos shows Rebecca several RCA checklists and explains that all checklists have three major questions in common, each of which contains other more specific subquestions about what constitutes a well- conducted study for the research design under review (see Example of a Rapid Critical Appraisal Checklist).

Although the EBP team will be looking at how well the re- searchers conducted their studies and discussing what makes a “good” research study, Carlos reminds them that the goal of critical appraisal is to determine the worth of a study to practice, not solely to find flaws. He also

suggests that they consult their glossary when they see an unfa- miliar word. For example, the term randomization, or random assignment, is a relevant feature of research methodology for in- tervention studies that may be unfamiliar. Using the glossary, he explains that random assignment and random sampling are often confused with one another, but that they’re very different. When researchers select subjects from within a certain population to participate in a study by using a random strategy, such as tossing a coin, this is random sampling. It allows the entire population

to be fairly represented. But because it requires access to a particular population, random sampling is not always feasible. Carlos adds that many health care studies are based on a con- venience sample—participants recruited from a readily available population, such as a researcher’s affiliated hospital, which may or may not represent the desired population. Random assignment, on the other hand, is the use of a random strategy to assign study

ajn@wolterskluwer.com

AJN ▼ September 2010 ▼ Vol. 110, No. 9 41

participants to the intervention or control group. Random as- signment is an important feature of higher-level studies in the hier- archy of evidence.

Carlos also reminds the team that it’s important to begin the RCA with the studies at the high- est level of evidence in order to see the most reliable evidence first. In their pile of studies, these are the three systematic reviews, includ- ing the meta-analysis and the Cochrane review, they retrieved from their database search (see “Searching for the Evidence,” and “Critical Appraisal of the Evidence: Part I,” Evidence- Based Practice, Step by Step, May and July). Among the RCA checklists Carlos has brought

with him, Rebecca and Chen find the checklist for systematic reviews.

As they start to rapidly criti- cally appraise the meta-analysis, they discuss that it seems to be biased since the authors included only studies with a control group. Carlos explains that while hav- ing a control group in a study is ideal, in the real world most stud- ies are lower-level evidence and don’t have control or compari- son groups. He emphasizes that, in eliminating lower-level studies, the meta-analysis lacks evidence that may be informative to the question. Rebecca and Chen— who are clearly growing in their appraisal skills—also realize that three studies in the meta-analysis

are the same as three of their potential “keeper” studies. They wonder whether they should keep those studies in the pile, or if, as duplicates, they’re unnecessary. Carlos says that because the meta- analysis only included studies with control groups, it’s impor- tant to keep these three studies so that they can be compared with other studies in the pile that don’t have control groups. Rebecca notes that more than half of their 15 studies don’t have control or comparison groups. They agree as a team to include all 15 stud- ies at all levels of evidence and go on to appraise the two remaining systematic reviews.

The MERIT trial1 is next in the EBP team’s stack of studies.

Example of a Rapid Critical Appraisal Checklist

Rapid Critical Appraisal of Systematic Reviews of Clinical Interventions or Treatments

1. Are the results of the review valid?

A. Are the studies in the review randomized controlled trials? B. Does the review include a detailed description of the search

strategy used to find the relevant studies? C. Does the review describe how the validity of the individual

studies was assessed (such as, methodological quality, including the use of random assignment to study groups and complete follow-up of subjects)?

D. Are the results consistent across studies? E. Did the analysis use individual patient data or aggregate data?

2. What are the results?

A. How large is the intervention or treatment effect (odds ratio, relative risk, effect size, level of significance)?

B. How precise is the intervention or treatment (confidence interval)?

3. Will the results assist me in caring for my patients?

A. Are my patients similar to those in the review? B. Is it feasible to implement the findings in my practice setting? C. Were all clinically important outcomes considered, including

both risks and benefits of the treatment? D. What is my clinical assessment of the patient, and are there any

contraindications or circumstances that would keep me from

implementing the treatment? E. What are my patients’ and their families’ preferences and

values concerning the treatment? © Fineout-Overholt and Melnyk, 2005.

Yes No Yes No

Yes No

Yes No Patient Aggregate

Yes No Yes No

Yes No

Yes No Yes No

42 AJN ▼ September 2010 ▼ Vol. 110, No. 9

ajnonline.com

As we noted in the last install- ment of this series, MERIT is a good study to use to illustrate the different steps of the critical appraisal process. (Readers may want to retrieve the article, if possible, and follow along with the RCA.) Set in Australia, the MERIT trial examined whether the introduction of a rapid re- sponse team (RRT; called a med- ical emergency team or METin the study) would reduce the incidence of cardiac arrest, death, and unplanned admissions to the ICU in the hospitals studied. To follow along as the EBP team addresses each of the essential elements of a well-conducted randomized controlled trial (RCT) and how they apply to the MERIT study, see their notes in Rapid Critical Appraisal of the MERIT Study.ARE THE RESULTS OF THE STUDY VALID?The first section of every RCA checklist addresses the validity of the study at hand—did the researchers use sound scientific methods to obtain their study results? Rebecca asks why valid- ity is so important. Carlos replies that if the study’s conclusion can be trusted—that is, relied upon to inform practice—the study must be conducted in a way that reduces bias or eliminates con- founding variables (factors that influence how the intervention affects the outcome). Researchers typically use rigorous research methods to reduce the risk of bias. The purpose of the RCA checklist is to help the user deter- mine whether or not rigorous methods have been used in the study under review, with most questions offering the option of a quick answer of “yes,” “no,” or “unknown.”Were the subjects randomly assigned to the intervention and control groups? Carlos explainsthat this is an important question when appraising RCTs. If a study calls itself an RCT but didn’t randomly assign participants, then bias could be present. In appraising the MERIT study, the team discusses how the research- ers randomly assigned entire hospitals, not individual patients, to the RRT intervention and control groups using a technique called cluster randomization. To better understand this method, the EBP team looks it up on the Internet and finds a PowerPoint presentation by a World Health Organization researcher that explains it in simplified terms: “Cluster randomized trials are experiments in which social units or clusters [in our case, hospitals] rather than individuals are ran- domly allocated to intervention groups.”2Was random assignment concealed from the individuals enrolling the subjects? Conceal- ment helps researchers reduce potential bias, preventing the person(s) enrolling participants from recruiting them into a study with enthusiasm if they’re des- tined for the intervention group or with obvious indifference if they’re intended for the control or comparison group. The EBP team sees that the MERIT trial used an independent statistician to conduct the random assign- ment after participants had already been enrolled in the study, which Carlos says meets the criteria for concealment.Were the subjects and pro- viders blind to the study group? Carlos notes that it would be difficult to blind participants or researchers to the interven- tion group in the MERIT study because the hospitals that were to initiate an RRT had to know it was happening. Rebecca and Chen wonder whether their “no” answer to this question makesthe study findings invalid. Carlos says that a single “no” may or may not mean that the study findings are invalid. It’s their job as clinicians interpreting the data to weigh each aspect of the study design. Therefore, if the answer to any validity question isn’t affirmative, they must each ask themselves: does this “no” make the study findings untrustworthy to the extent that I don’t feel comfortable using them in my practice?Were reasons given to explain why subjects didn’t complete the study? Carlos explains that sometimes par- ticipants leave a study before the end (something about the study or the participants themselves may prompt them to leave). If all or many of the participants leave for the same reason, this may lead to biased findings. Therefore, it’s important to look for an explanation for why any subjects didn’t complete a study. Since no hospitals dropped out of the MERIT study, this ques- tion is determined to be not applicable.Were the follow-up assess- ments long enough to fully study the effects of the intervention? Chen asks Carlos why a time frame would be important in studying validity. He explains that researchers must ensure that the outcome is evaluated for a long enough period of time to show that the intervention indeed caused it. The researchers in the MERIT study conducted the RRT intervention for six months be- fore evaluating the outcomes. The team discusses how six months was likely adequate to determine how the RRT affected cardio- pulmonary arrest rates (CR) but might have been too short to es- tablish the relationship between the RRT and hospital-wide mor- tality rates (HMR).ajn@wolterskluwer.comAJN ▼ September 2010 ▼ Vol. 110, No. 9 43

44 AJN ▼ September 2010 ▼ Vol. 110, No. 9 ajnonline.com

Rapid Critical Appraisal of the MERIT Study

1. Are the results of the study valid? A. Were the subjects randomly assigned to the intervention and control groups? Yes No Unknown

Random assignment of hospitals was made to either a rapid response team (RRT; intervention) group or no RRT (con- trol) group. To protect against introducing further bias into the study, hospitals, not individual patients, were randomly assigned to the intervention. If patients were the study subjects, word of the RRT might have gotten around, potentially influencing the outcome.

B. Was random assignment concealed from the individuals enrolling the subjects? Yes No Unknown An independent statistician randomly assigned hospitals to the RRT or no RRT group after baseline data had been

collected; thus the assignments were concealed from both researchers and participants.

C. Were the subjects and providers blind to the study group? Yes No Unknown

Hospitals knew to which group they’d been assigned, as the intervention hospitals had to put the RRTs into practice. Management, ethics review boards, and code committees in both hospitals knew about the intervention. The control hospitals had code teams and some already had systems in place to manage unstable patients. But control hospitals didn’t have a placebo strategy to match the intervention hospitals’ educational strategy for how to implement an RRT (a red flag for confounding!). If you worked in one of the control hospitals, unless you were a member of one of the groups that gave approval, you wouldn’t have known your hospital was participating in a study on RRTs; this lessens the chance of confounding variables influencing the outcomes.

D. Were reasons given to explain why subjects didn’t complete the study? Yes No Not Applicable This question is not applicable as no hospitals dropped out of the study.

E. Were the follow-up assessments long enough to fully study the effects of the inter vention? Yes No Unknown

The intervention was conducted for six months, which should be adequate time to have an impact on the outcomes of car- diopulmonary arrest rates (CR), hospital-wide mortality rates (HMR), and unplanned ICU admissions (UICUA). However, the authors remark that it can take longer for an RRT to affect mortality, and cite trauma protocols that took up to 10 years.

F. Were the subjects analyzed in the group to which they were randomly assigned? Yes No Unknown

All 23 (12 intervention and 11 control) hospitals remained in their groups, and analysis was conducted on an intention- to-treat basis. However, in their discussion, the authors attempt to provide a reason for the disappointing study results; they suggest that because the intervention group was “inadequately implemented,” the fidelity of the intervention was compromised, leading to less than reliable results. Another possible explanation involves the baseline quality of care; if high, the improvement after an RRT may have been less than remarkable. The authors also note a historical confounder: in Australia, where the study took place, there was a nationwide increase in awareness of patient safety issues.

G. Was the control group appropriate? Yes No Unknown

See notes to question C. Controls had no time built in for education and training as the intervention hospitals did, so this time wasn’t controlled for, nor was there any known attempt to control the organizational “buzz” that something was going on. The study also didn’t account for the variance in how RRTs were implemented across hospitals. The researchers indicate that the existing code teams in control hospitals “did operate as [RRTs] to some extent.” Because of these factors, the appropriateness of the control group is questionable.

H. Were the instruments used to measure the outcomes valid and reliable? Yes No Unknown

The primary outcome was the composite of HMR (that is, unexpected deaths, excluding do not resuscitates [DNRs]), CR (that is, no palpable pulse, excluding DNRs), and UICUA (any unscheduled admissions to the ICU).

I. Were the demographics and baseline clinical variables of the subjects in each of the groups similar? Yes No Unknown

The researchers provided a table showing how the RRT and control hospitals compared on several variables. Some variability existed, but there were no statistical differences between groups.

2. What are the results?

A. How large is the intervention or treatment effect?

The researchers reported outcome data in various ways, but the bottom line is that the control group did better than the intervention group. For example, RRT calling criteria were documented more than 15 minutes before an event by more hospitals in the control group than in the intervention group, which is contrary to expectation. Half the HMR cases in the intervention group met the criteria compared with 55% in the control group (not statistically significant). But only 30% of CR cases in the intervention group met the criteria compared with 44% in the control group, which was statistically significant (P = 0.031). Finally, regarding UICUA, 51% in the intervention group compared with 55% in the control group met the criteria (not significant). This indicates that the control hospitals were doing a better job of documenting unstable patients before events occurred than the intervention hospitals.

B. How precise is the intervention or treatment?

The odds ratio (OR) for each of the outcomes was close to 1.0, which indicates that the RRT had no effect in the intervention hospitals compared with the control hospitals. Each confidence interval (CI) also included the num- ber 1.0, which indicates that each OR wasn’t statistically significant (HMR OR = 1.03 (0.84 – 1.28); CR OR = 0.94 (0.79 – 1.13); UICUA OR = 1.04 (0.89 – 1.21). From a clinical point of view, the results aren’t straightfor- ward. It would have been much simpler had the intervention hospitals and the control hospitals done equally badly; but the fact that the control hospitals did better than the intervention hospitals raises many questions about the results.

3. Will the results help me in caring for my patients?

A. Were all clinically important outcomes measured? Yes No Unknown

It would have been helpful to measure cost, since participating hospitals that initiated an RRT didn’t eliminate their code team. If a hospital has two teams, is the cost doubled? And what’s the return on investment? There’s also no mention of the benefits of the code team. This is a curious question . . . maybe another PICOT question?

B. What are the risks and benefits of the treatment?

This is the wrong question for an RRT. The appropriate question would be: What is the risk of not adequately introduc- ing, monitoring, and evaluating the impact of an RRT?

C. Is the treatment feasible in my clinical setting? Yes No Unknown

We have administrative support, once we know what the evidence tells us. Based on this study, we don’t know much more than we did before, except to be careful about how we approach and evaluate the issue. We need to keep the following issues, which the MERIT researchers raised in their discussion, in mind: 1) allow adequate time to measure outcomes; 2) some outcomes may be reliably measured sooner than others; 3) the process of implementing an RRT is very important to its success.

D. What are my patients’ and their families’ values and expectations for the outcome and the treatment itself?

We will keep this in mind as we consider the body of evidence.

ajn@wolterskluwer.com AJN ▼ September 2010 ▼ Vol. 110, No. 9 45

Were the subjects analyzed in the group to which they were randomly assigned? Rebecca sees the term intention-to-treat analysis in the study and says that it sounds like statistical language. Carlos confirms that it is; it means that the researchers kept the hos- pitals in their assigned groups when they conducted the analysis, a technique intended to reduce possible bias. Even though the MERIT study used this technique, Carlos notes that in the discussion section the authors offer some important caveats about how the study was conducted, including poor intervention implementation, which may have contributed to MERIT’s unexpected findings.1

Was the control group appro- priate? Carlos explains that it’s challenging to establish an ap- propriate comparison or control group without an understanding of how the intervention will be implemented. In this case, it may be problematic that the interven- tion group received education and training in implementing the RRT and the control group re- ceived no comparable placebo (meaning education and training about something else). But Car- los reminds the team that the re- searchers attempted to control for known confounding variables by stratifying the sample on char- acteristics such as academic versus nonacademic hospitals, bed size, and other important parameters. This method helps to ensure equal representation of these pa- rameters in both the intervention and control groups. However, a major concern for clinicians con- sidering whether to use the MERIT findings in their decision making involves the control hos- pitals’ code teams and how they may have functioned as RRTs, which introduces a potential con- founder into the study that could possibly invalidate the findings.

Were the instruments used to measure the outcomes valid and reliable? The overall measure in the MERIT study is the compos- ite of the individual outcomes: CR, HMR, and unplanned ad- missions to the ICU (UICUA). These parameters were defined reasonably and didn’t include do not resuscitate (DNR) cases. Car- los explains that since DNR cases are more likely to code or die, in- cluding them in the HMR and CR would artificially increase these outcomes and introduce bias into the findings.

As the team moves through the questions in the RCA check- list, Rebecca wonders how she and Chen would manage this kind of appraisal on their own. Carlos assures them that they’ll get better at recognizing well- conducted research the more RCAs they do. Though Rebecca feels less than confident, she appre- ciates his encouragement nonethe- less, and chooses to lead the team in discussion of the next question.

Were the demographics and baseline clinical variables of the subjects in each of the groups similar? Rebecca says that the intervention group and the con- trol or comparison group need to be similar at the beginning of any intervention study because any differences in the groups could influence the outcome, poten- tially increasing the risk that the outcome might be unrelated to the intervention. She refers the team to their earlier discussion about confounding variables. Carlos tells Rebecca that her explana- tion was excellent. Chen remarks that Rebecca’s focus on learning appears to be paying off.

WHAT ARE THE RESULTS?

As the team moves on to the sec- ond major question, Carlos tells them that many clinicians are apprehensive about interpreting

statistics. He says that he didn’t take courses in graduate school on conducting statistical analysis; rather, he learned about different statistical tests in courses that re- quired students to look up how to interpret a statistic whenever they encountered it in the articles they were reading. Thus he had a context for how the statistic was being used and interpreted, what question the statistical analysis was answering, and what kind of data were being analyzed. He also learned to use a search engine, such as Google.com, to find an explanation for any statistical tests with which he was unfamil- iar. Because his goal was to un- derstand what the statistic meant clinically, he looked for simple Web sites with that same focus and avoided those with Greek symbols or extensive formulas that were mostly concerned with conducting statistical analysis.

How large is the intervention or treatment effect? As the team goes through the studies in their RCA, they decide to construct a list of statistics terminology for quick reference (see A Sampling of Statistics). The major statistic used in the MERIT study is the odds ratio (OR). The OR is used to provide insight into the measure of association between an inter- vention and an outcome. In the MERIT study, the control group did better than the intervention group, which is contrary to what was expected. Rebecca notes that the researchers discussed the pos- sible reasons for this finding in the final section of the study. Carlos says that the authors’ discussion about why their findings occurred is as important as the findings themselves. In this study, the discussion communicates to any clinicians considering initiating an RRT in their hospital that they should assess whether the current code team is already functioning

46 AJN ▼ September 2010 ▼ Vol. 110, No. 9

ajnonline.com

A Sampling of StatisticsStatisticSimple DefinitionImportant ParametersUnderstanding the StatisticClinical ImplicationsOdds Ratio (OR)The odds of an outcome occurring in the intervention group compared with the odds of it occurring in the comparison or control group.· IfanORisequalto1,thenthe intervention didn’t make a differ- ence.· Interpretation depends on the out- come.· If the outcome is good (for exam- ple, fall prevention), the OR is pre- ferred to be above 1.· If the outcome is bad (for example, mortality rate), the OR is preferred to be below 1.The OR for hospital-wide mor- tality rates (HMR) in the MERIT study was 1.03 (95% CI, 0.84 – 1.28). The odds of HMR in the intervention group were about the same as HMR in the comparison group.From the HMR OR data alone, a clinician may not feel confident that a rapid response team (RRT) is the best intervention to reduce HMR but may seek out other evidence before making a decision.Relative Risk (RR)The risk of an out- come occurring in the intervention group compared with the risk of it occurring in the comparison or control group.· IfanRRisequalto1,thenthe intervention didn’t make a differ- ence.· Interpretation depends on the out- come.· If the outcome is good (for example fall prevention), the RR is preferred to be above 1.· If the outcome is bad (for example, mortality rate), the RR is preferred to be below 1.The RR of cardiopulmonary ar- rest in adults was reported in the Chan PS, et al., 2010 sys- tematic reviewa as 0.66 (95% CI, 0.54 – 0.80), which is sta- tistically significant because there’s no 1.0 in the CI.Thus, the RR of cardiopulmo- nary arrest occurring in the intervention group compared with the RR of it occurring in the control group is 0.66, or less than 1. Since cardiopulmonary arrest is not a good outcome, this is a desirable finding.The RRT significantly reduced the RR of cardiopulmonary arrest in this study. From these data, clinicians can be reasonably confident that ini- tiating an RRT will reduce CR in hospitalized adults.Confidence Interval (CI)The range in which clinicians can expect to get results if they pres- ent the interven- tion as it was in the study.· CI provides the precision of the study finding: a 95% CI indicates that clinicians can be 95% con- fident that their findings will be within the range given in the study.· CI should be narrow around the study finding, not wide.· If a CI contains the number that indicates no effect (for OR it’s 1; for effect size it’s 0), the study finding is not statistically significant.See the two previous examples.In the Chan PS, et al., 2010 systematic review,a the CI is a close range around the study finding and is statistically significant. Clinicians can be 95% confident that if they conduct the same interven- tion, they’ll have a result simi- lar to that of the study (that is, a reduction in risk of cardio- pulmonary arrest) within the range of the CI, 0.54 – 0.80. The narrower the CI range, the more confident clinicians can be that, using the same intervention, their results will be close to the study findings.Mean (X)Average• Caveat: Averaging captures only those subjects who surround a central tendency, missing those who may be unique. For example, the mean (average) hair color in a classroom of schoolchildren cap- tures those with the predominant hair color. Children with hair color different from the predominant hair color aren’t captured and are con- sidered outliers (those who don’t converge around the mean).In the Dacey MJ, et al., 2007 study,a before the RRT the aver- age (mean) CR was 7.6 per 1,000 discharges per month; after the RRT, it decreased to3 per 1,000 discharges per month.Introducing an RRT decreased the average CR by more than 50% (7.6 to 3 per 1,000 discharges per month).a For study details on Chan PS, et al., and Dacey MJ, et al., go to http://links.lww.com/AJN/A11.ajn@wolterskluwer.com AJN ▼ September 2010 ▼ Vol. 110, No. 9 47

as an RRT prior to RRT imple- mentation.

How precise is the interven- tion or treatment? Chen wants to tackle the precision of the findings and starts with the OR for HMR, CR, and UICUA, each of which has a confidence interval (CI) that includes the number 1.0. In an EBP workshop, she learned that a1.0inaCIforORmeansthat the results aren’t statistically sig- nificant, but she isn’t sure what statistically significant means. Car- los explains that since the CIs for the OR of each of the three out- comes contains the number 1.0, these results could have been ob- tained by chance and therefore aren’t statistically significant. For clinicians, chance findings aren’t reliable findings, so they can’t confidently be put into practice. Study findings that aren’t statisti- cally significant have a probabil- ity value (P value) of greater than 0.5. Statistically significant find- ings are those that aren’t likely to be obtained by chance and have a P value of less than 0.5.

WILL THE RESULTS HELP ME IN CARING FOR MY PATIENTS? The team is nearly finished with their checklist for RCTs. The third and last major question addresses the applicability of the study— how the findings can be used to help the patients the team cares for. Rebecca observes that it’s easy to get caught up in the de- tails of the research methods and findings and to forget about how they apply to real patients.

Were all clinically important outcomes measured? Chen says that she didn’t see anything in the study about how much an RRT costs to initiate and how to com- pare that cost with the cost of one code or ICU admission. Carlos agrees that providing costs would have lent further insight into the results.

What are the risks and ben- efits of the treatment? Chen won- ders how to answer this since the findings seem to be confounded by the fact that the control hos- pital had code teams that func- tioned as RRTs. She wonders if there was any consideration of the risks and benefits of initiating an RRT prior to beginning the study. Carlos says that the study doesn’t directly mention it, but the consideration of the risks and benefits of an RRT is most likely what prompted the researchers to conduct the study. It’s helpful to remember, he tells the team, that often the answer to these questions is more than just “yes” or “no.”

Is the treatment feasible in my clinical setting? Carlos acknowl- edges that because the nursing administration is open to their project and supports it by provid- ing time for the team to conduct its work, an RRT seems feasible in their clinical setting. The team discusses that nursing can’t be the sole discipline involved in the project. They must consider how to include other disciplines as part of their next step (that is, the im- plementation plan). The team con- siders the feasibility of getting all disciplines on board and how to address several issues raised by the researchers in the discussion sec- tion (see Rapid Critical Appraisal of the MERIT Study), particu- larly if they find that the body of evidence indicates that an RRT does indeed reduce their chosen outcomes of CR, HMR, and UICUA.

What are my patients’ and their families’ values and expec- tations for the outcome and the treatment itself? Carlos asks Rebecca and Chen to discuss with their patients and their patients’ families their opinion of an RRT and if they have any objections to the intervention. If there are

objections, the patients or fami- lies will be asked to reveal them. The EBP team finally com- pletes the RCA checklists for the

15 studies and finds them all to be “keepers.” There are some studies in which the findings are less than reliable; in the case of MERIT, the team decides to in- clude it anyway because it’s con- sidered a landmark study. All the studies they’ve retained have something to add to their under- standing of the impact of an RRT on CR, HMR, and UICUA. Car- los says that now that they’ve determined the 15 studies to be somewhat valid and reliable, they can add the rest of the data to the evaluation table.

Be sure to join the EBP team for “Critical Appraisal of the Evi- dence: Part III” in the next install- ment in the series, when Rebecca, Chen, and Carlos complete their synthesis of the 15 studies and determine what the body of evi- dence says about implementing an RRT in an acute care setting. ▼

Ellen Fineout-Overholt is clinical pro- fessor and director of the Center for the Advancement of Evidence-Based Practice at Arizona State University in Phoenix, where Bernadette Mazurek Melnyk is dean and distinguished foun- dation professor of nursing, Susan B. Stillwell is clinical associate professor and program coordinator of the Nurse Educator Evidence-Based Practice Mentorship Program, and Kathleen M. Williamson is associate director of

the Center for the Advancement of Evidence-Based Practice. Contact author: Ellen Fineout-Overholt, ellen. fineout-overholt@asu.edu.

REFERENCES

1. Hillman K, et al. Introduction of the medical emergency team (MET) sys- tem: a cluster-randomised controlled trial. Lancet 2005;365, 2091-7.

2. Wojdyla D. Cluster randomized trials and equivalence trials [PowerPoint presentation]. Geneva, Switzerland: Geneva Foundation for Medical Education and Research; 2005. http:// www.gfmer.ch/PGC_RH_2005/pdf/ Cluster_Randomized_Trials.pdf.

Critical Appraisal of the Evidence: Part III

The process of synthesis: seeing similarities and differences across the body of evidence.

By Ellen Fineout-Overholt, PhD, RN, FNAP, FAAN, Bernadette Mazurek Melnyk, PhD, RN, CPNP/PMHNP,

FNAP, FAAN, Susan B. Stillwell, DNP, RN, CNE, and Kathleen M. Williamson, PhD, RN

This is the seventh article in a series from the Arizona State University College of Nursing and Health Innovation’s Center for the Advancement of Evidence-Based Practice. Evidence-based practice (EBP) is a problem-solving approach to the delivery of health care that integrates the best evidence from studies and patient care data with clinician exper- tise and patient preferences and values. When delivered in a context of caring and in a supportive organizational culture, the highest quality of care and best patient outcomes can be achieved.

In September’s evidence- based practice (EBP) article, Rebecca R., our hypothetical staff nurse, Carlos A., her hospi- tal’s expert EBP mentor, and Chen M., Rebecca’s nurse colleague, ra- pidly critically appraised the 15 articles they found to answer their clinical question—“In hospital- ized adults (P), how does a rapid response team (I) compared with no rapid response team (C) affect the number of cardiac arrests (O) and unplanned admissions to the ICU (O) during a three-month period (T)?”—and determined that they were all “keepers.” The team now begins the process of evaluation and synthesis of the articles to see what the evidence says about initiating a rapid re- sponse team (RRT) in their hos- pital. Carlos reminds them that evaluation and synthesis are syn- ergistic processes and don’t neces- sarily happen one after the other. Nevertheless, to help them learn, he will guide them through the EBP process one step at a time.

STARTING THE EVALUATION

Rebecca, Carlos, and Chen begin to work with the evaluation table

they created earlier in this process when they found and filled in the essential elements of the 15 stud- ies and projects (see “Critical Ap- praisal of the Evidence: Part I,” July). Now each takes a stack of the “keeper” studies and system- atically begins adding to the table any remaining data that best re- flect the study elements pertain- ing to the group’s clinical question (see Table 1; for the entire table with all 15 articles, go to http:// links.lww.com/AJN/A17). They had agreed that a “Notes” sec- tion within the “Appraisal: Worth to Practice” column would be a good place to record the nuances

of an article, their impressions of it, as well as any tips—such as what worked in calling an RRT— that could be used later when they write up their ideas for ini- tiating an RRT at their hospital, if the evidence points in that direc- tion. Chen remarks that although she thought their initial table con- tained a lot of information, this final version is more thorough by far. She appreciates the opportu- nity to go back and confirm her original understanding of the study essentials.

The team members discuss the evolving patterns as they complete the table. The three systematic

Need Help with Evidence-Based Practice? Chat with the Authors on November 16!

On November 16 at 3 PM EST, join the “Chat with the Au- thors” call. It’s your chance to get personal consultation from the experts! Dial-in early! U.S. and Canada, dial 1-800-947-5134 (International, dial 001-574-941-6964). When prompted, enter code 121028#.

Go to www.ajnonline.com and click on “Podcasts” and then on “Conversations” to listen to our interview with Ellen Fineout- Overholt and Bernadette Mazurek Melnyk.

44 AJN ▼ November 2010 ▼ Vol. 110, No. 11 ajnonline.com

Table 1. Final Evaluation Table

First Author (Year)

Conceptual Framework

Design/Method

Sample/Setting

Major Variables Studied (and Their Definitions)

Measurement

Data Analysis

Findings

Appraisal: Worth to Practice

Chan PS, et al.

None

SR Purpose: effect of RRT on HMR and CR • Searched 5

N = 18 out of 143 potential studies

IV: RRT DV1: HMR (including DNR, excluding DNR, not treated in ICU, no HMR definition) DV2: CR

RRT: was the MD involved?

• Fre – quency

13/16 studies reporting team structure

Weaknesses: • Potential missed evi-

Arch Intern Med

• Relative risk

dence with exclusion of all studies except those with control groups

2010;170(1): 18-26

Setting: acute care hospitals; 13 adult, 5 peds

HMR: overall hospital deaths (see definition)

7/11 adult and 4/5 peds studies had sig- nificant reduc- tion in CR

databases from 1950–2008 and “grey literature” from MD confer- ences

Average no. beds: NR

CR: cardio and/or pulmo- nary arrest; cardiac arrest calls

• Grey literature search limited to medical meet- ings

• Included only 1) RCTs and

CR: • In adults,

• Only included HMR and CR outcomes

prospective

Strengths: • Identified no. of activa-

studies with 2) a control group or

tions of RRT/1,000 admissions Identified variance in outcome definition and measurement (for example, 10 of 15 stud- ies included deaths from DNRs in their mortality measurement)

control period

•

and 3) hospital

mortality well described as outcome

• Excluded 5 studies that met criteria due to no response to e-mail by primary authors

HMR: • In adults,

Conclusion: • RRT reduces CR in

Attrition: NR

21%–48% reduction in CR; RR 0.66 (95% CI, 0.54–0.80) In peds, 38% reduction in CR; RR 0.62 (95% CI, 0.46–0.84)

• No cost data

HMR RR 0.96 (95% CI, 0.84– 1.09)

adults, and CR and HMR in peds

• In peds, HMR RR

Feasibility: • RRT is reasonable to

0.79 (95% CI, 0.63– 0.98)

implement; evaluating cost will help in making decisions about using RRT

• Risk/Benefit (harm): benefits outweigh risks

McGaughey J, et al. Cochrane Database Syst Rev 2007;3: CD005529

None

SR (Cochrane review)

N = 2 studies

IV: RRT DV1: HMR

HMR: Australia: overall hospital mortality with- out DNR

OR of Aus- tralian study, 0.98 (95% CI, 0.83–1.16)

Weaknesses: • Didn’t include full body

Winters BD, et al. Crit Care Med 2007;35(5): 1238-43

None

SR Purpose: effect of RRT on HMR and CR • Searched 3

N = 8 studies

IV: RRT DV1: HMR DV2: CR

HMR: overall death rate

Risk ratio

HMR: • Observa-

Strengths: • Provides comparison

Purpose: effect of RRT on HMR • Searched 6

Acute care set- tings in Australia and the UK

of evidence • Conflicting results of

databases from

Attrition: NR

UK: Simplified Acute Physiol- ogy Score (SAPS) II death probabil- ity estimate

OR of UK study, 0.52 (95% CI, 0.32–0.85)

retained studies, but no discussion of the impact of lower-level evidence

1990 –2006 • Excluded all but

• Recommendation “need more research”

2 RCTs

Conclusion: • Inconclusive

databases from

Attrition: NR

4–82 months) Sample size (range,

1990 –2005 • Included only studies with a control group

• Cluster RCTs, risk ratio for RRT on HMR, 0.76 (95% CI, 0.39– 1.48)

2,183–199,024) Criteria for RRT initia-

Average no. beds: 500

CR: no. of in- hospital arrests

tional studies, risk ratio for RRT on HMR, 0.87 (95% CI, 0.73– 1.04)

across studies for Study lengths (range,

CI = confidence interval; CR = cardiopulmonary arrest or code rates; DNR = do not resuscitate; DV = dependent variable; HMR = hospital-wide mortality rates; ICU = intensive care unit; IV = independent variable; MD = medical doctor; NR = not reported; OR = odds ratio; Peds = pediatrics; RCT = randomized controlled trial; RR = relative risk; RRT = rapid response team; SR = systematic review; UK = United Kingdom

CR: • Observa-

tion (common: respira- tory rate, heart rate, blood pressure, mental status change; not all studies, but notewor- thy: oxygen saturation, “worry”)

tional studies, risk ratio for RRT on CR, 0.70 (95% CI, 0.56– 0.92)

Conclusion: • Some support for RRT,

• Cluster RCTs, risk ratio for RRT on CR, 0.94 (95% CI, 0.79– 1.13)

but not reliable enough to recommend as stan- dard of care

•

Includes ideas about future evidence gen- eration (conducting research)—finding out what we don’t know

reviews, which are higher-level evidence, seem to have an inher- ent bias in that they included only studies with control groups. In general, these studies weren’t in favor of initiating an RRT. Carlos asks Rebecca and Chen whether,

Chen in their efforts to appraise the MERIT study and comments on how well they’re putting the pieces of the evidence puzzle to- gether. The nurses are excited that they’re able to use their new knowledge to shed light on the

as well as a good number of jour- nals have encouraged their use. When they review the actual guidelines, the team notices that they seem to be focused on re- search; for example, they require a research question and refer to

It’s not the number of studies or projects that determines the reliability of their ndings, but the uniformity and quality of their methods.

now that they’ve appraised all the evidence about RRTs, they’re con- fident in their decision to include all the studies and projects (in- cluding the lower-level evidence) among the “keepers.” The nurses reply with an emphatic affirma- tive! They tell Carlos that the proj- ects and descriptive studies were what brought the issue to life for them. They realize that the higher- level evidence is somewhat in conflict with the lower-level evi- dence, but they’re most interested in the conclusions that can be drawn from considering the entire body of evidence.

Rebecca and Chen admit they have issues with the systematic reviews, all of which include the MERIT study.1-4 In particular, they discuss how the authors of the systematic reviews made sure to report the MERIT study’s finding that the RRT had no effect, but didn’t emphasize the MERIT study authors’ discussion about how their study methods may have influenced the reliability of the findings (for more, see “Critical Appraisal of the Evidence: Part II,” September). Carlos says that this is an excellent observation. He also reminds the team that clinicians may read a systematic review for the conclusion and never consider the original stud- ies. He encourages Rebecca and

study. They discuss with Carlos how the interpretation of the MERIT study has perhaps con- tributed to a misunderstanding of the impact of RRTs.

Comparing the evidence. As the team enters the lower-level evi- dence into the evaluation table, they note that it’s challenging to compare the project reports with studies that have clearly described methodology, measurement, anal- ysis, and findings. Chen remarks that she wishes researchers and clinicians would write study and project reports similarly. Although each of the studies has a process or method determining how it was conducted, as well as how out- comes were measured, data were analyzed, and results interpreted, comparing the studies as they’re currently written adds another layer of complexity to the eval- uation. Carlos says that while it would be great to have studies and projects written in a similar for- mat so they’re easier to compare, that’s unlikely to happen. But he tells the team not to lose all hope, as a format has been developed for reporting quality improve- ment initiatives called the SQUIRE Guidelines; however, they aren’t ideal. The team looks up the guide- lines online (www.squire-statement. org) and finds that the Institute for Healthcare Improvement (IHI)

the study of an intervention, whereas EBP projects have PICOT questions and apply evidence to practice. The team discusses that these guidelines can be confusing to the clinicians authoring the re- ports on their projects. In addition, they note that there’s no mention of the synthesis of the body of evidence that should drive an evidence-based project. While the SQUIRE Guidelines are a step in the right direction for the future, Carlos, Rebecca, and Chen con- clude that, for now, they’ll need to learn to read these studies as they find them — looking care- fully for the details that inform their clinical question.

Once the data have been en- tered into the table, Carlos sug- gests that they take each column, one by one, and note the similari- ties and differences across the studies and projects. After they’ve briefly looked over the columns, he asks the team which ones they think they should focus on to an- swer their question. Rebecca and Chen choose “Design/Method,” “Sample/Setting,” “Findings,” and “Appraisal: Worth to Practice” (see Table 1) as the initial ones to consider. Carlos agrees that these are the columns in which they’re most likely to find the most pertinent information for their synthesis.

46 AJN ▼ November 2010 ▼ Vol. 110, No. 11

ajnonline.com

SYNTHESIZING: MAKING DECISIONS BASED ON THE EVIDENCE Design/Method. The team starts with the “Design/Method” column because Carlos reminds them that it’s important to note each study’s level of evidence. He suggests that they take this information and create a synthesis table (one in which data is extracted from the evaluation table to better see the similarities and differences between studies) (see Table 21-15). The synthesis table makes it clear that there is less higher-level and more lower-level evidence, which will impact the reliability of the overall findings. As the team noted, the higher-level evidence is not without methodological issues, which will increase the challenge of coming to a conclusion aboutthe impact of an RRT on the out- comes.Sample/Setting. In reviewing the “Sample/Setting” column, the group notes that the number of hospital beds ranged from 218 to 662 across the studies. There were several types of hospitals represented (4 teaching, 4 com- munity, 4 no mention, 2 acute care hospitals, and 1 public hos- pital). The evidence they’ve col- lected seems applicable, since their hospital is a community hospital.Findings. To help the team better discuss the evidence, Car- los suggests that they refer to all projects or studies as “the body of evidence.” They don’t want to get confused by calling them all studies, as they aren’t, but at thesame time continually referring to “studies and projects” is cum- bersome. He goes on to say that, as part of the synthesis process, it’s important for the group to determine the overall impact of the intervention across the body of evidence. He helps them create a second synthesis table contain- ing the findings of each study or project (see Table 31-15). As they look over the results, Rebecca and Chen note that RRTs reduce code rates, particularly outside the ICU, whereas unplanned ICU admissions (UICUA) don’t seem to be as affected by them. However, 10 of the 15 studies and projects reviewed didn’t evaluate this outcome, so it may not be fair to write it off just yet.Table 2: The 15 Studies: Levels and Types of Evidence123456789101112131415Level I: Systematic review or meta-analysisXXXLevel II: Randomized con- trolled trialXLevel III: Controlled trial without randomizationLevel IV: Case-control or cohort studyXXLevel V: Systematic review of qualitative or descrip- tive studiesLevel VI: Qualitative or descriptive study (includes evidence implementation projects)XXXXXXXXXLevel VII: Expert opinion or consensusAdapted with permission from Melnyk BM, Fineout-Overholt E, editors. Evidence-based practice in nursing and healthcare: a guide to best practice. 2nd ed. Philadelphia: Wolters Kluwer Health / Lippincott Williams and Wilkins; 2010.1 = Chan PS, et al. (2010); 2 = McGaughey J, et al.; 3 = Winters BD, et al.; 4 = Hillman K, et al.; 5 = Sharek PJ, et al.; 6 = Chan PS, et al. (2009); 7 = DeVita MA, et al.; 8 = Mailey J, et al.; 9 = Dacey MJ, et al.; 10 = McFarlan SJ, Hensley S.; 11 = Offner PJ, et al.; 12 = Bertaut Y, et al.; 13 = Benson L, et al.; 14 = Hatler C, et al.; 15 = Bader MK, et al.ajn@wolterskluwer.comAJN ▼ November 2010 ▼ Vol. 110, No. 11 47

Table 3: Effect of the Rapid Response Team on Outcomes

1 = Chan PS, et al. (2010); 2 = McGaughey J, et al.; 3 = Winters BD, et al.; 4 = Hillman K, et al.; 5 = Sharek PJ, et al.; 6 = Chan PS, et al. (2009); 7 = DeVita MA, et al.; 8 = Mailey J, et al.; 9 = Dacey MJ, et al.; 10 = McFarlan SJ, Hensley S.; 11 = Offner PJ, et al.; 12 = Bertaut Y, et al.; 13 = Benson L, et al.; 14 = Hatler C, et al.; 15 = Bader MK, et al.

CR = cardiopulmonary arrest or code rates; CRO = code rates outside the ICU; HMR = hospital-wide mortality rates; NE = not evaluated; NR = not reported; UICUA = unplanned ICU admissions

a higher-level evidence; b statistically significant findings; c statistical significance not reported; d non-ICU mortality was reduced

The EBP team can tell from reading the evidence that research- ers consider the impact of an RRT on hospital-wide mortality rates (HMR) as the more important outcome; however, the group re- mains unconvinced that this out- come is the best for evaluating the purpose of an RRT, which, according to the IHI, is early in- tervention in patients who are unstable or at risk for cardiac or respiratory arrest.16 That said, of the 11 studies and projects that evaluated mortality, more than half found that an RRT reduced it. Carlos reminds the group that four of those six articles are level-VI evidence and that some weren’t research. The findings produced at this level of evidence are typi- cally less reliable than those at higher levels of evidence; how- ever, Carlos notes that two articles

having level-VI evidence, a study and a project, had statistically significant (less likely to occur by chance, P < 0.05) reductions in HMR, which increases the reli- ability of the results.

Chen asks, since four level-VI reports documented that an RRT reduces HMR, should they put more confidence in findings that occur more than once? Carlos re- plies that it’s not the number of studies or projects that determines the reliability of their findings, but the uniformity and quality of their methods. He recites something he heard in his Expert EBP Mentor program that helped to clarify the concept of making decisions based on the evidence: the level of the evidence (the design) plus the quality of the evidence (the validity of the methods) equals the strength of the evidence, which is

what leads clinicians to act in con- fidence and apply the evidence (or not) to their practice and expect similar findings (outcomes). In terms of making a decision about whether or not to initiate an RRT, Carlos says that their evidence stacks up: first, the MERIT study’s results are questionable because of problems with the study meth- ods, and this affects the reliability of the three systematic reviews as well as the MERIT study itself; second, the reasonably conducted lower-level studies/projects, with their statistically significant find- ings, are persuasive. Therefore, the team begins to consider the possibility that initiating an RRT may reduce code rates outside the ICU (CRO) and may impact non- ICU mortality; both are outcomes they would like to address. The evidence doesn’t provide equally

48 AJN ▼ November 2010 ▼ Vol. 110, No. 11

ajnonline.com

promising results for UICUA, but the team agrees to include it in the outcomes for their RRT proj- ect because it wasn’t evaluated

in most of the articles they ap- praised.

As the EBP team continues to discusses probable outcomes, Rebecca points to one study’s

data in the “Findings” column that shows a financial return on investment for an RRT.9 Carlos remarks to the group that this is only one study, and that they’ll need to make sure to collect data on the costs of their RRT as well as the cost implications of the outcomes. They determine that

the important outcomes to mea- sure are: CRO, non-ICU mortality (excluding patients with do not resuscitate [DNR] orders), UICUA, and cost.

Appraisal: Worth to Practice.

As the team discusses their syn- thesis and the decision they’ll make based on the evidence,

Table 4. Defined Criteria for Initiating an RRT Consult

	4	8	9	13	15
Respiratory distress (breaths/min)	Airway threatened Respiratory arrest RR < 5 or > 36	RR < 10 or > 30	RR < 8 or > 30Unexplained dys- pnea	RR < 8 or > 28New-onset difficulty breathing	RR < 10 or > 30 Shortness of breath
Change in mental status	Change in LOC Decrease in Glasgow Coma Scale of > 2 points	ND	Unexplained change	Sudden decrease in LOC with normal blood glucose	Decreased LOC
Tachycardia (beats/ min)	>140	> 130	Unexplained > 130 for 15 min	> 120	> 130
Bradycardia (beats/ min)	< 40	< 60	Unexplained < 50 for 15 min	< 40	< 40
Blood pressure (mmHg)	SBP < 90	SBP < 90 or > 180	Hypotension (unex- plained)	SBP > 200 or < 90	SBP < 90
Chest pain	Cardiac arrest	ND	ND	Complaint of nontrau- matic chest pain	Complaint of nontraumatic chest pain
Seizures	Sudden or extended	ND	ND	Repeated or pro- longed	ND
Concern/worry about patient	Serious concern about a patient who doesn’t fit the above criteria	NE	Nurse concern about overall deterioration in patients’ condi- tion without any of the above criteria(p. 2077)	Nurse concern	• Uncontrolled pain • Failure to respond totreatment • Unable to obtain promptassistance for unstable patient
Pulse oximetry (SpO2)	NE	NE	NE	< 92%	< 92%
Other				· Color change of patient· Unexplained agita- tion for > 10 min· CIWA > 15 points	• UOP < 50 cc/4 hr • Color change of patient(pale, dusky, gray, orblue) • New-onset limb weak-ness or smile droop • Sepsis: ≥2 SIRS criteria

4 = Hillman K, et al.; 8 = Mailey J, et al.; 9 = Dacey MJ, et al.; 13 = Benson L, et al.; 15 = Bader MK, et al.

cc = cubic centimeters; CIWA = Clinical Institute Withdrawal Assessment; hr = hour; LOC = level of consciousness; min = minute; mmHg = millimeters of mercury; ND = not defined; NE = not evaluated; RR = respiratory rate; SBP = systolic blood pressure; SIRS = systemic inflammatory response syndrome; SpO2= arterial oxygen saturation; UOP = urine output

ajn@wolterskluwer.com

AJN ▼ November 2010 ▼ Vol. 110, No. 11 49

Rebecca raises a question that’s been on her mind. She reminds them that in the “Appraisal: Worth to Practice” column, teaching was identified as an important factor in initiating an RRT and expresses concern that their hospital is not an academic medical center. Chen reminds her that even though theirs is not a designated teaching hospital with residents on staff 24 hours a day, it has a culture

of teaching that should enhance the success of an RRT. She adds that she’s already hearing a buzz

of excitement about their project, that their colleagues across all disciplines have been eager to hear the results of their review of the evidence. In addition, Carlos says that many resources in their hos- pital will be available to help them get started with their project and reminds them of their hospital administrators’ commitment to support the team.

ACTING ON THE EVIDENCE

As they consider the synthesis of the evidence, the team agrees

that an RRT is a valuable inter- vention to initiate. They decide to take the criteria for activating an RRT from several successful studies/projects and put them into a synthesis table to better see their major similarities (see Table 44, 8, 9, 13, 15). From this com- bined list, they choose the criteria for initiating an RRT consult that they’ll use in their project (see Table 5). The team also begins discussing the ideal make up for their RRT. Again, they go back to the evaluation table and look

Table 5. Defined Criteria for Initiating an RRT Consult at Our Hospital

Pulmonar y
Ventilation	Color change of patient (pale, dusky, gray, or blue)
Respiratory distress	RR < 10 or > 30 breaths/min or unexplained dyspnea or new-onset difficulty breathing or shortness of breath
Cardiovascular
Tachycardia	Unexplained > 130 beats/min for 15 min
Bradycardia	Unexplained < 50 beats/min for 15 min
Blood pressure	Unexplained SBP < 90 or > 200 mmHg
Chest pain	Complaint of nontraumatic chest pain
Pulse oximetry	< 92% SpO2
Perfusion	UOP < 50 cc/4 hr
Neurologic
Seizures	Initial, repeated, or prolonged
Change in mental status	• Sudden decrease in LOC with normal blood glucose • Unexplained agitation for > 10 min • New-onset limb weakness or smile droop
Concern/worry about patient	Nurse concern about overall deterioration in patients’ condition without any of the above criteria
Sepsis
	• Temp, > 38°C • HR, > 90 beats/min • RR, > 20 breaths/min • WBC, > 12,000, < 4,000, or > 10% bands

cc = cubic centimeters; hr = hours; HR = heart rate; LOC = level of consciousness; min = minute; mmHg = millimeters of mercury; RR = respiratory rate; SBP = systolic blood pressure; SpO2 = arterial oxygen saturation; Temp = temperature; UOP = urine output; WBC = white blood count

AJN ▼ November 2010 ▼ Vol. 110, No. 11

ajnonline.com

over the “Major Variables Studied” column, noting that the composition of the RRT varied among the studies/projects. Some

evidence that led to the project, how to call an RRT, and out- come measures that will indicate whether or not the implementation

3. Winters BD, et al. Rapid response sys- tems: a systematic review. Crit Care Med 2007;35(5):1238-43.

4.Hillman K, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised con- trolled trial. Lancet 2005;365(9477): 2091-7.

5. Sharek PJ, et al. Effect of a rapid re- sponse team on hospital-wide mortal- ity and code rates outside the ICU in a children’s hospital. JAMA 2007; 298(19):2267-74.

6.Chan PS, et al. Hospital-wide code rates and mortality before and after implementation of a rapid response team. JAMA 2008;300(21):2506-13.

7. DeVita MA, et al. Use of medical emergency team responses to reduce hospital cardiopulmonary arrests. Qual Saf Health Care 2004;13(4): 251-4.

8.Mailey J, et al. Reducing hospital standardized mortality rate with early interventions. J Trauma Nurs 2006; 13(4):178-82.

9. Dacey MJ, et al. The effect of a rapid response team on major clinical out- come measures in a community hos- pital. Crit Care Med 2007;35(9): 2076-82.

10. McFarlan SJ, Hensley S. Implementa- tion and outcomes of a rapid response team. J Nurs Care Qual 2007;22(4): 307-13.

11. Offner PJ, et al. Implementation of a rapid response team decreases cardiac arrest outside the intensive care unit. J Trauma 2007;62(5):1223-8.

12. Bertaut Y, et al. Implementing a rapid- response team using a nurse-to-nurse consult approach. J Vasc Nurs 2008; 26(2):37-42.

13. Benson L, et al. Using an advanced practice nursing model for a rapid re- sponse team. Jt Comm J Qual Patient Saf 2008;34(12):743-7.

14. Hatler C, et al. Implementing a rapid response team to decrease emergen- cies. Medsurg Nurs 2009;18(2):84-90, 126.

15. Bader MK, et al. Rescue me: saving the vulnerable non-ICU patient popu- lation. Jt Comm J Qual Patient Saf 2009;35(4):199-205.

16. Institute for Healthcare Improvement. Establish a rapid response team. n.d. http://www.ihi.org/IHI/topics/ criticalcare/intensivecare/changes/ establisharapidresponseteam.htm.

As they consider the synthesis of the evidence, the team agrees that an RRT is a valuable intervention to initiate.

RRTs had active physician partic- ipation (n = 6), some had desig- nated physician consultation on an as-needed basis (n = 2), and some were nurse-led teams (n = 4). Most RRTs also had a respira- tory therapist (RT). All RRT mem- bers had expertise in intensive care and many were certified in advanced cardiac life support (ACLS). They agree that their team will be comprised of ACLS- certified members. It will be led by an acute care nurse practi- tioner (ACNP) credentialed for advanced procedures, such as central line insertion. Members will include an ICU RN and an RT who can intubate. They also discuss having physicians will- ing to be called when needed. Although no studies or projects had a chaplain on their RRT, Chen says that it would make sense in their hospital. Carlos, who’s been on staff the longest

of the three, says that interdisci- plinary collaboration has been a mainstay of their organization. A physician, ACNP, ICU RN, RT, and chaplain are logical choices for their RRT.

As the team ponders the evi- dence, they begin to discuss the next step, which is to develop ideas for writing their project implementation plan (also called a protocol). Included in this pro- tocol will be an educational plan to let those involved in the proj- ect know information such as the

of the evidence was successful. They’ll also need an evaluation plan. From reviewing the studies and projects, they also realize that it’s important to focus their plan on evidence implementation, in- cluding carefully evaluating both the process of implementation and project outcomes.

Be sure to join the EBP team in the next installment of this se- ries as they develop their imple- mentation plan for initiating an RRT in their hospital, including the submission of their project proposal to the ethics review board. ▼

Ellen Fineout-Overholt is clinical pro- fessor and director of the Center for the Advancement of Evidence-Based Prac- tice at Arizona State University in Phoe- nix, where Bernadette Mazurek Melnyk is dean and distinguished foundation professor of nursing, Susan B. Stillwell is clinical associate professor and pro- gram coordinator of the Nurse Educator Evidence-Based Practice Mentorship Program, and Kathleen M. Williamson is associate director of the Center for the Advancement of Evidence-Based Practice. Contact author: Ellen Fineout- Overholt, ellen.fineout-overholt@asu. edu.

REFERENCES

1. Chan PS, et al. (2010). Rapid re- sponse teams: a systematic review and meta-analysis. Arch Intern Med 2010;170(1):18-26.

2. McGaughey J, et al. Outreach and early warning systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards. Cochrane Database Syst Rev 2007;3:CD005529.

ajn@wolterskluwer.com

AJN ▼ November 2010 ▼ Vol. 110, No. 11 51

Evidence-based practice (EBP) is an approach that enables psychiatric mental health care practitioners as well as all clinicians to provide the highest quality of care using the best evidence available (Melnyk & Fineout-Overholt, 2005). One of the key steps of EBP is to critically appraise evidence to best answer a clinical question. For many mental health questions, understanding levels of evidence, qualitative inquiry methods, and questions used to appraise the evidence are necessary to implement the best qualitative evi- dence into practice. Drawing conclusions and making judgments about the evidence are imperative to the EBP process and clinical decision making (Melnyk & Fineout-Overholt, 2005; Polit & Beck, 2008). The over- all purpose of this article is to familiarize clinicians with qualitative research as an important source of evidence to guide practice decisions. In this article, an overview of the goals, methods and types of qualita- tive research, and the criteria used to appraise the quality of this type of evidence will be presented.

QUALITATIVE BELIEFS

Qualitative research aims to generate insight, describe, and understand the nature of reality in

Kathleen M. Williamson, PhD, RN, associate director, Center for the Advancement of Evidence-Based Practice, Arizona State University, College of Nursing & Healthcare Innovation, Phoenix, Arizona; Kathleen.Williamson@asu.edu.

human experiences (Ayers, 2007; Milne & Oberle, 2005; Polit & Beck, 2008; Saddler, 2006; Sandelowski, 2004; Speziale & Carpenter, 2003; Thorne, 2000). Qualitative researchers are inquisitive and seek to understand knowledge about how people think and feel, about the circumstances in which they find themselves, and use methods to uncover and decon- struct the meaning of a phenomenon (Saddler, 2006; Thorne, 2000). Qualitative data are collected in a natural setting. These data are not numerical; rather, they are full and rich descriptions from participants who are experiencing the phenomenon under study. The goal of qualitative research is to uncover the truths that exist and develop a complete understand- ing of reality and the individual’s perception of what is real. This method of inquiry is deeply rooted in descriptive modes of research. “The idea that multiple realties exist and create meaning for the individuals studied is a fundamental belief of qualitative research- ers” (Speziale & Carpenter, 2003, p. 17). Qualitative research is the studying, collecting, and understand- ing the meaning of individuals’ lives using a variety of materials and methods (Denzin & Lincoln, 2005).

WHAT IS A QUALITATIVE RESEARCHER?

Qualitative researchers commonly believe that indi- viduals come to know and understand their reality in

202

TABLE 1. Most Commonly Used Qualitative Research Methods

Critical Appraisal of Qualitative Evidence

Method Purpose

Research question(s)

Sample size (on average) Data sources/collection

Ethnography Describe culture of people

What is it like to live . . . What is it . . . 30-50 Interviews, observations, field

notes, records, chart data, life histories

Phenomenology Describe phenomena, the

appearance of things, as lived experience of humans in a natural setting

What is it like to have this experience? What does it feel like?

6-8 Interviews, videotapes, observations,

in-depth conversations

Grounded theory To develop a theory rather than

describe a phenomenon

Questions emerge from the data

25-50 Taped interview, observation,

diaries, and memos from researcher

Source. Adapted from Polit and Beck (2008) and Speziale and Carpenter(2003).

different ways. It is through the lived experience and the interactions that take place in the natural setting that the researcher is able to discover and understand the phenomenon under study (Miles & Huberman, 1994; Patton, 2002; Speziale & Carpenter, 2003). To ensure the least disruption to the environ- ment/natural setting, qualitative researchers care- fully consider the best research method to answer the research question (Speziale & Carpenter, 2003). These researchers are intensely involved in all aspects of the research process and are considered participants and observers in setting or field (Patton, 2002; Polit & Beck, 2008; Speziale & Carpenter, 2003). Flexibility is required to obtain data from the richest possible sources of information. Using a holistic approach, the researcher attempts to cap- ture the perceptions of the participants from an “emic” approach (i.e., from an insider’s viewpoint; Miles & Huberman, 1994; Speziale & Carpenter, 2003). Often, this is accomplished through the use of a variety of data collection methods, such as inter- views, observations, and written documents (Patton, 2002). As the data are collected, the researcher simultaneously analyzes it, which includes identi- fying emerging themes, patterns, and insights within the data. According to Patton (2002), quali- tative analysis engages exploration, discovery, and inductive logic. The researcher uses a rich literary account of the setting, actions, feelings, and mean- ing of the phenomenon to report the findings (Patton, 2002).

COMMONLY USED QUALITATIVE DESIGNS

According to Patton (2002), “Qualitative methods are first and foremost research methods. They are ways of finding out what people do, know, think, and

feel by observing, interviewing, and analyzing docu- ments” (p. 145). Qualitative research designs vary by type and purpose: data collection strategies used and the type of question or phenomenon under study. To critically appraise qualitative evidence for its valid- ity and use in practice, an understanding of the types of qualitative methods as well as how they are employed and reported is necessary.

Many of the methods are routed in the anthropol- ogy, psychological, and sociology disciplines. Many commonly used methods in the health sciences research are ethnography, phenomenology, and grounded theory (see Table 1).

Ethnography

Ethnography has its traditions in cultural anthropology, which describe the values, beliefs, and practice of cultural groups (Ploeg, 1999; Polit & Beck, 2008). According to Speziale and Carpenter (2003), the characteristics that are central to eth- nography are that (a) the research is focused on culture, (b) the researcher is totally immersed in the culture, and (c) the researcher is aware of her/ his own perspective as well as those in the study. Ethnographic researchers strive to study cultures from an emic approach. The researcher as a par- ticipant observer becomes involved in the culture to collect data, learn from participants, and report on the way participants see their world (Patton, 2002). Data are primarily collected through obser- vations and interviews. Analysis of ethnographic results involves identifying the meanings attrib- uted to objects and events by members of the cul- ture. These meanings are often validated by members of the culture before finalizing the results (called member checks). This is a labor-intensive method that requires extensive fieldwork.

Journal of the American Psychiatric Nurses Association,Vol. 15, No. 3 203

Williamson

Phenomenology

Phenomenology has its roots in both philosophy and psychology. Polit and Beck (2008) reported, “Phenomenological researchers believe that lived experience gives meaning to each person’s percep- tion of a particular phenomenon” (p. 227). According to Polit and Beck, there are four aspects of the human experience that are of interest to the phe- nomenological researcher: (a) lived space (spatial- ity), (b) lived body (corporeality), (c) lived human relationships (relationality), and (d) lived time (tem- porality). Phenomenological inquiry is focused on exploring how participants in the experience make sense of the experience, transform the experience into consciousness, and the nature or meaning of the experience (Patton, 2002). Interpretive phenom- enology (hermeneutics) focuses on the meaning and interpretation of the lived experience to better understand social, cultural, political, and historical context. Descriptive phenomenology shares vivid reports and describes the phenomenon.

In a phenomenological study, the researcher is an active participant/observer who is totally immersed in the investigation. It involves gaining access to participants who could provide rich descriptions from in-depth interviews to gather all the informa- tion needed to describe the phenomenon under study (Speziale & Carpenter, 2003). Ongoing analyses of direct quotes and statements by participants occur until common themes emerge. The outcome is a vivid description of the experience that captures the meaning of the experience and communicates clearly and logically the phenomenon under study (Speziale & Carpenter, 2003).

Grounded Theory

Grounded theory has its roots in sociology and explores the social processes that are present within human interactions (Speziale & Carpenter, 2003). The purpose is to develop or build a theory rather than test a theory or describe a phenomenon (Patton, 2002). Grounded theory takes an inductive approach in which the researcher seeks to generate emergent categories and integrate them into a theory grounded in the data (Polit & Beck, 2008). The research does not start with a focused problem; it evolves and is discovered as the study progresses. A feature of grounded theory is that the data collection, data analysis, and sampling of participants occur simultaneously (Polit & Beck, 2008; Powers, 2005). The

researchers using ground theory methodology are able to critically analyze situations, not remove themselves from the study but realize that they are part of it, recognize bias, obtain valid and reliable data, and think abstractly (Strauss & Corbin, 1990).

Data collection is through in-depth interview and observations. A constant comparative process is used for two reasons: (a) to compare every piece of data with every other piece to more accurately refine the relevant categories and (b) to assure the researcher that saturation has occurred. Once saturation is reached the researcher connects the categories, pat- terns, or themes that describe the overall picture that emerged that will lead to theory development.

ASPECTS OF QUALITATIVE RESEARCH

The most important aspects of qualitative inquiry is that participants are actively involved in the research process rather than receiving an intervention or being observed for some risk or event to be quantified. Another aspect is that the sample is purposefully selected and is based on experience with a culture, social process, or phenomena to collect information that is rich and thick in descriptions. The final essential aspect of qualitative research is that one or more of the following strategies are used to collect data: interviews, focus groups, narratives, chat rooms, and observation and/or field notes. These methods may be used in combination with each other. The researcher may choose to use triangulation strategies on data collection, investigator, method, or theory and use multiple sources to draw conclusions about the phenomenon (Patton, 2002; Polit & Beck, 2009).

SUMMARY

This is not an inclusive list of qualitative methods that researchers could choose to use to answer a research question, other methods include historical research, feminist research, case study method, and action research. All qualitative research methods are used to describe and discover meaning, understand- ing, or develop a theory and transport the reader to the time and place of the observation and/or inter- view (Patton, 2002).

THE HIERARCHY OF QUALITATIVE EVIDENCE

Clinical questions that require qualitative evi- dence to answer them focus on human response and

204 Journal of the American Psychiatric Nurses Association,Vol. 15, No. 3

Critical Appraisal of Qualitative Evidence

TABLE 2.

How were they selected?

Was it adequate?

How were the data collected?

Subquestions to Further Answer, Are the Study Findings Valid?

Participants Sample

Data collection

Did they provide rich and thick descriptions?

Was the setting appropriate to acquire an adequate sample?

Were the tools adequate?

Were the participants’ rights protected?

Was the sampling method

appropriate?

How were the data coded? If so how?

Did the researcher eliminate bias?

Do the data accurately represent the study participants?

How accurate and complete were the data?

Was the group or population adequately described?

Was saturation achieved?

Does gathering the data adequately portray the phenomenon?

Source. Adapted from Powers (2005), Polit and Beck (2008), Russell and Gregory (2003), and Speziale and Carpenter (2003).

meaning. An important step in the process of appraising qualitative research as a guide for clinical prac- tice is the identification of the level of evidence or the “best” evidence. The level of evidence is a guide that helps identify the most appropriate, rigorous, and clinically relevant evidence to answer the clinical question (Polit & Beck, 2008). Evidence hierarchy for qualitative research ranges from opinion of authori- ties and/or reports of expert committees to a single qualitative research study to metasynthesis (Melnyk & Fineout-Overholt, 2005; Polit & Beck, 2008). A metasynthesis is comparable to meta-analysis (i.e., systematic reviews) of quantitative studies. A meta- synthesis is a technique that integrates findings of multiple qualitative studies on a specific topic, pro- viding an interpretative synthesis of the research findings in narrative form (Polit & Beck, 2008). This is the strongest level of evidence in which to answer a clinical question. The higher the level of evidence the stronger the evidence is to change practice. However, all evidence needs be critically appraised based on (a) the best available evidence (i.e., level of evidence), (b) the quality and reliability of the study, and (c) the applicability of the findings to practice.

CRITICAL APPRAISAL OF QUALITATIVE EVIDENCE

Once the clinical issue has been identified, the PICOT question constructed, and the best evidence located through an exhaustive search, the next step is to critically appraise each study for its validity (i.e., the quality), reliability, and applicability to use in practice (Melnyk & Fineout-Overholt, 2005). Although there is no consensus among qualitative researchers on the quality criteria (Cutcliffe & McKenna, 1999; Polit & Beck, 2008; Powers, 2005; Russell & Gregory, 2003; Sandelowski, 2004), many have published excellent tools that guide the process

for critically appraising qualitative evidence (Duffy, 2005; Melnyk & Fineout-Overholt, 2005; Polit & Beck, 2008; Powers, 2005; Russell & Gregory, 2003; Speziale & Carpenter, 2003). They all base their cri- teria on three primary questions: (a) Are the study findings valid? (b) What were the results of the study? (c) Will the results help me in caring for my patients? According to Melnyk and Fineout-Overholt (2005), “The answers to these questions ensure rele- vance and transferability of the evidence from the search to the specific population for whom the practi- tioner provides care” (p. 120). In using the questions in Tables 2, 3, and 4, one can evaluate the evidence and determine if the study findings are valid, the method and instruments used to acquire the knowl- edge credible, and if the findings are transferable.

The qualitative process contributes to the rigor or trustworthiness of the data (i.e., the quality). “The goal of rigor in qualitative research is to accurately represent study participants’ experiences” (Speziale & Carpenter, 2003, p. 38). The qualitative attributes of validity include credibility, dependability, confirm- ability, transferability, and authenticity (Guba & Lincoln, 1994; Miles & Huberman, 1994; Speziale & Carpenter, 2003).

Credibility is having confidence and truth about the data and interpretations (Polit & Beck, 2008). The credibility of the findings hinges on the skill, competence, and rigor of the researcher to describe the content shared by the participants and the abil- ity of the participants to accurately describe the phenomenon (Patton, 2002; Speziale & Carpenter, 2003). Cutcliffe and McKenna (1999) reported that the most important indicator of the credibility of findings is when a practitioner reads the study find- ings and regards them meaningful and applicable and incorporates them into his or her practice.

Confirmability refers to the way the researcher documents and confirms the study findings (Speziale

Journal of the American Psychiatric Nurses Association,Vol. 15, No. 3 205

Williamson

TABLE 3. Subquestions to Further Answer, What Were the Results of the Study?

Was the purpose of the study clear?

Is the research design

appropriate for the research question?

Is the description of findings thorough?

Do findings fit the data from which they were generated? follow?

Were all themes identified, useful, creative, and convincing of the phenomena?

Are the results logical,

consistent, and easy to

Source. Adapted from Powers (2005), Russell and Gregory (2003), and Speziale and Carpenter (2003). TABLE 4. Sub-questions to Further Answer, Will the Results Help Me in Caring for My Patients?

What meaning and relevance does this study have for my patients?

How would I use these findings in my practice?

How does the study help provide perspective on my practice?

Are the conclusions appropriate to my patient population?

Are the results applicable to my patients?

How would patient and family values be considered in applying these results?

Source. Adapted from Powers (2005), Russell and Gregory (2003), and Speziale and Carpenter (2003).

& Carpenter, 2003). Confirmability is the process of confirming the accuracy, relevance, and meaning of the data collected. Confirmability exists if (a) the researcher identifies if saturation was reached and (b) records of the methods and procedures are detailed enough that they can be followed by an audit trail (Miles & Huberman, 1994).

Dependability is a standard that demonstrates whether (a) the process of the study was consistent, (b) data remained consistent over time and conditions, and (c) the results are reliable (Miles & Huberman, 1994; Polit & Beck, 2008; Speziale & Carpenter, 2003). For example, if study methods and results are depend- able, the researcher consistently approaches each occurrence in the same way with each encounter and results were coded with accuracy across the study.

Transferability refers to the probability that the study findings have meaning and are usable by oth- ers in similar situations (i.e., generalizable to others in that situation; Miles & Huberman, 1994; Polit & Beck, 2008; Speziale & Carpenter, 2003). To deter- mine if the findings of a study are transferable and can be used by others, the clinician must consider the potential client to whom the findings may be applied (Speziale & Carpenter, 2003).

Authenticity is when the researcher fairly and faithfully shows a range of different realities and develops an accurate and authentic portrait for the phenomenon under study (Polit & Beck, 2008). For example, if a clinician were to be in the same

environment as the researcher describes, they would experience the phenomenon similarly. All mental health providers need to become familiar with these aspects of qualitative evidence and hone their criti- cal appraisal skills to enable them to improve the outcomes of their clients.

CONCLUSION

Qualitative research aims to impart meaning of the human experience and understand how people think and feel about their circumstances. Qualitative researchers use a holistic approach in an attempt to uncover truths and understand a person’s reality. The researcher is intensely involved in all aspects of the research design, collection, and analysis pro- cesses. Ethnography, phenomenology, and grounded theory are some of the designs that a researcher may use to study a culture, phenomenon, or theory. Data collection strategies vary based on the research question, method, and informants. Methods such as interviews, observations, and journals allow for information-rich participants to provide detailed lit- erary accounts of the phenomenon. Data analysis occurs simultaneously as data collection and is the process by which the researcher identifies themes, concepts, and patterns that provide insight into the phenomenon under study.

One of the crucial steps in the EBP process is to critically appraise the evidence for its use in practice

206 Journal of the American Psychiatric Nurses Association,Vol. 15, No. 3

Based Nursing, 3, 68-70. For reprints and permission queries, please visit SAGE’s Web site at http://www.sagepub.com/journalsPermissions.nav.

Critical Appraisal of Qualitative Evidence

and determine the value of findings. Critical appraisal is the review of the evidence for its validity (i.e., strengths and weaknesses), reliability, and usefulness for clients in daily practice. “Psychiatric mental health clinicians are practicing in an era emphasizing the use of the most current evidence to direct their treatment and interventions” (Rice, 2008, p. 186). Appraising the evidence is essential for assurance that the best knowledge in the field is being applied in a cost-effective, holistic, and effective way. To do this, one must incorporate the critically appraised findings with their abilities as clinicians and their clients’ preferences. As professionals, clinicians are expected to use the EBP process, which includes appraising the evidence to determine if the best results are believable, useable, and dependable. Clinicians in psychiatric mental health must use qualitative evidence to inform their practice deci- sions. For example, how do clients newly diagnosed with bipolar and their families perceive the life impact of this diagnosis? Having a well done meta- synthesis that provides an accurate representation of the participants’ experiences, and is trustworthy (i.e., credible, dependable, confirmable, transferable, and authentic), will provide insight into the situational context, human response, and meaning for these cli- ents and will assist clinicians in delivering the best care to achieve the best outcomes.

REFERENCES

Ayers, L. (2007). Qualitative research proposals—Part I. Journal Wound Ostomy Continence Nursing, 34, 30-32.

Cutcliffe, J. R., & McKenna, H. P. (1999). Establishing the credibil- ity of qualitative research findings: The plot thickens. Journal of Advanced Nursing, 30, 374-380.

Denzin, N. K., & Lincoln, Y. S. (2005). The Sage handbook of qualitative research (3rd ed.). Thousand Oaks, CA: Sage.

Duffy, M. E. (2005). Resources for critically appraising qualitative research evidence of nursing practice clinical question. Clinical Nursing Specialist, 19, 288-290.

Guba, E. G., & Lincoln, Y. S. (1994). Competing paradigms in qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 105-117). Thousand Oaks, CA: Sage.

Melnyk, B. M., & Fineout-Overholt, E. (Eds.). (2005). Evidence-based practice in nursing and healthcare. Philadelphia: Lippincott Williams & Wilkins.

Miles, M. B., & Huberman, A. M. (1994). An expend sourcebook qualitative data analysis (4th ed.). Thousand Oaks, CA: Sage.

Milne, J., & Oberle, K. (2005). Enhancing rigor in qualitative description: A case study. Journal Wound Ostomy Continence Nursing, 32, 413-420.

Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks: Sage.

Ploeg, J. (1999). Identifying the best research design to fit the question. Part 2: Qualitative designs. Evidence-Based Nursing, 2, 36-37.

Polit, D. F., & Beck, C. T. (2008). Nursing research: Generating and assessing evidence fro nursing practice. Philadelphia: Lippincott Williams & Wilkins.

Powers, B. A. (2005). Critically appraising qualitative evidence. In B. M. Melnyk & E. Fineout-Overholt (Eds.), Evidence-based practice in nursing and healthcare (pp. 127-162). Philadelphia: Lippincott Williams & Wilkins.

Rice, M. J. (2008). Evidence-based practice in psychiatric care: Defining levels of evidence. Journal of the American Psychiatric Nurses Association, 14(3), 181-187.

Russell, C. K., & Gregory, D. M. (2003). Evaluation of qualitative research studies. Evidence-Based Nursing, 6, 36-40.

Saddler, D. (2006). Research 101. Gastroenterology Nursing, 30, 314-316.

Sandelowski, M. (2004). Using qualitative research. Qualitative Health Research, 14, 1366-1386.

Speziale, H. J. S., & Carpenter, D. R. (2003). Qualitative research in nursing: Advancing the humanistic imperative. Philadelphia: Lippincott Williams & Wilkins.

Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. London: Sage. Thorne, S. (2000). Data analysis in qualitative research. Evidence-

Journal of the American Psychiatric Nurses Association,Vol. 15, No. 3 207

Chapter 7. Clinical Issues, Challenges, and Strategies in Intensive Outpatient Treatment

Once clients are engaged actively in treatment, retention becomes a priority. Many obstacles may arise during treatment. Lapses may occur. Frequently, clients are unable or unwilling to adhere to program requirements. Repeated admissions and dropouts can occur. Clients may have conflicting mandates from various service systems. Concerns about client and staff relationships, including setting appropriate boundaries, can compromise care. Intensive outpatient treatment (IOT) programs need to have clear decisionmaking processes and retention strategies to address these and other circumstances.

This chapter discusses common issues that IOT programs face and offers practical approaches to retaining clients in treatment. Experience has taught IOT clinicians that every problem can have many solutions and that the input and ideas of colleagues lead to creative approaches and solutions. The chapter presents specific scenarios and options from clinical practice and experience for clinicians to consider, modify, or implement.

Go to:

Client Retention

Reducing client attrition during treatment must be a priority for IOT providers. Compared with clients who drop out, those who are retained in outpatient treatment tend to be White, male, and employed (McCaul et al. 2001). Client attributes associated with higher dropout rates are labeled “red flags” by White and colleagues (1998); these red flags include marginalized status (e.g., racial minorities, people who are economically disadvantaged), lack of a professional skill, recent hospitalization, and family history of substance abuse. Being aware of these red flags can help clinicians intervene early to assist clients at increased risk of dropping out. Veach and colleagues (2000) found that clients who abuse alcohol were more likely to be retained and those who abuse cocaine were less likely to be retained in outpatient treatment. Other studies have found that the substance a client abuses is not a good predictor of retention (McCaul et al. 2001).

The following strategies improve retention of clients in treatment:

· Form a working relationship with the client. The counselor should foster a respectful and understanding relationship with the client. This therapeutic relationship reduces resistance and successfully engages the client in working toward mutually defined treatment goals.

· Learn the client’s treatment history. If the client has dropped out of treatment previously, the counselor should find out why. If the client has engaged and been retained successfully in treatment before, the counselor should ask what made treatment appealing.

· Use motivational interviewing. The counselor should help clients work through ambivalence by supporting their efforts to change and helping them identify discrepancies between their goals and values and their substance use. Involving clients in activities, such as support groups, also is effective.

· Provide flexible schedules. IOT providers need to consider the client populations they serve and schedule groups accordingly. For example, morning groups can be for clients who work swing and night shifts and for women with school-age children and evening groups for those working regular business hours. It can be difficult for clients to fit many hours of treatment into their week.

Multiple Retention Challenges

Clinical issue. A man, age 35, single, and an immigrant from El Salvador, has failed to return to treatment or contact his counselor in the last 3 days.

Approach

The counselor writes a note to the client in Spanish, encouraging him to return to treatment.

The counselor arranges for the client to get a ride to the next group session and for public transportation vouchers for subsequent sessions.

The counselor schedules an individual counseling session for the client to discuss several retention problems, which include lack of transportation, language barriers, and shame over lapses to his previous drinking pattern.

· Use the group to engage and reengage the client. The counselor should encourage members to talk about their ambivalence, how they are overcoming it, and their experiences of dropping out of treatment, as well as the negative consequences of dropping out. The counselor can supply all group members with an updated telephone list and encourage them to talk to at least two other members daily. The counselor can ask members to call those who are absent to let them know that they were missed and are important to the group. It is important to check with clients to be sure that they are receptive to these phone calls; some may view them as intrusive and disrespectful.

· Increase the frequency of contact during the early treatment period. Clients often feel vulnerable or ambivalent during the first few weeks of treatment. Counselors need to contact each client frequently during this period to enhance retention. These contacts can be brief and made by telephone, e-mail, or letter. At the same time, counselors should encourage clients to contact other group members to reinforce the value of reaching out for support.

· Use network interventions. Counselors need to work with individuals in the community who are invested in the client’s recovery to encourage the client to stay in treatment. These individuals can be probation officers, ministers, employee assistance program counselors, friends, and co-workers. If the program identifies supportive individuals early in treatment and obtains a written consent for release of information from the client, the counselor can ask these individuals to encourage the client to attend sessions or increase his or her commitment to recovery.

· Deliver additional services throughout the treatment period. Fishman and colleagues (1999) found that attrition was lower during the intensive “services-loaded” phase of IOT and, conversely, that attrition increased during the less rigorous program phases.

· Never give up. The counselor should make continual efforts to follow up with clients who have dropped out. Successful techniques include telephone calls, letters, and home visits to encourage the client to return to the program. This level of dedication can affect the client’s attitude and willingness to complete treatment.

Go to:

Relapse and Continued Substance Use

Lapses often happen in the difficult early months in treatment. These brief returns to substance use can be used as a therapeutic tool; the goal is to keep them from becoming full relapses with a return to substance use. IOT clients living in the community are exposed to pressures to relapse, often while struggling with cravings and their own resistance to change. Clients need to use relapse prevention strategies when they are exposed to alcohol and drugs, experience cravings, are encouraged by others to return to substance use, or are exposed to personal relapse triggers (Irvin et al. 1999). (See appendix 7-A for descriptions of several instruments for assessing clients’ relapse potential.)

The Difference Between a Lapse and Relapse

Jack’s experience: A lapse.

Jack comes to group distressed because he drank on the weekend. He has been abstinent for 2 months and is concerned that he has jeopardized his employment and the return of his driver’s license. He discusses the episode with his counselor, and they identify treatment options. The therapeutic goal is to reinforce Jack’s desire to stay abstinent, and the episode becomes an opportunity to strengthen his relapse prevention skills.

This is a lapse, that is, a brief return to substance use following a sustained period of abstinence (a month or more). The client still is committed to his recovery and has not experienced loss of control. The event is used to help the client identify relapse triggers and increase his understanding and ability to withstand pressures to use substances.

Phil’s experience: A relapse.

Phil is in treatment for methamphetamine use. He has disappeared from treatment again.

When he returns, he is hyperactive, has a positive drug test, and refuses to talk about the test results or his return to drug use. He then fails again to return to the program. He is seen on the street obviously intoxicated. The compulsion to use is strong.

This is a relapse, that is, a prolonged episode of substance use during which the client is not open to therapeutic intervention or learning. Often a relapse can lead to dropout and indicates a continuing struggle by the client with his or her disease.

General relapse prevention strategies are to

· Educate clients and their family members about addiction and recovery. Clients and family members need information about the disease of addiction and its stages, cues to relapse, early signs of relapse, how addiction affects relationships, and how to find resources for support (e.g., Al-Anon). Counselors need to enlist the support of family members and significant others to keep them from sabotaging treatment. Family members need advice on how to support the client in recovery and how to cease enabling behaviors.

· Conduct an early assessment of specific relapse triggers. Together with the counselor, clients can conduct a functional analysis of their substance use, working to identify and understand with whom, where, when, and why they use substances. Functional analysis is a tool that identifies not only clients’ high-risk circumstances for substance use but also the ways in which triggers are linked to the effects that substance use produces. TIP 33, Treatment for Stimulant Use Disorders (CSAT 1999e ), and TIP 35, Enhancing Motivation for Change in Substance Abuse Treatment (CSAT 1999c ), explain how to perform a functional analysis.

· Develop a relapse prevention plan immediately. A relapse prevention plan should include coping strategies developed by the counselor and client, such as going to support group meetings, avoiding places where the client used substances in the past, identifying good things about a substance-free life, and telephoning the client’s sponsor regularly. TIP 33 (CSAT 1999e ) contains information and worksheets to develop a relapse prevention plan. Technical Assistance Publication (TAP) 8, Relapse Prevention and the Substance-Abusing Criminal Offender (Gorski et al. 1993), and TAP 19, Counselor’s Manual for Relapse Prevention With Chemically Dependent Criminal Offenders (Gorski and Kelley 1996), are helpful in developing a relapse prevention plan.

· Provide intensive monitoring and support. These activities include random drug testing (including urine samples that are collected under observation of program staff to prevent tampering), family counseling or education sessions about supporting the client during and after treatment, and the client’s self-monitoring of exposure and response to substance use triggers.

· Evaluate and review all slips and lapses. Despite their negative consequences, lapses can be used therapeutically. The counselor and client can learn more about what constitutes high-risk situations for the client. The client needs to consider the slip or lapse a discrete, unique event that does not need to be repeated or continued. The client should remember that abstinence can be regained and that the client can renew his or her commitment to abstinence. Clients should be reminded to contact the counselor, other group members, their sponsor, or other mutual-help group members when they sense that they are verging on relapse.

· Use the behavioral contract with clients. A behavioral contract spells out treatment expectations and goals, the rewards when goals are met, and the consequences if the contract is broken. The counselor should involve clients in writing the contract, encouraging them to use their own words. The behavioral contract helps bind clients to their commitment to abstinence and change. TIP 35 (CSAT 1999c ) provides more information on behavioral contracts.

· Introduce the stages of change. Marlatt and Gordon (1985) and Prochaska and colleagues (1994) recommend using relapse prevention interventions that are matched to the client’s stage of change. Joe and colleagues (1998) and Connors and colleagues (2001a ) argue that for clients who are ambivalent about abstinence, for example, initial interventions might focus on strengthening their resolve by analyzing the pros and cons of use, rolling with resistance, and never directly confronting clients. Subsequent interventions support abstinence by altering stimulus control and developing skills for negotiating high-risk situations. After a client experiences a period of abstinence, emphasis shifts to lifestyle modifications that promote long-term abstinence.

A Relapse Prevention Quiz

This quiz can be a tool to support and strengthen a client’s readiness to avoid relapse. Having senior members in a group answer the questions reinforces their knowledge while they educate newer members in relapse prevention skills.

• What might you say to co-workers if they ask you to have a drink or get high with them?

• Craving a drink or drug is quite natural for people who are dependent on alcohol or drugs. What three things can you do to get past the craving?

• What are three common reasons for feeling that you don’t belong in a support group such as Alcoholics Anonymous (AA) or Narcotics Anonymous (NA)?

• What two things can you do if someone at an AA or NA meeting annoys you?

• Why must recovery from your disease be your highest priority?

• What three qualities should you look for in a sponsor?

• Emotional discomfort takes a variety of forms. What are the three biggest problems for you? Anger, depression, self-pity, loneliness, boredom, worry, frustration, shame, guilt, or another emotion?

• What three things can you do to handle each emotional discomfort you identified?

• What are the key elements of an assertive response when offered alcohol or drugs?

• Why is it important to avoid starting romantic relationships during early recovery?

Multiple Dropouts and Readmissions

Some clients relapse or drop out of treatment and return repeatedly to treatment before they achieve a stable recovery. Providers may be reluctant to keep offering scarce treatment resources to the same individuals or to readmit individuals who drop out continually. Programs can respond to multiple dropouts and readmissions strategically by

• Conducting a comprehensive evaluation of each client to determine whether IOT is the appropriate level of care. Some clients, for example, may benefit from a brief inpatient placement to ready them for IOT (see chapter 5).

• Reviewing the client’s cycle of dropouts and admissions. Several cycles may be appropriate for a client with severe, complex needs and issues. Arbitrary rules regarding the number of permitted admissions and dropouts may be too rigid to support recovery of a severely impaired individual.

• Establishing an admissions committee to review and recommend action regarding clients who seek readmission following repeated dropouts. The committee can include staff and alumni representatives.

• Developing a profile of clients likely to drop out and designing a plan for them.

• Arranging a psychiatric evaluation for the client, which may indicate that psychiatric treatment and medication are required.

Go to:

Substance Use by Family Members

A client may have one or more family members who also actively abuse substances. In fact, research shows that individuals with substance use disorders are more likely than others to have family histories of substance use disorders (Johnson and Leff 1999). The client may be in regular contact with members of the extended family, a close friend, spouse, or a boyfriend or girlfriend who uses substances. Active substance use by someone living in the same place as the client or who is part of the client’s social support network clearly threatens a client’s recovery. The IOT counselor can consider using these options:

· Stay alert for others using substances. Construct and update regularly a genogram or social network assessment (see chapter 6) to identify possible substance use among family members, significant others, and friends who are likely to influence the client’s recovery. Gather information from the family and client about the nature, extent, and frequency of any substance use.

· Request that the family and client develop an agreement about substance use in the home. It is important to enlist family members in the treatment process to help the client and any other family members who are using substances (see chapter 6). A substance use agreement, signed by family members, identifies substances that will not be kept or consumed in the home and the consequences for violating the agreement. Part of the agreement can be to report all substance use to IOT program staff for discussion during group and individual sessions.

· Assist the client in identifying alternative housing if needed. Recovery homes, halfway houses, and shelters, among others, may be necessary temporary alternatives for a client who needs alcohol- or drug-free housing during and after treatment. If the client’s recovery is undermined continually in current housing, the counselor should consider such a housing referral.

· Provide information about treatment to a family member who needs it. Offer information about treatment options or referrals to a family member with a substance use disorder in a manner that ensures the privacy of the individual and does not divert attention from the client’s treatment and recovery.

Go to:

Group Work Issues

Group work is a core service of IOT and offers many opportunities for educating, supporting, and nurturing clients. Clients’ feelings toward their peers are important factors in shaping the way clients view the treatment experience. Clients are more likely to continue with treatment when they feel accepted, supported, and “normal” and receive empathy and kindness from others in the treatment group.

Many issues can affect group work and impede the progress of clients. For example, clients may be disruptive or withdrawn, have poor English or comprehension skills, and attend sessions sporadically. TIP 41, Substance Abuse Treatment: Group Therapy, provides additional information on working with clients in therapeutic groups (CSAT 2005f ).

Developing Group Cohesion

Group cohesion can be a central element in a client’s recovery process. Frequent changes in group membership make it difficult to build group cohesion. Washton (1997) suggests that frequent shifting of clients among groups can result in higher dropout rates. This observation argues for limiting changes in group composition that sometimes occur in a “phased” or “stage-oriented” IOT program. Adding new clients to groups generates challenges for the counselor who must become oriented to new clients. The following approaches help create effective IOT groups and group cohesion:

· Create group rituals. When new clients join a group or others depart, group rituals promote a sense of acceptance, safety, and support. Current members should orient new members to group rules and speak about their group experience. A ritual can mark a client’s graduation from the program and celebrate his or her success. Departure rituals may include a client’s demonstration of recovery knowledge and skills, a group discussion of the departing client’s strengths and how group members can be supportive, a review of the client’s relapse prevention plan and options if the plan should fail, and presentation of the program’s emblem (see below).

· Institute a program emblem. Staff and clients can design a program emblem to build and sustain group cohesion. The emblem is a visual symbol that represents the essence of the treatment program. For example, a coin, badge, or cup might be inscribed with a recovery motto such as “Serenity and Strength Day by Day” or “Hope, Freedom, and Recovery.” A logo might feature the rising sun, a stately oak, or clasped hands. These emblems can incorporate and reflect various cultural and ethnic values and designs. Some programs leave space in the emblem to inscribe each client’s name and his or her program completion date. Programs that have emblems have found that clients keep them and use them as reminders of their commitment to recovery and their success in remaining abstinent. The emblem and motto should convey a message of support while maintaining the confidentiality of the client (e.g., by not including the name of the treatment program).

· Explore the group’s feelings about clients who drop out. When a member relapses and drops out of the group, the group provides a safe environment for other members to discuss their feelings or fears about failure and relapse and their own relapse prevention strategies. Because a client’s perception of his or her ability to complete the program influences the outcome, counselors need to support group members with positive statements about their potential to do well in treatment.

· Encourage identification with the program in addition to the group. It can be helpful if clients develop a sense of belonging to the group and the treatment program. For instance, IOT staff can share information about the overall goals of the program, use guest counselors or supervisors to co-facilitate groups, and encourage former clients to return to share their experiences. Contacts with alumni outside treatment can be valuable, too.

· Maintain effective group size and staffing. The ideal adult IOT group consists of 8 to 12 clients, although up to 15 clients may be on the group roster (CSAT 2005f ). Programs may need to adjust group sizes according to staff resources, the availability of co-therapists, the experience of the counselors, and the composition of the client population (e.g., adult or adolescent, women or men, people with co-occurring mental disorders).

At least one therapist should have the required academic credentials for group therapy; a co-therapist can be an intern or trainee who assists with managing client behaviors and observing the dynamics of the group.

Example of a Sendoff for a Treatment Program Graduate

As a client leaves treatment, he or she is invited to take a marble from a bowl of marbles. The group leader then tells the graduate: “Now that you have begun this new stage in your recovery, keep this marble with you always—perhaps in your pocket or purse. Keep it where you will see it often to remind you of how hard your addiction was on you and your family. More important, it will remind you of how firm and resolved you must be in your commitment to stay clean and work on a healthy recovery program.

“Each time you reach into your pocket or purse and touch that marble, you will be reminded of the hard times that are behind you and those that may lie ahead. If, after all this, you decide that you do not care about the hard times and suffering that your addiction has caused and may cause again, and you decide that you want to sink back down into the mess of your addiction, then take the marble and toss it as far as you can, because you will have already lost the rest of your marbles!”

Preparing Clients for Group

IOT programs should orient new clients about how group therapy is conducted and how they are to use the group counseling sessions (see chapter 4). One way to do this is with a pregroup interview that allows the counselor to assess clients’ readiness for treatment, learn more about clients’ circumstances, and help shape clients’ expectations by answering questions and supplying information (CSAT 2005f ). This information should include group norms and expectations and be reviewed with clients so that it is clear from the outset. Programs also should consider posting group norms on the wall of the meeting room and having clients read them aloud at the beginning of each group session.

Working With Uncommitted, Ambivalent Clients

Some clients in group treatment may not be committed to their recovery from substance use disorders. Clients who have been mandated to treatment by the justice system may feel that they do not have a problem but are only following a judge’s orders. Some clients may be late habitually or talk about their continuing interest in a substance-abusing lifestyle. The counselor cannot permit the client to attend group while under the influence of drugs or alcohol because this behavior can compromise the progress of other members of the group. However, the counselor can address behaviors displayed by uncommitted clients by

· Discussing the behaviors with the client individually to identify the issues and discuss options

· Moving the client to a precontemplator or other group or terminating the client from the program

· Introducing more structure into the group to enhance its therapeutic value for all members (e.g., by combining theme-oriented information with client discussion and concentrating less on process and more on organized content)

Working With Clients Who Have Severe Mental Disorders

Individuals diagnosed with severe mental disorders often require a high level of management by trained medical and substance abuse treatment professionals. These clients may have difficulty bonding with a group and may be disruptive or unable to focus for long periods. To enhance the effectiveness of group for individuals diagnosed with severe mental disorders, IOT providers are encouraged to consider these approaches:

· Treatment should be coordinated with the client’s psychiatric care provider to determine how best to respond to crises that may arise during group.

· Group treatment should be guided by clients’ readiness for and ability to engage in group work (Substance Abuse and Mental Health Services Administration 2002).

· Group treatment staff members should be educated and trained about mental disorders so that they are familiar with the signs and symptoms of psychoses and crisis intervention techniques.

For more information about treating this population, see chapter 9 of this volume or TIP 42, Substance Abuse Treatment for Persons With Co-Occurring Disorders (CSAT 2005e).

Treating Individuals Who Have Severe Mental Disorders

Sam increasingly was unable to control his outbursts when in group. Although he usually was able to return to a calm state, the incidents persisted. His counselor was aware that Sam experienced hallucinations and, with input from Sam’s psychiatrist, determined that Sam was receiving little benefit from being in a group. His treatment plan was revised to increase his individual counseling sessions in place of group participation.

Marjorie was diagnosed with bipolar disorder and functioned well while taking prescribed medications. Her counselor noticed behavior changes in group (such as flirting with male members, hyperactivity) over several days. After Marjorie was referred to her psychiatrist, it was determined that she had stopped taking her medications. After she resumed taking her medications, her symptoms disappeared.

Working With Disruptive Clients

Clients in group express a wide range of feelings, thoughts, and behaviors. Some members may disrupt the work of the group by challenging or interrupting others, demonstrating their impatience and restlessness, or otherwise offending other group members. Some strategies to address these disruptions are to

· Ensure that all clients know the group rules; provide them in writing, if possible.

· Consistently point out group rules about disruptive behaviors and the consequences for engaging in them.

· Reassess the client’s level of readiness to change, and assign the client to another group if appropriate.

· Hold individual counseling sessions to discuss specific disruptive behaviors, how they are disruptive, and why they are not allowed; then explore and identify factors that may underlie the behaviors.

· Refer the client to a mental health professional if needed.

The Angry Client in Group

Problem behaviors

• Yelling

• Foul language

• Interrupting

• Being mean or insulting to others

Key concepts for counselors

• Be in control.

• Avoid a power struggle.

• Address the behavior, not the content.

• Don’t raise your voice.

What to do

Listen reflectively to validate the client’s feelings and to deescalate the situation. If the client remains angry, use these approaches:

• State that you are there to protect and safeguard the members of the group.

• Identify specific behaviors that are inappropriate.

• State that these behaviors are not allowed.

• Identify the consequences if the behaviors continue (e.g., being removed from the group, not being permitted to participate in discussion for the remainder of the group session).

• Follow through with the stated consequences if the behaviors are repeated.

• Transfer the client to a different group or clinical service.

Working With Quiet, Withdrawn Clients

Clients may be reluctant to participate in group therapy for many reasons. They may be fearful or ashamed of revealing to strangers the extent of their substance use and related behaviors. Cultural values may inhibit the sharing of personal problems with those outside the family. Language and comprehension barriers may make it difficult to follow or participate in the conversation.

Clients may refuse to take part in group discussions beyond the level of perfunctory comments because they resent being in treatment, are depressed or have some other mental disorder, find the group boring, or are uncomfortable in a group. Some clients resist treatment because they believe that they do not have a disease or do not belong in treatment.

Some strategies to assist withdrawn clients are to

· Ask clients individually why they are quiet; then explore options based on the feedback.

· Assess and diagnose language and comprehension skills, and assign clients to a group that functions at an appropriate pace and level.

· Provide individual mentoring to ensure that treatment information is conveyed and understood.

· Create a “buddy system,” pairing clients to encourage a sense of acceptance and belonging among the members of the group.

· Contract with the client to increase participation in the group incrementally.

· Refer the client for psychiatric evaluation, if needed.

· Adjust the client’s treatment plan to include individual rather than group counseling if that seems to be in the client’s best interest.

Helping the Client “Speak”

A counselor noted that, time after time, a client sat quietly in group and spoke only a few words, usually when she was called on. Despite gentle, persistent encouragement from the members of the group and the counselor, the client was quiet and watchful.

After a week, the counselor suggested this reticent client write out whatever she might want to communicate. The client was instructed to take an open-ended approach to the writing, similar to writing in a journal.

The counselor also asked the client to complete the following statements:

• My health concerns are

• The most stress this week came from

• This week I’d rate my stress level as ___, with 1 being low and 10 being high 1.

• The best thing that happened this week was

• I’m working on my treatment goals by

• How I’m feeling about group is

• My most likely relapse trigger is

• I get support for the healthy changes I’m making from

• I participated in the following substance-free activities this week

After several days, the client returned with a sheet containing her thoughts and comments about daily events, her concerns for her children, and the statements completed. The counselor used the information to begin developing a relationship with the client that helped her feel more comfortable in the program and ultimately with the group.

Responding to Intermittent Attendance

It takes time for a group to become a cohesive unit, and clients who do not attend sessions regularly can impede the group process. The client who misses sessions may feel left out of discussions and may jeopardize the development of trust among group members that is at the heart of forthright communication. Counselors may find that such clients are strongly ambivalent about being in treatment, have practical barriers that prevent them from attending regularly, or feel uncomfortable in the group.

Some strategies to assist these clients are to

· Assess their readiness to change, and assign them to a precontemplator or other group whose members are at a similar stage of readiness.

· Identify and address any barriers such as lack of reliable transportation, conflicting work hours, lack of child care, protests by the spouse or significant others to treatment, and fear of violence from a domestic partner.

· Assign these clients to a group whose members share a similar cultural orientation, age range, gender, substance used, or level of psychological functioning.

· Provide refreshments on days when attendance is high to reward desired behavior.

· Monitor attendance and seek guidance from the supervising clinician.

Go to:

Safety and Security

Clients, family members, and staff members must feel comfortable and safe when coming to the IOT program. IOT programs that treat high-risk clients need to monitor these clients carefully, anticipate problems, and plan appropriate interventions. Common safety and security issues that IOT programs face are identified by examples in exhibit 7-1along with the counselor responses.

Exhibit 7-1. Examples of Immediate Safety Concerns and Counselor Responses

Threat of violence against another. While in group, a male client expressed strong feelings of anger toward another man involved with the client’s ex-wife. The client stated that he had a gun and wanted to kill the other man.

Counselor response. The counselor removed the client from the group and engaged him in a discussion about his feelings and remarks. The counselor expressed concern about the client’s well-being and assessed whether he understood the seriousness of his statements. The client’s anger began to subside, and the counselor had him sign a “no violence” contract.

For several days thereafter, the counselor telephoned or spoke in person with the client to assess his feelings and thoughts. The client stated he would “never do anything like that” and had regretted his outburst.

Threat of suicide. A female client telephoned her counselor and said she was tired of struggling with her addictions and other problems and was thinking about killing herself.

Counselor response. The counselor assessed the immediacy of the threat by reviewing the case record to determine whether there had been any previous attempts at suicide and asking the client whether she had a specific plan and the means to carry out the plan. If the counselor were still concerned, he or she would have consulted immediately with the supervisor or program director to develop and document a plan to inform the police, relatives, and the client’s doctor and scheduled an immediate one-on-one session. Because these criteria were not met, the counselor, with the agreement of the client, scheduled an individual therapy session. During the session the counselor and client negotiated a “no suicide” contract that included a commitment by the client to see a psychiatrist for evaluation as soon as possible.

The counselor recorded the incident in the case record and discussed it further with the supervisor.

Presence of Drug Dealers or Gang Members at the Facility

Every IOT program should post prominent signs (in multiple languages where appropriate) inside and outside its facility that prohibit loitering, drug-related activity, or unauthorized persons on the premises. One or more trained staff members promptly and firmly should ask individuals not in treatment or not participating as family members to leave. Police assistance should be requested if there is any resistance to the request or if unauthorized individuals return.

In some cases, a client may encourage the presence of drug dealers or gang members. Criminal justice-mandated clients and individuals who are ambivalent about treatment, for example, may be susceptible to the influence of individuals who use substances and are part of their social networks. If the counselor finds this to be true, the counselor should inform the client that program rules prohibit such activity and explain the consequences of the client’s continued involvement with drug dealers or gang members. A client may need the encouragement of the counselor and the support of program rules and policies to end harmful associations.

Stalking, Domestic Violence, and Threats Against Clients

IOT programs must take appropriate steps to ensure the safety of clients and staff members during treatment. Safety may be threatened by stalkers, violent domestic partners, former spouses and significant others, drug-related associates, or gang members. Counselors should consider following these steps:

· Privately and in a nonjudgmental way, ask the client about restraining orders, threats, or violent incidents that have occurred or that may occur. Knowing about possible problems helps staff members and the client take needed precautions. They can be alert for evidence of any immediate danger and attempt to prevent it. Treatment staff have a duty to warn if the danger is clear and imminent, provided that confidentiality regulations are met (CSAT 2004b ).

· Intervene early to deescalate any situation that potentially could become violent.

· Place violence-related information, such as occurrences of stalking, in the client’s case record. Help the client create a detailed, personal safety plan, and include it in the case record. (See TIP 25, Substance Abuse Treatment and Domestic Violence [CSAT 1997b ], for a sample plan.)

· Require the client to sign a no-contact agreement that prohibits contact with a batterer during the course of treatment, with clearly delineated consequences for violations.

· Assist the client in obtaining a civil protection order that prohibits harassment, contact, communication, or physical proximity by a batterer, stalker, or other threatening individual.

· Connect the client to community services that address domestic violence, such as advocates, counselors, emergency housing, and financial assistance.

Treating Violent Clients

Occasionally, a client may display violent behaviors while in treatment, such as brandishing a weapon or threatening others. IOT staff can take these steps:

· Have all newly admitted clients sign a client code of conduct that states that threats of violence or acts of violence result in immediate termination of treatment and possible criminal prosecution. Give examples.

· Notify a law enforcement agency if a threat to safety exists or an assault or other crime occurs on the program premises; report the incident and client’s name, address, and treatment status, as permitted by Federal regulations.

· If the client is mandated into treatment from the justice system, follow the steps prescribed in the program’s agreement with the justice agency. Certain rule violations, for instance, may require that the IOT provider notify the justice agency. Response to other violations may fall within the discretion of the treatment program. (See TIP 44, Substance Abuse Treatment for Adults in the Criminal Justice System [CSAT 2005d ].)

· Notify supervisors about threats.

Clients Arriving Under the Influence of Drugs or Alcohol

Clients in IOT programs are expected to attend sessions drug and alcohol free. Arriving under the influence interferes with clients’ participation, their ability to recall material covered, and the ability of other group members to benefit from therapy. It also indicates that a client’s substance use disorder is active and that an alternative treatment plan is indicated, at least for that day. Strategies to respond to such occurrences are as follows:

· Develop clear program rules regarding use of drugs during treatment. If a client arrives under the influence, a therapeutic response is called for. The counselor takes the client aside, reviews the rules, and helps the client arrange alternative transportation if the client drove to the program. The client is instructed to return when abstinent and is informed that the substance use will be discussed in the next session. The counselor also can write a note to or call the client to emphasize that the client is expected to return to the group—actions that are intended to normalize the event and reduce any feelings of failure and shame.

· Assess the client’s health status. When a client arrives under the influence of drugs or alcohol, the counselor should assess the client’s need for acute care or detoxification. If it is indicated, the counselor should refer the client to detoxification. In a life-threatening overdose situation, no signed release is required to arrange for emergency medical care. If indicated, emergency personnel can be called. If acute care is refused, the counselor should contact a family member or significant other to escort the client home. (Unless the situation is life threatening, the significant other can be contacted only if the client has signed a release specifying such contact is permitted.) The counselor also should provide the family member with emergency care numbers.

Under the Influence in Group

George arrives at group intoxicated. His speech is slurred, he staggers somewhat, and he laughs loudly and inappropriately.

Counselor response.

• Inserts an educational video, and instructs the group to continue on its own for the next 15 minutes. Alternatively, asks another staff member to sit in temporarily with the group.

• Escorts George from the group.

• Obtains a urine sample and conducts a Breathalyzer™ test to determine the substances consumed.

• Asks George in a one-on-one session how he will return home. Because George drove to the facility, the counselor tells him that he cannot drive home and that the counselor will contact police if George tries to drive. The counselor reviews with George the names of family members who can provide a ride home. The counselor follows applicable Federal, State, and local laws regarding contacts with authorities (CSAT 2004b ).

• Allows George to use the phone to call his wife to pick him up. Note: Some programs pay for a cab.

• Expresses concern about the substance use and encourages George to return to the next session where the episode will be discussed therapeutically.

Key point. The counselor did not engage George in a discussion about his substance use, such as why it occurred and the circumstances. Instead, the counselor immediately focused on confirming George’s substance use, ensuring his safety, encouraging him to return to treatment when sober, and preserving group time for the benefit of the other clients.

Go to:

Client Privacy

Treatment programs often receive inquiries about clients or unsolicited information about clients. Some clients in treatment may be HIV positive but indicate they have not reported their status to their partners or a well-known leader or celebrity may enter the program. Each situation presents client privacy and ethical issues for IOT providers.

Inquiries About Clients

Federal confidentiality regulations do not permit providers to reveal, even indirectly, that someone is a client unless a signed release has been obtained from the client and is on file. IOT staff members must consult a list of client-approved individuals before they (CSAT 2004b )

· Acknowledge that a client is a participant in the program.

· Share any information.

· Transfer a telephone call to the client.

· Take a message for a client.

Unsolicited Information About Clients

Clients’ spouses, domestic partners, or other acquaintances may leave messages with information about clients’ continued substance abuse or other activities and history while they are in treatment. Sometimes these individuals share their identities but do not want them revealed to clients because they fear for their safety. The counselor can respond to unsolicited information by (1) raising the general topic with the client during individual counseling and revising the treatment plan accordingly and (2) increasing the frequency of drug testing if substance use has been reported.

The Informant

Maria calls the IOT counselor to say that her husband Juan (an IOT client) is drinking almost every night and gets really drunk every weekend. She insists that the program “has to do something about it—treatment isn’t working.”

Counselor response. Because Juan has signed a release that permits the counselor to speak with Maria, the counselor asks for her permission to confront Juan with this information. Maria refuses permission because she is afraid Juan will be angry with her. The counselor schedules a session with the couple to discuss problems at home.

The counselor tells Maria that, without her permission, the information will not be conveyed directly; rather, it will be used in the most therapeutic manner possible. That is, the counselor will pay increased attention to Juan’s behavior and communications and will perform breath tests more frequently to obtain evidence of alcohol use.

Key points.

• The counselor avoids being drawn into keeping the wife’s secrets; a couples session is scheduled to discuss openly the relationship and the husband’s drinking.

• IOT staff members must have a written release to discuss Juan’s behavior with anyone.

• Spouses and others who provide information about clients need to be protected from possible harm.

• Information obtained “anonymously” can be therapeutically useful.

• Clients may continue in the program, even though they may be surreptitiously using substances, if all other program criteria are met.

Knowledge of HIV Status Withheld From Partner

Substance abuse, particularly the injection of drugs, increases risk of HIV infection (Pickens et al. 1993). During treatment the IOT counselor may learn that a client has not informed a partner of his or her HIV-positive status, exposing the partner to potential infection. The following approaches help reduce this risk while maintaining client confidentiality:

· Ensure that the client is informed fully about the connections among drug use, unprotected sex, and the transmission of HIV/AIDS.

· Acknowledge and discuss with the client any fears, feelings of embarrassment, and guilt about revealing his or her HIV status to a partner.

· Include information about HIV transmission in educational materials and presentations made to family members.

· Assist the client in finding ways to talk about the issue with the partner, offer assistance in informing the partner if the client consents, and refer the client to an HIV/AIDS counselor for assistance.

· Encourage the client to participate in a support group for HIV-positive individuals, and provide a specific program referral.

· Discuss possible referrals to community-based providers if notifying the partner results in a need for services.

(See TIP 37, Substance Abuse Treatment for Persons With HIV/AIDS [CSAT 2000c].)

Entry of a Well-Known Individual Into Treatment

Recovery from substance use disorders is the focus of treatment for all clients, regardless of their position or visibility in the community. When a well-known person, such as a political leader, sports personality, artist, member of the clergy, or media representative, enters an IOT program, a variety of issues may surface. Examples include

· Increased risk to maintaining privacy and confidentiality. Interest in the client may result in inquiries by media representatives, curious callers, or program visitors. Remind all staff, including administrative and support personnel, as well as clients, to adhere to the program’s confidentiality procedures that protect the privacy of every client.

· Feelings of privilege. Well-known clients may enter treatment with a belief that they do not need to follow all the program’s procedures or meet each requirement. Counselors must assist these clients in assimilating as quickly as possible into the treatment milieu by (1) relating to the private and not the public individual, (2) communicating treatment procedures and requirements, and (3) securing a signed behavioral contract. Individuals who are well known in the community may be concerned about protecting their privacy. The IOT counselor can assist these clients by (1) acknowledging their concerns while assuring them that others in similar circumstances have completed treatment and are recovering successfully, (2) evaluating the feasibility of their being treated out of town, (3) reviewing and discussing the program’s confidentiality regulations and policies, and (4) encouraging clients to attend support group meetings, which have a strong tradition of protecting the identity of participants.

· Effect on the treatment milieu. The presence of high-profile clients or relatives and friends of such clients may mean that the treatment environment is tense or unsettled because of media attention; group cohesion based on trust may be slow to develop. The IOT counselor might consider these approaches: (1) discuss interpersonal issues that a client may have with other clients in individual counseling sessions, (2) use the group process to discuss confidentiality, trust, or other concerns, and (3) place any clients who express a concern about being in a group with a high-profile client in different groups.

· Dual relationships. High-profile clients may offer to help the counselor or program financially, through a personal appearance, or through their influence. Acceptance of such an offer from a client introduces a “dual relationship,” which is unethical. Programs should not accept gifts or favors from clients beyond the published fee schedules. Only after a client has been out of treatment for an extended period (which many programs consider to be 1 year or longer) should the person be considered a successful alumnus and eligible to support the program in these ways.

Go to:

Clients Who Work

Many clients have employment-related challenges, which can include schedule conflicts, associating with co-workers who use substances, and unrealistic employer requests.

Conflicting Work and Treatment Schedules

Individuals who enter IOT may face conflicts between work responsibilities and attending IOT group sessions. Some clients may rotate shifts or be asked to work overtime or work on weekends. Work schedules may interfere with treatment sessions. This situation most likely occurs when the employer is unaware that the employee is in treatment. The following approaches may be helpful, depending on the client’s situation:

· Encourage clients to make treatment and recovery their first priority; help clients understand that by doing so they are better able to meet their work obligations.

· Support clients in making treatment a high priority by being flexible with treatment schedules.

· Encourage clients to inform their employers that they have a health condition and to ask the employers to cooperate with efforts to address the health condition.

Working and Socializing With Co-Workers Who Use Substances

Clients may have used substances with co-workers and may find it difficult to renegotiate their relationships with co-workers and to avoid circumstances that can lead to relapse. Options for addressing these issues include

· Assisting the client in identifying specific work-related circumstances that may be uncomfortable or increase the risk of relapse

· Encouraging the client to distance himself or herself from co-workers who use substances

· Using role plays and other counselor-client interactions so the client can practice responding to questions about treatment and invitations to use substances in ways that preclude uncomfortable discussions and limit risk-oriented situations

· Encouraging the client to transfer to another work environment that is more supportive of recovery, if possible

Employer Requests

If the employer referred the client to treatment, the employer may expect information from the IOT provider about whether the client can assume his or her job responsibilities. Many large employers have policies that address this question, specifying when an employee can resume driving a bus or carrying a gun and mandating regular drug testing for a specified period. Key points concerning this issue include that

· IOT providers do not have the expertise to determine whether a client can perform his or her job duties. Only the employer can determine this.

· IOT providers can inform an employer (with the client’s consent) about the client’s progress in treatment and the drug test results.

· IOT providers can refer the employer to resources such as professional associations and the drug-free workplace information available on the Internet from the Center for Substance Abuse Prevention Workplace Resource Center (workplace.samhsa.gov).

· IOT providers can negotiate with the employer for an additional period of continuing care for the employee; this period reinforces treatment gains and reduces the risk of relapse.

Millions of private-sector workers in the aviation, maritime, railroad, mass transit, pipeline, and motor carrier industries are governed by Federal legislation (the Omnibus Transportation Employee Testing Act of 1991) that makes workplace drug testing mandatory. If an employee of one of these industries fails a workplace drug test and is mandated to treatment, the treatment program is required to inform the employer in writing of assessment results and treatment recommendations (Macdonald and Kaplan 2003).

Helping Clients Achieve Balance

Once in treatment, clients sometimes try to make up for past harmful behavior during periods of substance abuse. Feeling guilty and remorseful, clients may take on additional work, extend their workdays, and try to become perfect employees. IOT providers should caution clients about the risk of compromising their recovery efforts by taking on too much responsibility too quickly. The following responses may assist a client who tries to overcompensate:

· Remind the client that recovery is the first priority.

· Encourage the client to maintain balance and perspective with respect to the type and intensity of activities that are undertaken.

· Assist the client in understanding that there will be time to address past mistakes once recovery is solidly underway.

Conflicting Schedules

Emily decided to seek treatment for her substance use disorder. She was employed at a firm that depended on her to work on key projects. During treatment entry, the IOT counselor learned that Emily’s supervisor sometimes expected her to work beyond regular hours. On these occasions she would be unable to attend IOT group sessions consistently.

Counselor response. After exploring this issue, the counselor concluded that Emily was unable to resolve her schedule conflicts with her employer without jeopardizing her position. The counselor then arranged for Emily to attend a Saturday group session and to increase the number of individual counseling sessions to compensate for the reduced number of group sessions. Emily was able to complete treatment successfully.

Co-Workers Who Use Substances

John and several co-workers went out together every Friday evening after work and drank heavily. They drank on Saturday and continued drinking during the Sunday football games they watched together. After making a decision to stop drinking and enter treatment, John wondered what he could say to his co-workers.

Counselor response. The counselor suggested that John follow these steps:

• Maintain distance from friends and co-workers who use substances.

• Avoid explaining or defending his decision to enter treatment.

• Avoid giving detailed explanations for refusing invitations to activities where substances are used.

• Practice using concrete statements to avoid situations in which substances are used, such as “I need to attend to personal problems in the family”; “Thanks, but no.” Practice these statements in group sessions; role play the responses in individual counseling sessions.

The counselor also worked with John to develop a new social network and find recreational activities that would support his recovery.

Go to:

Boundary Issues

Clients in treatment and IOT program staff members interact with one another on many levels—intellectual, emotional, and spiritual. The IOT experience is intense for all participants. Forming a therapeutic relationship with the client helps the counselor focus on the client’s recovery and influence the client’s behavior. At the same time, clients work together in group sessions over weeks and months on issues of profound significance to them. Furthermore, group members may attend community-based support groups together during and after IOT. In the process, they often develop trust and genuine concern and caring for one another. The intensity and environment of an IOT program can lead to behaviors and issues that challenge the boundaries between staff members and clients. The following are examples of these challenges and suggested responses.

Clients Giving Gifts to Staff

Gift giving is relatively common and may have meanings and consequences that require careful consideration by counselors. For example, the customs and traditions of some cultures encourage gift giving to show respect for someone who offers a valuable service. Recent immigrants from these cultures may continue this practice and bring a small gift or food item to the IOT counselor or other program staff members. In some cases, failure to accept the gift may be viewed as a lack of courtesy and result in the client’s dropping out of treatment.

Other gifts given by clients to IOT staff members may be inappropriate and should be refused politely and tactfully. Most program rules prohibit staff members from accepting gifts if they

· Exceed a certain value (e.g., more than $20)

· Are not the result of a religious or cultural tradition

· Are offered in anticipation of some response or benefit (e.g., special treatment or favor)

· Are obviously personal in nature

· Are likely to cause discomfort, questions, or confusion for others about the relationship between counselor and client

Other programs permit only such gifts as flowers, candy, cookies, or plants that can be shared by all staff members and clients rather than given to an individual staff member.

IOT providers should develop program rules that discourage gift giving and discuss these rules with clients. However, the rules should permit some flexibility for individual circumstances. It is recommended that programs require staff members to report all gifts to supervisory personnel and in the case record. Counselors should be familiar with the program’s policies on these issues.

The Meaning of Gifts: A Cultural Perspective

A gift has meaning both to the individual who gives it and to the one who receives it. Understanding and appropriately acknowledging the true meaning of a gift always require an awareness of the giver’s cultural background.

For example, many cultures place significant value on relationships rather than on individual priorities or achievement. The giving of a gift recognizes and reflects the value of the relationship and signals respect and caring. Gifts are given frequently and generally are not connected to an expectation of favor or privilege. By accepting modest and especially handmade gifts from these clients, IOT staff members acknowledge the respect, cultural values, and practices of these individuals.

Socializing Among Clients

IOT programs differ in the degree of socializing expected outside group sessions. Some programs encourage clients to attend mutual-help meetings together and support one another in other aspects of their lives. Other programs discourage contact between clients except within the program. Most IOTs have rules regarding dating, sexual involvement, or other pairing of clients that could undermine treatment.

Client Relationships Involving Substance Use

Sometimes clients meet in an IOT program and decide to use drugs or alcohol together. Others may be acquainted before entering treatment and continue a relationship that includes substance use. Options for the counselor include the following:

· Reassess the readiness of clients for treatment and recovery.

· Develop a written contract for abstinence, and have clients sign it.

· Refer clients to separate treatment programs.

· Provide individual therapy for one client until the other client graduates from the program.

Socializing Between Staff and Clients

The therapeutic relationship between an IOT counselor and a client is built on caring, trust, and genuine interest in the recovery of the client. These three elements form a basic building block of the treatment alliance. To safeguard the therapeutic dyad and maintain the quality of the treatment environment, IOT programs typically prohibit staff-client activities such as socializing and doing favors. Program consequences for violations of these rules of professional conduct should be clear and applied consistently to all program staff, from administrators to support personnel. Consequences may vary, based on the circumstances, and can include supervisory reprimand and counseling, oral or written warnings, probation, and dismissal. In some cases, the counselor who violates prohibitions must be reported to his or her licensing or certification board.

The Client Is My Neighbor

The IOT counselor recognizes a new client in the waiting room as her neighbor. The neighbor is surprised to see the counselor.

Counselor response. The counselor asks to speak privately to the neighbor in her office. The counselor acknowledges the social relationship that exists between them and states that she will not be involved in any way with the neighbor’s treatment. The counselor also explains confidentiality regulations and indicates that the neighbor is in charge of how they relate to each other outside the treatment setting. The counselor also discloses the relationship to his or her supervisor to ensure that the counselor is not involved, even tangentially, in the client’s case.

Counselor Observes the Client Using Substances in the Community

Residents in a small, rural community occasionally enjoy dancing at the local nightspot. One evening an IOT counselor observes a client drinking at the bar.

Counselor response. The counselor leaves the establishment as soon as possible and does not acknowledge the client. Subsequently, in the treatment setting, the counselor meets with the client one on one. The counselor states the facts of the incident, expresses concern about the possible relapse, reminds the client of the agreement not to use substances, and, using motivational interviewing techniques, asks the client to determine how to handle the return to drinking.

Counselors With Dual Roles

Many IOT counselors are also members of mutual-help programs and must maintain appropriate boundaries between these two roles. For example, it would not be appropriate for an IOT counselor to become a client’s sponsor. A counselor also might meet an IOT program client by chance at a mutual-help meeting, particularly in a small community. Counselors should avoid attending meetings that current or former clients attend. When this is not possible, an IOT counselor should avoid sharing his or her personal issues at that meeting. If a counselor in this situation needs to talk, he or she should take someone aside after the meeting or call his or her sponsor. Some cities have “counselor only” meetings that are not listed in directories. The mutual-help program’s intergroup office or other counselors are good resources for locating such meetings.

Go to:

Appendix 7-A. Instruments for Assessing Relapse Potential

Clinicians have access to several instruments that help clients identify situations that pose high risks of relapse and understand their personal relapse triggers. Most instruments are not under copyright and can be used free of charge. More information about these tools, including information on obtaining copies and links to downloadable versions, can be found at the National Institute on Alcohol Abuse and Alcoholism’s Web site www.niaaa.nih.gov by entering “Alcoholism Treatment Assessment Instruments” into the site’s search engine.

Alcohol Abstinence Self-Efficacy Scale (AASE)

AASE evaluates a client’s confidence in the ability to abstain from drinking in 20 situations that present common drinking cues. The instrument comprises 40 items that gauge a client’s risk of relapse on four scales: when the client is experiencing

· Negative emotions (e.g., depression, frustration)

· Feelings of well-being (e.g., celebrating, on vacation)

· Physical pain (e.g., headache, fatigue)

· Cravings (e.g., testing willpower, experimenting with one drink)

AASE is a paper-and-pencil instrument that can be administered and scored in 20 minutes. No training is required to use it. It can be used to evaluate clients admitted to an IOT program, to guide treatment, or to design individualized relapse prevention strategies. A user-friendly version of AASE can be found at https://pubs.niaaa.nih.gov/publications/assessingalcohol/instrumentpdfs/08_aase.pdf.

Alcohol Effects Questionnaire (AEQ)

AEQ assesses the positive and negative effects that clients expect alcohol to have. Based on their beliefs about alcohol, clients respond “agree” or “disagree” to 40 statements. AEQ yields scores in eight different categories that describe the expected effects of alcohol: general positive feelings, social and physical pleasure, sexual enhancement, power and aggression, social expressiveness, relaxation and tension reduction, cognitive and physical impairment, and unconcern. Administration and scoring of the pencil-and-paper AEQ take 10 minutes, and no special training is required. Although AEQ has been used largely as a research instrument, it can be used therapeutically to assess the effects a client desires to achieve by drinking and to initiate discussions about alternative methods of attaining those effects. The AEQ has proved especially helpful with college students who use alcohol.

Alcohol-Specific Role Play Test (ASRPT)

ASRPT uses role playing to gauge client responses to 10 different situations that pose a threat of relapse. Clients listen to taped prompts and then act out their responses, which are videotaped for scoring purposes. Five of the situations involve clients playing out an interaction with another person (e.g., a scenario in which a business contact asks the person in recovery to complete a deal over drinks at a local bar); five require clients to act out their responses to an internal conflict (e.g., a scenario in which the person in recovery has been working in the yard all day and suddenly thinks that a cold beer sounds good). The ASRPT can be administered in 20 minutes; male and female role-play partners and a videotape technician are necessary. Training is required to give the test, and trained judges must score it.

Situational Confidence Questionnaire (SCQ)

SCQ assesses a client’s confidence in the ability to cope with eight types of high-risk drinking situations. For each of the SCQ’s 39 items, clients indicate on a 6-point scale (ranging from “not at all confident” to “very confident”) how they feel about their ability to resist the urge to drink. SCQ is available in paper-and-pencil and computerized versions and can be self-administered in 8 minutes. (Scoring for the paper-and-pencil version takes 5 minutes; the computerized version is scored as soon as the questionnaire is completed.) Required minimal training is available from a user’s guide that can be purchased with SCQ.

Turn in your highest-quality paper
Get a qualified writer to help you with

“ Evidence-Based Project, Part 4: Critical Appraisal Of Research Assignment ”

Get high-quality paper

NEW! AI matching with writer

Calculate the price

Make an order in advance and get the best price

Type of paper

Academic level

Deadline

Pages (550 words)

$0.00

*Price with a welcome 15% discount applied.

Pro tip: If you want to save more money and pay the lowest price, you need to set a more extended deadline.

We know how difficult it is to be a student these days. That's why our prices are one of the most affordable on the market, and there are no hidden fees.

Instead, we offer bonuses, discounts, and free services to make your experience outstanding.

How it works

Receive a 100% original paper that will pass Turnitin from a top essay writing service

step 1

Upload your instructions

Fill out the order form and provide paper details. You can even attach screenshots or add additional instructions later. If something is not clear or missing, the writer will contact you for clarification.

step 2

Control the process

Once you place an order with our professional essay writing services, we will email you login details to your account. There, you'll communicate with the writer and support team and track the writer's progress.

step 3

Download your paper on time

As soon as your work is ready, we’ll notify you via email. You'll then be able to download it from your account and request a revision if needed. Please note that you can also rate the writer's work in your account.

Pro service tips

How to get the most out of your experience with Scholary Essays

One writer throughout the entire course

If you like the writer, you can hire them again. Just copy & paste their ID on the order form ("Preferred Writer's ID" field). This way, your vocabulary will be uniform, and the writer will be aware of your needs.

The same paper from different writers

You can order essay or any other work from two different writers to choose the best one or give another version to a friend. This can be done through the add-on "Same paper from another writer."

Copy of sources used by the writer

Our college essay writers work with ScienceDirect and other databases. They can send you articles or materials used in PDF or through screenshots. Just tick the "Copy of sources" field on the order form.

Testimonials

See why 20k+ students have chosen us as their sole writing assistance provider

Check out the latest reviews and opinions submitted by real customers worldwide and make an informed decision.

Advanced Technical Writing

Excellent service as always, Thank you!

Customer 452995, January 20th, 2022

Communications

Thank you.

Customer 452723, April 22nd, 2021

Sociology

I have never experienced receiving a paper past the due date and time. That is the only thing that displeases. I don't have time o Overall, your team does a great job.

Customer 452919, November 18th, 2021

English 101

Great work!!

Customer 452989, November 21st, 2021

Management

Excellent work done. Thank you

Customer 452829, August 1st, 2021

Sociology

Thanks to the Team

Customer 452919, March 22nd, 2022

Sociology

Thank you, the journal wa submitted today and I hope to continue receiving services.

Customer 452919, October 28th, 2021

Humanities

Best Serivces

Customer 452995, January 24th, 2022

Psychology

Communication on the small delay was appreciated and final result was worth the wait. Thank you.

Customer 452665, March 18th, 2021

Human Resources Management (HRM)

Thank you for your time and help. I was concerned that I would not make my deadline due to a family emergency, and you guys came through in the clutch. you are appreciated and i will be letting friends and family know of my experience here.

Customer 453045, February 24th, 2022

Natural Sciences

Revision in a short period of time!

Customer 452947, November 9th, 2021

Sociology

Thank you for your service. I hope that everyone had a happy Thanksgiving.

Customer 452919, November 23rd, 2021

11,595

Customer reviews in total

96%

Current satisfaction rate

3 pages

Average paper length

37%

Customers referred by a friend

OUR GIFT TO YOU

15% OFF your first order

Use a coupon FIRST15 and enjoy expert help with any task at the most affordable price.

Claim my 15% OFF Order in Chat