Evidence And Non-Evidence Based Treatment Options 2
Table of Contents
Evidence And Non-Evidence Based Treatment
Prior to beginning work on this discussion, please read both “Limitations to Evidence-Based Practice” and “Rationale and Standards of Evidence-Based Practice,”and listen to the Case Studies in Non-evidence Based Treatment Part One.
For your initial post, you will choose one of the case studies from this week’s audio file selection on which to base your remarks. Based on the available information, evaluate the symptoms and presenting problems for the patient in the chosen case study and propose a provisional diagnosis. Describe one evidence-based treatment for this diagnosis and provide a rationale for your choice. Research at least two peer-reviewed articles to support your evidence-based treatment selection.
Brewer, S., Cervantes, E., & Simpelo, V. (2014). Case studies in non-evidence-based treatment: Part one [Audio file]. College of Health and Human Services, Ashford University: San Diego, CA.
Maier, T. (2012). Limitations to evidence-based practice. In P. Sturmey & M. Hersen (Series Eds.). Handbook of evidence-based practice in clinical psychology: Vol. 2. Adult disorders. Hoboken, N.J.: John Wiley & Sons.
Mudford, O. C., McNeill, R., Walton, L., & Phillips, K. J. (2012). Rationale and standards of evidence-based practice. In P. Sturmey & M. Hersen (Series Eds.), Handbook of evidence-based practice in clinical psychology: Vol. 1. Child and adolescent disorders. Hoboken, N.J.: John Wiley & Sons.
Limitations to Evidence Based Practice
The promotion of evidence-based medicine (EBM) or, more generally, of evidence-based practice (EBP) has strongly characterized most medical disciplines over the past 15 to 20 years. Evidence-based medicine has become a highly influential concept in clinical practice, medical education, research, and health policy. Although the evidence-based approach has also been increasingly applied in related fields such as psychology, education, social work, or economics, it was and still is predominantly used in medicine and nursing. Evidence-based practice is a general and nonspecific concept that aims to improve and specify the way decision makers should make decisions. For this purpose it delineates methods of how professionals should retrieve, summarize, and evaluate the available empirical evidence in order to identify the best possible decision to be taken in a specific situation. So EBP is, in a broader perspective, a method to analyze and evaluate large amounts of statistical and empirical information to understand a particular case. It is therefore not limited to specific areas of science and is potentially applicable in any field of science using statistical and empirical data. Many authors often cite Sackett, Rosenberg, Muir Gray, Haynes, and Richardson’s (1996) article entitled “Evidence-based medicine:What it is and what it isn’t” as the founding deed of evidence-based practice. David L. Sackett (born 1934), an American-born Canadian clinical epidemiologist, was professor at the Department of Clinical Epidemiology and Biostatistics of McMaster University Medical School of Hamilton, Ontario, from 1967 to 1994. During that time, he and his team developed and propagated modern concepts of clinical epidemiology. Sackett later moved to England, and from 1994 to 1999, he headed the National Health Services’ newly founded Centre for Evidence-Based Medicine at Oxford University. During that time, he largely promoted EBM in Europe by publishing articles and textbooks as well as by giving numerous lectures and training courses. David Sackett is seen by many as the founding father of EBM as a proper discipline, although he would not at all claim this position for himself. In fact, Sackett promoted and elaborated concepts that have been described and used by others before; the origins of EBM are rooted back in much earlier times. The foundations of clinical epidemiology were already laid in the 19th century mainly by French, German, and English physicians systematically studying the prevalence and course of diseases and the effects of therapies. As important foundations of the EBMmovement, certainly the works and insights of the Scottish epidemiologist Archibald (Archie) L. Cochrane (1909–1988) have to be c04 18 April 2012; 19:44:27 55 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. mentioned. Cochrane, probably the true founding father of modern clinical epidemiology, had long before insisted on sound epidemiological data, especially from RCTs, as the gold standard to improve medical practice (Cochrane, 1972). In fact, the evaluation of epidemiological data has always been one of the main sources of information in modern academic medicine, and many of the most spectacular advances of medicine are direct consequences of the application of basic epidemiological principles such as hygiene, aseptic surgery, vaccination, antibiotics, and the identification of cardiovasular and carcinogenic risk factors. One of the most frequent objections against the propagation of EBM is, “It’s nothing new, doctors have done it all the time.” Rangachari, for example, apostrophized EBM as “old French wine with a new Canadian label” (Rangachari, 1997, p. 280) alluding to the French 19th century epidemiology pioneer Pierre Louis, who was an influencing medical teacher in Europe and North America, and to David L. Sackett, the Canadian epidemiologist. Even though the “conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients” (Sackett et al., 1996, p. 71) seemsto be a perfectly reasonable and unassailable goal, EBM has been harshly criticized from the very beginning of its promotion (Berk &Miles Leigh, 1999; B. Cooper, 2003; Miles, Bentley, Polychronis, Grey, and Price, 1999; Norman, 1999; Williams & Garner, 2002). In 1995, for example, the editors of The Lancet chose to publish a rebuking editorial against EBM entitled “Evidence-based medicine, in its place” (The Lancet, 1995): The voice of evidence-based medicine has grown over the past 25 years or so from a subversive whisper to a strident insistence that it is improper to practise medicine of any other kind. Revolutionaries notoriously exaggerate their claims; nonetheless, demands to have evidence-based medicine hallowed as the new orthodoxy have sometimes lacked finesse and balance, and risked antagonising doctors who would otherwise have taken many of its principles to heart. The Lancet applauds practice based on the best available evidence–bringing critically appraised news of such advances to the attention of clinicians is part of what peer-reviewed medical journals do–but we deplore attempts to foist evidencebased medicine on the profession as a discipline in itself. (p. 785) This editorial elicited a fervid debate carried on for months in the letter columns of The Lancet. Indeed, there was a certain doggedness on both sides at that time, astonishing neutral observers and rendering the numerous critics even more suspicious. The advocates of EBM on their part acted with great self-confidence and claimed no less than to establish a new discipline and to put clinical medicine on new fundaments; journals, societies, conferences, and EBM training courses sprang up like mushrooms; soon academic lectures and chairs emerged; however, this clamorous and pert appearance of EBM repelled many. A somehow dogmatic, almost sectarian, tendency of the movement was noticed with discontent, and even the deceased patron saint of EBM, Archie Cochrane, had to be invoked in order to push the zealots back: How would Archie Cochrane view the emerging scene? His contributions are impressive, particularly to the development of epidemiology as a medical science, but would he be happy about all the activities linked with his name? He was a freethinking, iconoclastic individual with a healthy cynicism, who would not accept dogma. He brought an open sceptical approach to medical problems and we think that he would be saddened to find that his name now embodies a new rigid medical orthodoxy while the real impact of his many achievments might be overlooked. (Williams & Garner 2002, p. 10) THE DEMOCRATIZATION OF KNOWLEDGE How could such an emotional controversy arise about the introduction of a scientific 56 Overview and Foundational Issues c04 18 April 2012; 19:44:28 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. method (Ghali, Saitz, Sargious, & Hershman, 1999)? Obviously, the propagation and refusal of EBM have to be seen not only from a rational scientific standpoint but also from a sociological perspective (Miettinen, 1999; Norman, 1999): The rise of the EBM movement fundamentally reflects current developments in contemporary health care concerning the allocation of information, knowledge, authority, power, and finance (Berk & Miles Leigh, 1999), a process becoming more and more critical during the late 1980s and the 1990s. Medicine has, for quite some time, been losing its prestige as an intangible, moral institution. Its cost-value ratio is questioned more and more and doctors are no longer infallible authorities. We do not trust doctors anymore to know the solution for any problem; they are supposed to prove and to justify what they do and why they do it. These developments in medicine parallel similar tendencies in other social domains and indicate general changes in Western societies’ self-conception. Today we are living in a knowledge society, where knowledge and information is democratized, available and accessible to all. There is no retreat anymore for secret expert knowledge and for hidden esoteric wisdom. The hallmarks of our time are free encyclopedic databases, open access, the World Wide Web, and Google©. In the age of information, there are no limitations for filing, storage, browsing, and scanning of huge amounts of data; however, this requires more and more expert knowledge to handle it. So, paradoxically, EBM represents a new specialized expertise that aims to democratize or even to abolish detached expert knowledge. The democratization of knowledge increasingly questions the authority and selfsufficiency of medical experts and has deeply unsettled many doctors and medical scientists. Of course, this struggle is not simply about authority and truth; it is also about influence, power, and money. For all the unsettled doctors, EBM must have appeared like a guide for the perplexed leading them out of insecurity and doubt. Owing to its paradoxical nature, EBM offers them a new spiritual home of secluded expertise allowing doctors to regain control over the debate and to reclaim authority of interpretation from bold laymen. For this purpose, EBM features and emphasizes the most valuable label of our time that is so believable in science: science- or evidencebased. In many areas of contention, terms like evidence-based or scientifically proven are used for the purpose of putting opponents on the defensive. Nobody is entitled to question a fact, which is declared evidence-based or scientifically proven. By definition, these labels are supposed to convey unquestioned and axiomatic truth. It requires rather complex and elaborate epistemological reasoning to demonstrate how even true evidence-based findings can at the same time be wrong, misleading, and/or useless. All these accounts and arguments apply in particular to the disciplines of psychiatry and clinical psychology, which have always had a marginal position among the apparently respectable disciplines of academic medicine. Psychiatrists and psychologists always felt particularly pressured to justify their actions and are constantly suspected to practice quackery rather than rational science. It is therefore not surprising that among other marginalized professionals, such as the general practitioners, psychiatrists and psychotherapists made particularly great efforts over the last years to establish their disciplines as serious matters of scholarly medicine by diligently adopting the methods of EBM (Geddes & Harrison, 1997; Gray & Pinson, 2003; OakleyBrowne, 2001; Sharpe, Gill, Strain, & Mayou, 1996). Yet, there are also specific problems limiting the applicability of EBP in these disciplines.
EMPIRICISM AND REDUCTIONISM In order to understand the role and function of EBP within the scientific context, it may be Limitations to Evidence-Based Practice 57 c04 18 April 2012; 19:44:28 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. helpful to give a brief overview of the theoretical backgrounds of science in general. What is science and how does it proceed? Science can be seen as a potentially endless human endeavour that aims to understand and determine reality. Not only are physical objects matters of science, but also immaterial phenomena like language, history, society, politics, economics, human behavior, thoughts, or emotions. Starting with the Greek scientists in the ancient world, but progressing more rapidly with the philosophers of the Enlightenment, modern science adopted defined rules of action and standards of reasoning that delineate science from nonscientific knowledge such as pragmatics, art, or religion. Unfortunately, notions like science, scientific, or evidence are often wrongly used in basically nonscientific contexts causing unnecessary confusion. The heart and the starting point of any positive science is empiricism, meaning the systematic observation of phenomena. Scientists of any kind must start their reasoning with observations, possibly refined through supportive devices or experimental arrangements. Although positive science fundamentally believes in the possibility of objective perception, it also knows the inherent weaknesses of reliability and potential sources of errors. Rather than have confidence in single observations, science trusts repeated and numerous observations and statistical data. This approach rules out idiosyncratic particularities of single cases to gain the benefit of identifying the common characteristics of general phenomena (i.e., reductionism). This approach of comprehending phenomena by analytically observing and describing them has in fact produced enormous advancements in many fields of science, especially in technical disciplines; however, contrasting and confusing gaps of knowledge prevail in other areas such as causes of human behavior, mind–body problems, or genome–environment interaction. Some areas of science are apparently happier and more successful using the classical approach of positive science, while other disciplines feel less comfortable with the reductionist way of analyzing problems. The less successful areas of science are those studying complex phenomena where idiosyncratic features of single cases can make a difference, in spite of perfect empirical evidence. This applies clearly to medicine, but even more to psychology, sociology, or economics. Medicine, at least in its academic version, usually places itself among respectable sciences, meeting with and observing rules of scientific reasoning; however, this claim may be wishful thinking and medicine is in fact a classical example of a basically atheoretical, mainly pragmatic undertaking pretending to be based on sound science. Inevitably, it leads to contradictions when trying to bring together common medical practice and pure science. COMPLEXITY Maybe the deeper reasons for these contradictions are not understood well enough. Maybe they still give reason for unrealistic ideas to some scientists. A major source of misconception appears to be the confused ontological perception of some objects of scientific investigation. What is a disease, a disorder, a diagnosis? What is human behavior? What are emotions? Answering these questions in a manner to provide a basis for scientific reasoning in a Popperian sense (see later) is far from trivial. Complex objects of science, like human behavior, medical diseases, or emotions, are in fact not concrete, tangible things easily accessible to experimental investigation. They are emergent phenomena, hence they are not stable material objects, but exist only as transitory, nonlocal appearances fluctuating in time. They continuously emerge out of indeterminable complexity through repeated self-referencing operations in complex systems (i.e., autopoietic systems). Indeterminable complexity or deterministic chaos means that a huge number of mutually interacting parameters autopoietically 58 Overview and Foundational Issues c04 18 April 2012; 19:44:28 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. form a system, rendering any precise calculation of the system’s future conditions impossible. Each single element of the system perfectly follows the physical rules of causality; however, the system as a whole is nevertheless unpredictable. Its fluctuations and oscillations can be described only probabilistically. In order to obtain reasonable and useful information about a system, many scientific disciplines have elaborated probabilistic methods of approaching their objects of interest. Thermodynamics, meteorology, electroencephalography, epidemiology, and macroeconomics are only a few such examples. Most structures in biological, social, and psychological reality can be conceived as emergent phenomena in this sense. Just as the temperature of an object is not a quality of the single molecules forming the object—a single molecule has no temperature— but a statistic description of a huge number of molecules, human behavior cannot be determined through the description of composing elements producing the phenomenon—for example, neurons—even if these elements are necessary and indispensable preconditions for the emergence of the phenomenon. The characteristics of the whole cannot be determined by the description of its parts. When the precise conditions of complex systems turn out to be incalculable, the traditional reaction of positive science is to intensify analytical efforts and to compile more information about the components forming the system. This approach allows scientists to constantly increase their knowledge about the system in question without ever reaching a final understanding and a complete determination ofthe function ofthe system. This is exactly what happens currently in neurosciences. Reductionist approaches have their inherent limitations when it comes to the understanding of complex systems. A similar problem linked to complexity that is particularly important is the assumed comparability of similar cases. In order to understand an individual situation, science routinely compares defined situations to similar situations or, even better, to a large number of similar situations. Through the pooling of large numbers of comparable cases, interfering individual differences are statistically eliminated, and only the common ground appears. The conceptual assumption behind this procedure is that similar—but still not identical— cases will evolve similarly under identical conditions. One of the most important insights from the study of complex phenomena is that in complex systems very small differences in initial conditions may lead to completely different outcomes after a short time—the socalled butterfly effect. This insight is well known to natural scientists; however, clinical epidemiologists do not seem to be completely aware of the consequences of the butterfly effect to their area of research. FROM KARL POPPER TO THOMAS S. KUHN Based on epistemological considerations, the Anglo-Austrian philosopher Karl Popper (1902–1994) demonstrated in the 1930s the limitations of logical empiricism. He reasoned that general theories drawn from empirical observations can never be proven to be true. So, all theories must remain tentative knowledge, waiting to be falsified by contrary observations. In fact, Popper conceived the project of science as a succession of theories to be falsified sooner or later and to be replaced by new theories. This continuous succession of new scientific theories is the result of natural selection of ideas through the advancement of science. According to Popper, any scientific theory must be formulated in a way to render it potentially falsifiable through empirical testing. Otherwise, the theory is not scientific: It may be metaphysical, religious, or spiritual instead. This requires that a theory must be formulated in terms of clearly defined notions and measurable elements. Popper’s assertions were later qualified as being less absolute by the American philosopher of science Thomas S. Kuhn (1922–1996). Limitations to Evidence-Based Practice 59 c04 18 April 2012; 19:44:28 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. Kuhn, originally a physicist, pointed out that in real science any propagated theory could be falsified immediately by contrary observations because contradicting observations are always present; however, science usually ignores or even suppresses observations dissenting with the prevailing theory in order to maintain the accepted theory. Kuhn calls the dissenting observations anomalies, which are—according to him—always obvious and visible to all, but nevertheless blinded out of perception in order to maintain the ruling paradigm. In Kuhn’s view, science will never come to an end and there will never be a final understanding of nature. No theory will ever be able to integrate and explain consistently all the observations drawn from nature. At this point, even the fundamental limitations to logical scientific reasoning demonstrated by Go¨del’s incompleteness theorems become recognizable (cf. also Sleigh, 1995). Based on his considerations, Kuhn clear-sightedly identified science to be a social system, rather than a strictly logical and rational undertaking. Science, as a social phenomenon, functions according to principles of Gestalt psychology. It sees the things it wants to see and overlooks the things that do not fit. In his chief work The Structure of Scientific Revolutions, Kuhn (1962) gives several examples from the history of science supporting this interpretation. It is in fact amazing to see how difficult it was for most important scientific breakthroughs to become acknowledged by the contemporary academic establishment. Kuhn uses the notion normal science to characterize the established academic science and emphasizes the self-referencing nature of its operating mode. Academic teachers teach students what the teachers believe is true. Students have to learn what they are taught by their teachers if they want to pass their exams and get their degrees. Research is mainly repeating and retesting what is already known and accepted. Journals, edited and peerreviewed by academic teachers, publish what conforms with academic teachers’ ideas. Societies and associations—headed by the same academic teachers—ensure the purity of doctrine by sponsoring those who confirm the prevailing paradigms. Dissenting opinions are unwelcome. Based on Kuhn’s view of normal science, EBP and EBM can be identified as classical manifestations of normal science. The EBP helps to ensure the implementation of mainstream knowledge by declaring to be most valid what is best evaluated. Usually the currently established practices are endorsed by the best and most complete empirical evidence; dissenting ideas will hardly be supported by good evidence, even if these ideas are right. Since EBP instructs its adherers to evaluate the available evidence on the basis of numerical rules of epidemiology, arguments like plausibility, logic consistency, or novelty are of little relevance. AN EXAMPLE FROM RECENT HISTORY OF CLINICAL MEDICINE When in 1982 the Australian physicians Barry Marshall and Robin Warren discovered Helicobacter pylori in the stomachs of patients with peptic ulcers, their findings were completely ignored and neglected by the medical establishment of that time. The idea that peptic ulcers are provoked by an infectious agent conflicted with the prevailing paradigm of academic gastroenterology, which conceptualized peptic ulcers as a consequence of stress and lifestyle. Although there had been numerous previous reports of helicobacteria in gastric mucosa, all these findings were completely ignored because they conflicted with the prevailing paradigm. As a consequence Marshall and Warren’s discovery was ignored for years because it fundamentally challenged current scientific opinion. They were outcast by the scientific community, and only 10 years later their ideas slowly started to convince more and more clinicians. Now, 25 years later, it is common basic clinical knowledge that 60 Overview and Foundational Issues c04 18 April 2012; 19:44:28 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. Helicobacter pylori is one of the major causes of peptic ulcers, and eradication therapy is the accepted and rational therapy for gastric ulcers. Finally, in 2005 Barry Marshall and Robin Warren gained the Nobel Price for their discovery (Parsonnet, 2005). BENEFITS AND RISKS OF EVIDENCE-BASED PRACTICE The true benefits of EBP for patients and society in terms of outcomes and costs have not been proven yet—at least not through sound empirical evidence (B. Cooper, 2003; Geddes & Harrison, 1997). Nevertheless, there is no doubt that the method has a beneficial and useful potential. Many achievements of EBP are undisputable and undisputed, hence they are evident. Owing to the spread of methodical skills in retrieving and evaluating the available epidemiological evidence, it has become much harder to apply any kind of obscure or idiosyncratic practices. The experts’ community, as well as the customers and the general public, are much more critical toward pretended effects of treatments and ask for sound empirical evidence of effectiveness and safety. It is increasingly important not only to know the best available treatment, but also to prove it. The EBP is therefore a helpful instrument for doctors and therapists to justify and legitimate their practices to insurance, judiciary, politics, and society. Furthermore, individual patients might be less at risk to wrong or harmful treatment due to scientific misapprehension. Of course, common malpractice owing to inanity, negligence, or viciousness will never be eliminated, not even by the total implementation of EBP; however, treatment errors committed by diligent and virtuous doctors are minimized through careful adherence to rational guidelines. In general, clinical decision-making paths have become more comprehensible and rational, probably also due to the spread of EBP. As medicine is in fact not a thoroughly scientific matter (Ghali et al. 1999), continuous efforts are needed to enhance and renew rationality. The EBP contributes to this task and helps clinicians to maintain rationality in a job where inscrutable complexity is daily business. In current medical education, the algorithms of EBP are now instilled into students as a matter of course. Seen from that perspective, EBP is also an instrument of discipline and education, for it compels medical students and doctors to reflect continuously all their opinions and decisions scientifically (Norman, 1999). Today EBP has a great impact on the education and training of future doctors, and it thereby enhances the uniformity and transparency of medical doctrine. This international alignment of medical education with the principles of EBP will, in the long run, allow for better comparability of medical practice all over the world. This is an important precondition for the planning and coordination of research activities. Thus, the circle of normal science is perfectly closed through the widespread implementation of EBP. GENERAL LIMITATIONS TO EVIDENCE-BASED PRACTICE It has been remarked, not without reason, that the EBP movement itself has adopted features of dogmatic authority (B. Cooper, 2003; Geddes et al., 1996; Miles et al., 1999). This appears particularly ironic, because EBP explicitly aims to fight any kind of orthodox doctrine. The ferocity of some EBP adherents may not necessarily hint at conceptual weaknesses of the method; rather, it is more likely a sign of an iconoclastic or even patricidal tendency inherent to EBP. Young, diligent scholars, even students, possibly without any practical experience, are now entitled to criticize and rectify clinical authorities (Norman, 1999). This kind of insurgence must evoke resistance from authorities. If the acceptance of EBP among clinicians should be enhanced, Limitations to Evidence-Based Practice 61 c04 18 April 2012; 19:44:28 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. it is advisable that the method is not only propagated by diligent theoreticians, but mainly by experienced practitioners. One of the first and most important arguments against EBP is reductionism (see earlier, Welsby, 1999). Complex and maybe fundamentally diverse clinical situations of individual patients have to be condensed and aggregated to generalized questions in order to retrieve empirical statistical evidence. Important specific information about the individual cases is inevitably lost owing to this generalization. The usefulness of the retrieved evidence is therefore inevitably diluted to a very general and dim level. Of course, there are some frequently used standard interventions, which are really based upon good empirical evidence (Geddes et al., 1996). EXAMPLES FROM CLINICAL MEDICINE Scabies, a parasitic infection of the skin, is an important public health problem, mainly in resource-poor countries. For the treatment of the disease, two treatment options are recommended: topical permethrin and oral ivermectin. Both treatments are known to be effective and are usually well tolerated. The Cochrane Review concluded from the available empirical evidence that topical permethrin appears to be the most effective treatment of scabies (Strong & Johnstone, 2007). This recommendation can be found in up-to-date medical textbooks and is familiar to any well-trained doctor. Acute otitis media in children is one of the most common diseases, one ofthe main causes for parents to consult a pediatrician, and a frequent motive for the prescription of antibiotics, even though spontaneous recovery is the usual outcome. Systematic reviews have shown that the role of antibiotic drugs for the course of the disease is marginal, and there is no consensus among experts about the identification of subgroups who would potentially profit from antibiotics. In clinical practice, in spite of lacking evidence of its benefit, the frequent prescription of antibiotic drugs is mainly the consequence of parents’ pressure and doctors’ insecurity. A recent meta-analysis (Rovers et al., 2006) found that children youngerthan 2 years of age with bilateral acute otitis media and those with otorrhea benefited to some extent from antibiotic treatment; however, even for these two particular conditions, differences were moderate: After 3–7 days, 30% of the children treated with antibiotics still had pain, fever, or both, while in the control group the corresponding proportion was 55%. So, the available evidence to guide a clinician when treating a child with acute otitis media is not really significant and the decision will mostly depend on soft factors like parents’ preferences or practical and economical considerations. Evidently, clinicians choosing these interventions do not really need to apply the algorithms of EBP to make their decisions. They simply administer what they had learned in their regular clinical training. The opponents of EBP rightly argue that the real problems in clinical practice arise from complex, multimorbid patients presenting with several illnesses and other factors that have to be taken into account by the treating clinician. In order to manage such cases successfully there is usually no specific statistical evidence available to rely on. Instead, clinicians have to put together evidence covering some aspects of the actual case and hope that the resulting treatment will still work even if it is not really designed and tested for that particular situation. Good statistical evidence meeting the highest standards of EBP is almost exclusively derived from ideal monomorbid patients, who are rarely seen in real, everyday practice (Williams & Garner, 2002). It is not clear at all—and far from evidence-based—whether evidence from ideal cases can be transferred to 62 Overview and Foundational Issues c04 18 April 2012; 19:44:28 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. more complex cases without substantial loss of validity. Another argument criticizing EBP points at an epistemological problem. Because the EBP operates retrospectively by evaluating what was done in the past, it cannot directly contribute to developing new strategies and to finding new therapies. The EBP helps to consolidate well-known therapies, but cannot guide researchers toward scientific innovations. No scientific breakthrough will ever be made owing to EBP. On the contrary, if all clinicians strictly followed recommendations drawn from available retrospective evidence and never dared to try something different, science would stagnate in fruitless selfreference. There is a basically conservative and backward tendency inherent to the method. Although it cannot exactly be called antiscientific on that account (B. Cooper, 2003; Miles et al., 1999), EBP is a classical phenomenon of normal science (Kuhn, 1962). It will not itself be the source of fundamental new insights. Finally, there is an external problem with EBP, which is probably most disturbing of all: Production and compilation of evidence available to clinicians is highly critical and exposed to different nonscientific influences (Miettinen, 1999). Selection of areas of research is based more and more on economic interests. Large, sound, and therefore scientifically significant epidemiologic studies are extremely complex and expensive. They can be accomplished only with the support of financially potent sponsors. Compared with public bodies or institutions, private companies are usually faster and more flexible in investing important amounts of money into medical research. So, for many ambitious scientists keen on collecting publishable findings, it is highly appealing to collaborate with commercial sponsors. This has a significant influence on the selection of diseases and treatments being evaluated. The resulting body of evidence is necessarily highly unbalanced because mainly diseases and interventions promising important profits are well evaluated. For this reason, more money is probably put into trials on erectile dysfunction, baldness, or dysmenorrhea than on malaria or on typhoid fever. So, even guidelines based on empirical evidence—considered to be the ultimate gold standard of clinical medicine—turn out to be arbitrary and susceptible to economical, political, and dogmatic arguments (Berk & Miles Leigh, 1999). So, EBP’s goals to replace opinion and tendency by knowledge are in danger of being missed, if the relativity of available evidence is unrecognized. The uncritical promotion of EBP opens a clandestine gateway to those who have interests in controlling the contents of medical debates and have the financial means to do so. Biasing clinical decisions in times of EBP is probably no longer possible by false or absent evidence; however, the selection of what is researched in an EBP-compatible manner and what is published may result in biased clinical decisions (Miettinen, 1999). One of the most effective treatment options in many clinical situations—watchful waiting—is notoriously under-researched because there is no commercial or academic interest linked to that treatment option. Unfortunately, there will never be enough time, money, and workforce to produce perfect statistical evidence for all useful clinical procedures. So, even in the very distant future, clinicians will still apply many of their probably effective interventions without having evidence about their efficacy and effectiveness; thus, EBP is a technique of significant but limited utility (Green & Britten, 1998; The Lancet, 1995; Sackett et al., 1996). EXAMPLE FROM CLINICAL MEDICINE Lumbar back pain is one of the most frequent health problems in Western countries. About 5% of all low back problems are caused by prolapsed lumbar discs. The treatment is Limitations to Evidence-Based Practice 63 c04 18 April 2012; 19:44:28 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. mainly nonsurgical and 90% of acute attacks of nerve root pain (sciatica) settle without surgical intervention; however, different forms of surgical treatments have been developed and disseminated. Usually these methods are considered for more rapid relief in patients whose recovery is unacceptably slow. The Cochrane reviewers criticize that “despite the critical importance of knowing whether surgery is beneficial for disc prolapse, only four trials have directly compared discectomy with conservative management and these give suggestive rather than conclusive results” (Gibson & Waddell, 2007, p. 1). They concluded: Surgical discectomy for carefully selected patients with sciatica due to lumbar disc prolapse provides faster relief from the acute attack than conservative management, although any positive or negative effects on the lifetime natural history of the underlying disc disease are still unclear. (p. 2) Surgical treatments of low back pain hold an enormous commercial potential due to the worldwide frequency of the problem. It appears obvious that there are only a few trials comparing conservative treatment with surgery. SPECIFIC LIMITATIONS TO EBP IN PSYCHIATRY, PSYCHOTHERAPY, AND CLINICAL PSYCHOLOGY In psychiatry and psychotherapy, there is an ambivalent attitude toward EBP. Attempting to increase their scientific respectability, some psychiatrists and clinical psychologists zealously adopted EBP algorithms (Geddes & Harrison, 1997; Gray & Pinson, 2003; OakleyBrowne, 2001; Sharpe et al., 1996) and started evidence-based psychiatry. Others remain hesitant or doubtful about the usefulness of EBP in their field, and several authors have addressed different critical aspects of evidence-based psychiatry (Berk & Miles Leigh, 1999; Bilsker, 1996; Brendel, 2003; Geddes & Harrison, 1997; Goldner & Bilsker, 1995; Harari, 2001; Hotopf, Churchill, & Lewis, 1999; Lawrie, Scott, & Sharpe, 2000; Seeman, 2001; Welsby, 1999; Williams & Garner, 2002) with all of them fundamentally concerning practical and scientific particularities of psychiatry and clinical psychology. Next, we shall try to clarify these arguments. The evidence-based approach to individual cases is critically dependent on the validity of diagnoses. This is an axiomatic assumption of EBP, which is rarely analysed or scrutinized in detail. If in a concrete case no diagnosis could be attributed, the case would not be amenable to EBP, and no evidence could support decisions in such a case. If the diagnosis is wrong, or—even more intricate—if cases labeled with a specific diagnosis are still not homogenous enough to be comparable in relevant aspects, EBP will provide useless results. EXAMPLE FROM PSYCHIATRY According to DSM-IV, eating disorders are classified in different categories: anorexia nervosa (AN), bulimia nervosa (BN), binge eating disorder (BED), and eating disorder not otherwise specified (EDNOS). These categories are clinically quite distinct and diagnostic criteria are clear and easily applicable. In spite of the phenomenological diversity of the disease patterns, there is a close relationship between the different forms of eating disorders. In clinical practice, switches between different diagnoses and temporary remissions and relapses are frequent. In the course of time, patients may change their disease pattern several times: At times they may not meet the criteria for a diagnosis anymore, although they are not completely symptom free, and later they may relapse to a full-blown eating disorder again or may be classified as having EDNOS. 64 Overview and Foundational Issues c04 18 April 2012; 19:44:28 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. Corresponding to these clinical impressions, longitudinal studies demonstrate that the stability of eating disorder diagnoses over time is low ( Fichter & Quadflieg, 2007; Grilo et al., 2007; Milos, Spindler, Schnyder, & Fairburn, 2005). Based on systematic evaluation of the available evidence, however, treatment guidelines give specific recommendations for the different conditions (National Institute for Clinical Excellence [NICE], 2004). For patients with AN, psychological treatment on an outpatient basis is recommended. The treatment should be offered by “a service that is competent in giving that treatment and in assessing the physical risk of people with eating disorders” (p. 60). For patients with BN, the NICE guideline proposes as a possible first step to follow an evidence-based self-help program. As an alternative, a trial with an antidepressant drug is recommended, followed by cognitive behavior therapy for bulimia nervosa. In the absence of evidence to guide the treatment of EDNOS, the NICE guideline recommends pragmatically that “the clinician considers following the guidance on the treatment of the eating problem that most closely resembles the individual patient’s eating disorder” (p. 60). So even though specific diagnoses of eating disorders are not stable and a patient with AN might be diagnosed with BN a few months later, treatment recommendations vary considerably for the two conditions. It becomes obvious that different treatment recommendations for seemingly different conditions reflect rather accidental differences in the availability of empirical evidence than real differences in the response of certain conditions to specific treatments. Hence, the guidance offered by the guideline is basically a rather unstable crutch, and of course, cognitive behavior therapy or an evidence-based self-help program might be just as beneficial in AN or in EDNOS than it is in BN, even though nobody has yet compiled the statistical evidence to prove this. What does the validity of a diagnosis mean? The question concerns epistemological issues and requires a closer look to the nature of medical diagnoses with special regard to psychiatric diagnoses. R. Cooper (2004) questioned if mental disorders as defined in diagnostic manuals are natural kinds. In her thoughtful paper, the author concluded that diagnostic entities are in fact theoretical conceptions, describing complex cognitive, behavioral, and emotional processes (R. Cooper, 2004; Harari, 2001). Diagnostic categories are based upon observations, still they are strongly influenced by theoretical, social, and even economical factors. The ontological structure of psychiatric diagnoses is therefore not one of natural kinds. They are not something absolutely existing that can be observed independently. Rather they are comprehensive theoretical definitions serving as tools for communication and scientific observation. Kendell and Jablensky (2003) have also recently addressed the issue of diagnostic entities and concluded that the validity of psychiatric diagnoses is limited. They analysed whether diagnostic entities are sufficiently separable from each other and from normality by zones of rarity. They concluded that this was not the case; rather, they concluded that psychiatric diagnoses often overlap (R. Cooper, 2004; Welsby, 1999), shift over time within the same patient, and several similar diagnoses can be present in the same patient at the same time (comorbidity). Not surprisingly, diagnosis alone is a poor predictor of outcome (Williams & Garner, 2002). Acknowledging this haziness of diagnoses, one realizes these problems when trying to match individual cases to empirical evidence. When even the presence of a correctly assessed diagnosis does not assure comparability to other cases with the same diagnosis, empirical evidence about mental disorders is highly questionable (Harari, 2001). Of course, limited validity does not imply complete absence of validity, and empirical evidence on mental disorders is still useful to some extent; however, insight Limitations to Evidence-Based Practice 65 c04 18 April 2012; 19:44:29 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. into the limitations is important and that insight points out that psychiatric diagnoses represent phenomenological descriptions rather than natural kinds. Several authors have treated the same issue when writing about the complexity of cases, the problem of subsyndromal cases, and of single cases versus statistical evidence (Harari, 2001; Welsby, 1999; Williams & Garner, 2002). NONLINEAR DYNAMICS IN THE COURSE OF DISEASES It might be fruitful to look at evidence-based psychiatry from another perspective and to address the issues of complexity and nonlinear dynamics. With regard to their physical and mental functioning, humans can be conceptualized as systems of high complexity (Luhmann, 1995). This means that they cannot be determined precisely, but only in a probabilistic manner; however, probabilistic determination is sufficient for most purposes in observable reality. Human life consists fundamentally in dealing with probabilities. Social systems and human communication are naturally designed to manage complexity more or less successfully. Medicine itself is a social system (Luhmann, 1995) trying to handle the effects of complexity (Harari, 2001), for example, by providing probabilistic algorithms for treatments of diseases. In most situations, medicine can ignore the particular effects emerging from the complex nonlinear structure of its objects, although such effects are always present. Only sometimes do these effects become obvious and irritating, as for example in fluctuations of symptoms in chronic diseases, variations in response to treatment, unexpected courses in chronic diseases, and so on. Such phenomena can be seen as manifestations of the butterfly effect (see earlier). This insight questions deeply the core principle of EBP that assumes that it is rational to treat similar cases in the same manner because similarity in the initial conditions will predict similar outcomes under the identical treatment. The uncertainty of this assumption is particularly critical in psychiatry and psychotherapy. In these fields similar appearance is just a palliation for untraceable difference, and this exact difference may crucially influence the outcome. Addressing such problems is daily business for psychiatrists and psychotherapists, so their disciplines have developed special approaches. Diagnostic and therapeutic procedures in these disciplines are much less focused on critical momentary decisions, but more on gradual, iterative procedures. Psychiatric treatments and even more psychotherapy are self-referencing processes, where assessments and decisions are constantly reevaluated. Instead, EBP focuses primarily on decision making as the crucial moment of good medical practice. One gets the impression that EBM clinicians are constantly making critical decisions, and after having made the right decision, the case is solved. Maybe it is because of this misfit between the proposals of the method and real daily practice that many psychiatrists are not too attracted by EBP. EXAMPLE FROM PSYCHIATRY The diagnosis of posttraumatic stress disorder (PTSD) was first introduced in the third edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-III) in 1980. Before that time, traumatized individuals were either diagnosed with different nonspecific diagnoses (e.g., anxiety disorders, depression, neurasthenia) or not declared ill at all. Astonishingly, the newly discovered entity appeared to be a clinically distinct disorder and the corresponding symptoms (re-experiencing, avoidance, hyperarousal) were quite characteristic and easily identifiable. Within a short time after its invention (Summerfield, 2001), PTSD became a very popular disorder; 66 Overview and Foundational Issues c04 18 April 2012; 19:44:29 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. clinicians and even patients loved the new diagnosis (Andreasen, 1995). The key point for the success of the new diagnosis was that it is explicitly based on the assumption of an external etiology; that is, the traumatic experience. This conception makes PTSD so appealing for the attribution of cause, responsibility, and guilt is neatly separated from the affected individual. PTSD allows for the exculpation of the victim, a feature that was particularly important when caring for Holocaust survivors and Vietnam War veterans. But what was almost proscribed for some time after the introduction of PTSD is now evidence-based: Preexisting individual factors play an important role in the shaping of posttraumatic response. Whether or not an individual develops PTSD after a traumatic experience is not only determined by the nature and the intensity of the traumatic impact, but also by various pretraumatic characteristics of the affected individual. Furthermore, PTSD is not the only posttraumatic mental disorder. A whole spectrum of mental disorders is closely linked to traumatic experiences, although they lack the monocausal appearance of PTSD. Anyway, the most frequent outcome after traumatic experiences is recovery. In the second rank of frequency comes major depression. Borderline personality disorder is fully recognized now as a disorder provoked by traumatic experiences in early childhood. Dissociative disorders, chronic somatoform pain, anxiety disorders, substance abuse, and eating disorders are equally related to traumatic experiences. Not surprisingly, PTSD is often occurring as a comorbid condition with one or more additional disorder or vice versa. In clinical practice, traumatized patients usually present more complex than expected. This may explain to some extent why PTSD was virtually overlooked by clinicians for many decades before its introduction, a fact that is sometimes hard to understand by younger therapists who are so familiar with the PTSD diagnosis. At any rate, the high-functioning, intelligent, monomorbid PTSD patient is indeed best evaluated in clinical trials, but rarely seen in everyday practice. PTSD was right in the focus of research since its introduction. Also from a scientific point of view, the disorder is appealing because it is provoked by an external event. PTSD allows ideally for the investigation of the human-environmentinteraction, whichis a crucial issue for psychiatry and psychology in general. The number of trials on diagnosis and treatment of PTSD is huge, and the disorder is now probably the best evaluated mental disorder. What is the benefit of the accumulated large body of evidence on PTSD for clinicians? There are several soundly elaborated guidelines on the treatment of PTSD (American Psychiatric Association, 2004; Australian Centre for PosttraumaticMental Health, 2007; NICE, 2005), meta-analyses, and Cochrane Reviews providing guidance for the assessment and treatment of the disorder. When we look at the existing conclusions and recommendations, we learn that: Debriefing is not recommended as routine practice for individuals who have experienced a traumatic event. When symptoms are mild and have been present for less than 4 weeks after the trauma, watchful waiting should be considered. Trauma-focused cognitive behavior therapy on an individual outpatient basis should be offered to people with severe posttraumatic symptoms. Eye movement desensitization and reprocessing is an alternative treatment option. Drug treatment should not be used as a routine first-line treatment in preference to a trauma-focused psychological therapy. Drug treatment (Specific Serotonin Reuptake Inhibitors) should be considered Limitations to Evidence-Based Practice 67 c04 18 April 2012; 19:44:29 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology : Adult Disorders, John Wiley & Sons, Incorporated, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817356. Created from ashford-ebooks on 2017-11-07 11:26:43. Copyright © 2011. John Wiley & Sons, Incorporated. All rights reserved. for the treatment of PTSD in adults who express a preference not to engage in trauma-focused psychological treatment. In the context of comorbid PTSD and depression, PTSD should be treated first. In the context of comorbid PTSD and substance abuse, both conditions should be treated simultaneously. These recommendations are obviously clear, useful, and practical. They give real guidance to therapists and do not leave much room for doubts or insecurity. On the other hand, they are basically very simple, almost trivial. For trauma therapists, these recommendations are commonplace and serve mainly to endorse what they are practicing anyway. The main points of the guidelines for the treatment of PTSD could be taught in a 1-hour workshop. The key messages of the guidelines represent basic clinical knowledge on a specific disorder as it has been instructed in times before EBP. Through their standardizing impact on the therapeutic community, guidelines may in fact align and improve the general service quality offered to traumatized individuals, although this effect has not yet been demonstrated by empirical evidence. The treatment of an individual patient remains a unique endeavor where interpersonal relationship, flexibility, openness, and cleverness are crucial factors. This challenge is not lessened by evidence or guidelines.
Rationale and Standards of Evidence in Evidence-Based Practice
OLIVER C. MUDFORD, ROB MCNEILL, LISA WALTON
AND KATRINA J. PHILLIPS
What is the purpose of collecting evidence to inform clinical practice in psychology concerning the effects of psychological or other interventions? To quote Paul’s (1967) article that has been cited 330 times before November 4, 2008, it is to determine the answer to the question: “What treatment, by whom, is most effective for this individual with that specific problem, under which set of circumstances?” (p. 111). Another answer is pitched at a systemic level, rather than concerning individuals. That is, research evidence can inform health-care professionals and consumers about psychological and behavioral interventions that are more effective than pharmacological treatments, and to improve the overall quality and cost-effectiveness of psychological health service provision (American Psychological Association [APA] Presidential Task Force on Evidence-Based Practice, 2006). The most general answer is that research evidence can be used to improve outcomes for clients, service providers, and society in general. The debate about what counts as evidence of effectiveness in answering this question has attracted considerable controversy (Goodheart, Kazdin, & Sternberg, 2006; Norcross, Beutler, & Levant, 2005). At one end of a spectrum, evidence from research on psychological treatments can be emphasized. Research-oriented psychologists have promoted the importance of scientific evidence in the concept of empirically supported treatment. Empirically supported treatments (ESTs) are those that have been sufficiently subjected to scientific research and have been shown to produce beneficial effects in wellcontrolled studies (i.e., efficacious), in more natural clinical environments (i.e., effective), and are the most cost-effective (i.e., efficient) (Chambless & Hollon, 1998). The effective and efficient criteria of Chambless and Hollon (1998) have been amalgamated under the term “clinical utility” (APA Presidential Task Force on Evidence-Based Practice, 2006; Barlow, Levitt, & Bufka, 1999). At the other end of the spectrum are psychologists who value clinical expertise as the source of evidence more highly, and they can rate subjective impressions and skills acquired in practice as providing personal evidence for guiding treatment (Hunsberger, 2007). Kazdin (2008) has asserted that the schism between clinical researchers and practitioners on the issue of evidence is deepening. Part of the problem, which suggests at least part of the solution, is that research had concentrated on empirical evidence of treatment efficacy, but more needs c01 20 April 2012; 12:43:29 3 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. to be conducted to elucidate the relevant parameters of clinical experience. In a separate dimension from the evidence– experience spectrum have been concerns about designing interventions that take into account the uniqueness of the individual client. Each of us can be seen as a unique mix of levels of variables such as sex, age, socioeconomic and social status, race, nationality, language, spiritual beliefs or personal philosophies, values, preferences, level of education, as well as number of symptoms, diagnoses (comorbidities), or problem behavior excesses and deficits that may bring us into professional contact with clinical psychologists. The extent to which there can be prior evidence from research or clinical experience to guide individual’s interventions when these variables are taken into account is questionable, and so these individual differences add to the mix of factors when psychologists deliberate on treatment recommendations with an individual client. Recognizing each of these three factors as necessary considerations in intervention selection, the APA Presidential Task Force on Evidence-Based Practice (2006, p. 273) provided its definition: “Evidence-based practice in psychology (EBPP) is the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences.” The task force acknowledged the similarity of its definition to that of Sackett, Straus, Richardson, Rosenberg, and Haynes (2000) when they defined evidence-based practice in medicine as “the integration of best research evidence with clinical expertise and patient values” (p. 1). Available research evidence is the base or starting point for EBPP. So, in recommending a particular intervention from a number of available ESTs, the psychologist, using clinical expertise with collaboration from the client, weighs up the options so that the best treatment for that client can be selected. As we understand it, clinical expertise is not to be considered as an equal consideration to research evidence, as some psychologists have implied (Hunsberger, 2007). Like client preferences, the psychologist’s expertise plays an essential part in sifting among ESTs the clinician has located from searching the evidence. Best research evidence is operationalized as ESTs. Treatment guidelines can be developed following review of ESTs for particular populations and diagnoses or problem behaviors. According to the APA (APA, 2002; Reed, McLaughlin, & Newman, 2002), treatment guidelines should be developed to educate consumers (e.g., clients and health-care systems) and professionals (e.g., clinical psychologists) about the existence and benefits of choosing ESTs for specific disorders over alternative interventions with unknown or adverse effects. Treatment guidelines are intended to recommend ESTs, but not make their use mandatory as enforceable professional standards (Reed et al., 2002). The declared status, implications, misunderstanding, and misuse of treatment guidelines based on ESTs continue to be sources of controversy (Reed et al., 2002; Reed & Eisman, 2006). Our chapter examines the issues just raised in more detail. We start with a review of the history and methods of determining evidencebased practice in medicine because the evidence-based practice movement started in that discipline, and has led the way for other human services. Psychologists, especially those working for children and young people, tend to work collaboratively with other professionals and paraprofessionals. Many of these groups of colleagues subscribe to the evidence-based practice movement through their professional organizations. We sample those organizations’ views. The generalizability to psychology of methods for evidence-based decision making in medicine is questionable, and questioned. Next we examine criteria for determining the strength of evidence for interventions in psychology. These criteria are notably different to those employed in medicine, particularly concerning the relative value to the evidence base of 4 Overview and Foundational Issues c01 20 April 2012; 12:43:29 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. research in psychology that has employed methods distinct from medicine (e.g., small-N design research). The controversies concerning treatment guidelines derived from ESTs are outlined briefly. The extent to which special considerations exist regarding treatment selection for children and adolescents is then discussed. Finally, we highlight some of the aspects of EBPP that require further work by researchers and clinicians. EVIDENCE-BASED MEDICINE Evidence-based practice can be perceived as both a philosophy and a set of problem-solving steps, using current best research evidence to make clinical decisions. In medicine, the rationale for clinicians to search for best evidence when making diagnostic and treatment decisions is the desire or duty, through the Hippocratic Oath, to use the optimal method to prevent or cure physical, mental, or social ailments and promote optimal health in individuals and populations (Jenicek, 2003). This section of the chapter provides an overview of the history of evidence-based practice in medicine (EBM), as well as a description of the current methodology of EBM. A critical reflection on the process and evidence used for EBM is also provided, leading to an introduction of the relevance of evidence-based practice (EBP) to other disciplines, including psychological interventions. History of Evidence-Based Practice in Medicine The earliest origins of EBM can be found in 19th-century Paris, with Pierre Louis (1787– 1872). Louis sought truth and medical certainty through systematic observation of patients and the statistical analysis of observed phenomena; however, its modern origins and popularity are found much more recently in the advances in epidemiological methods in the 1970s and 1980s (Jenicek, 2003). One of the key people responsible for the emergence and growth of EBM, the epidemiologist Archie Cochrane, proposed that nothing should be introduced into clinical practice until it was proven effective by research centers, and preferably through double-blind randomized controlled trials (RCTs). Cochrane criticized the medical profession for its lack of rigorous reviews of the evidence to guide decision making. In 1972, Cochrane reported the results of the first systematic review, his landmark method for systematically evaluating the quality and quantity of RCT evidence for treatment approaches in clinical practice. In an early demonstration of the value of this methodology, Cochrane demonstrated that corticosteroid therapy, given to halt premature labor in high-risk women, could substantially reduce the risk of infant death (Reynolds, 2000). Over the past three decades, the methods and evidence used for EBM have been extended, refined, and reformulated many times. From the mid-1980s, a proliferation of articles has instructed clinicians about the process of accessing, evaluating, and interpreting medical evidence; however, it was not until 1992 that the term evidence-based medicine was formally coined by Gordon Guyatt and the Evidence-Based Working Group at McMaster University in Canada. Secondary publication clinical journals also started to emerge in the early to mid-1990s, with the aim of summarizing original articles deemed to be of high clinical relevance and methodological rigor (e.g., Evidence-Based Medicine, ACP Journal Club, Evidence-Based Nursing, EvidenceBased Mental Health). Various guidelines for EBM have been published, including those from EvidenceBased Working Group (Guyatt et al., 2000), the Cochrane Collaboration, the National Institute for Clinical Excellence (NICE), and the British Medical Journal Clinical Evidence group, to name but a few. David Sackett, one of the strong proponents and authors in EBM, describes it as an active Rationale and Standards of Evidence in Evidence-Based Practice 5 c01 20 April 2012; 12:43:29 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. clinical decision-making process involving five sequential steps: (1) convert the patient’s problem into an answerable clinical question; (2) track down the best evidence to answer that question; (3) critically appraise the evidence for its validity—closeness to truth; impact—size of the effect; and applicability— usefulness in clinical practice; (4) integrate the appraisal with the practitioner’s clinical expertise and the patient’s unique characteristics, values, and circumstances; and (5) evaluate the change resulting from implementing the evidence in practice, and seek ways to improve (Sackett et al., 2000, pp. 3–4). The guidelines for EBM are generally characterized by this sort of systematic process for determining the level of evidence for treatment choices available to clinicians, while at the same time recognizing the unique characteristics of the individual’s characteristics, situation, and context. It is not difficult to see the inherent benefits of EBM, and health professionals have been quick to recognize the potential benefit of adopting it as standard practice. Citing a 1998 survey of UK general practitioners (GPs), Sackett and colleagues (2000) wrote that most reported using search techniques, with 74% accessing evidence-based summaries generated by others, and 84% seeking evidence-based practice guidelines or protocols. The process of engaging in EBM requires some considerable understanding of research and research methods, and there is evidence that health professionals struggle to use EBM in their actual practice. For example, Sackett et al. (2000) found that GPs had trouble using the rules of evidence to interpret the literature, with only 20% to 35% reporting that they understood appraising tools described in the guidelines. Clinicians’ ability to practice EBM also may be limited by lack of time to master new skills and inadequate access to instant technologies (Sackett et al., 2000). In addition to these practical difficulties, there have also been some criticisms of EBM’s dominant methodology. An early criticism of EBM, and nearly all EBP approaches, is that it appears to give greatest weight to science with little attention to the “art” that also underlies the practice of medicine, nursing, and other allied health professions. For example, Guyatt and colleagues cited attention to patients’ humanistic needs as a requirement for EBM (EvidenceBased Medicine Working Group, 1992). Nursing and other health-care disciplines note that EBP must be delivered within a context of caring to achieve safe, effective, and holistic care that meets the needs of patients (DiCenso, Cullum, Ciliska, & Guyatt, 2004). Evidence-based practice is also criticized as being “cookbook care” that does not take the individual into account. Yet a requirement of EBP is that incorporating research evidence into practice should consistently take account of the patient’s unique circumstances, preferences, and values. As noted by Sackett et al. (2000), when these three elements are integrated to inform clinical decision making, “clinicians and patients form a diagnostic and therapeutic alliance which optimizes clinical outcomes and quality of life” (p. 1). One of the key issues in the use of EBM is the debate around what constitutes best evidence from the findings of previous studies and how these various types of evidence are weighted, or even excluded, in the decisionmaking process. The following section first outlines the way in which evidence currently tends to be judged and then provides a more indepth analysis of each type of evidence. Levels and Types of Evidence Partly through the work of Archie Cochrane, the quality of health care has come to be judged in relation to a number of criteria: efficacy (especially in relation to effectiveness), efficiency, and equity. Along with acceptability, access, and relevance, these criteria have been called the “Maxwell Six” and have formed the foundation of decision making around service provision and funding in the United Kingdom’s National Health Service (Maxwell, 1992). Other health systems 6 Overview and Foundational Issues c01 20 April 2012; 12:43:29 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. around the world have also adopted these criteria to aid in policy decision making, in both publicly and privately funded settings. Despite this there has been a tendency for evidence relating to effectiveness to dominate the decision-making process in EBM and, in particular, effectiveness demonstrated through RCTs. There has been considerable debate about the ethical problems arising from clinicians focusing too much on efficacy and not enough on efficiency when making decisions about treatment (Maynard, 1997). In any health system with limited resources, the decision to use the most effective treatment rather than the most efficient one has the potential to impact on the likelihood of having sufficient resources to deliver that treatment, or any other, in the future. By treating one person with the most effective treatment the clinician may be taking away the opportunity to treat others, especially if that treatment is very expensive in relation to its effectiveness compared to less expensive treatment options. According to current EBM methods, the weight that a piece of evidence brings to the balance of information used to make a decision about whether a treatment is supported by evidence can be summarized in Table 1.1. Although there are subtle variations in this hierarchy between different EBM guidelines and other publications, they all typically start with systematic reviews of RCTs at the top and end with expert opinion at the bottom. Before discussing systematic reviews, metaanalyses, and clinical guidelines, it is important to understand the type of study that these sources of evidence are typically based on: the RCT. RCTs are a research design in which the participants are randomly assigned to treatment or control groups. RCTs are really a family of designs, with different components such as blinding (participants and experimenter), different randomization methods, and other differences in design. Analysis of RCTs is quantitative, involving estimation of the statistical significance or probability of the difference in outcomes observed between the treatment and control groups in the study. The probabilities obtained are an estimate of the likelihood of that size difference, or something larger, existing in the population. There are a number of international databases that store and provide information from RCTs, including the Cochrane Central Register of Controlled Trials (CENTRAL; mrw.interscience.wiley.com/cochrane/cochrane _clcentral_articles_fs.html), OTseeker (www .otseeker.com), PEDro (www.pedro.fhs.usyd .edu.au), and the Turning Research Into Practice (TRIP; www.tripdatabase.com) database. Another effort to increase the ease with which clinicians can access and use evidence from RCTs has come through efforts to standardize the way in which they are reported, such as the CONSORT Statement and other efforts from the CONSORT Group (www.consortstatement.org). The major strength of well-designed RCTs and other experimental designs is that the researcher has control over the treatment given to the experimental groups, and also has control over or the ability to control for any confounding factors. The result of this control TABLE 1.1 Typical Hierarchy of Evidence for EBM Level of Evidence Type of Evidence 1 (High) Systematic reviews or meta-analysis of randomized controlled trials (RCTs) OR Evidence-based clinical practice guidelines based on systematic reviews of RCTs 2 At least one well-designed RCT 3 Well-designed quasi-experimental studies 4 Well-designed case control and cohort studies 5 Systematic reviews of descriptive and qualitative studies 6 A single descriptive or qualitative study 7 (Low) The opinion of authorities and/or expert committees Source: Adapted from Melnyk and Fineout-Overholt (2005, p. 10) Rationale and Standards of Evidence in Evidence-Based Practice 7 c01 20 April 2012; 12:43:29 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. is the ability, arguably above all other designs, to infer causation—and therefore efficacy— from the differences in outcomes between the treatment and control groups. RCTs, and other purely experimental designs, are often criticized for having poor ecological validity, where the conditions of the experiment do not match the conditions or populations in which the treatment might be delivered in the real world. Low ecological validity can, although does not necessarily, lead to low external validity where the findings of the study are not generalizable to other situations, including the real world. Another issue with RCTs is that they are relatively resource intensive and this leads to increased pressure to get desired results or to suppress any results that do not fit the desired outcome (Gluud, 2006; Simes, 1986). In social sciences, RCTs are not always possible or advisable, so many systematic reviews and meta-analyses in these areas have less restrictive inclusion criteria. Commonly, these include studies that compare a treated clinical group with an untreated or attention control group (Kazdin & Weisz, 1998; Rosenthal, 1984). In these situations, a quasiexperimental design is often adopted. There is also evidence that RCTs are not necessarily any more accurate at estimating the effect of a given treatment than quasi-experimental and observational designs, such as cohort and case control studies (Concato, Shah, & Horwitz, 2008). Systematic reviews provide a summary of evidence from studies on a particular topic. For EBM this typically involves the formation of an expert committee or panel, followed by the systematic identification, appraisal, and synthesis of evidence from relevant RCTs relating to the topic (Melnyk & Fineout-Overholt, 2005). The result of the review is usually some recommendation around the level of empirical support from RCTs for a given diagnostic tool or treatment. The RCT, therefore, is seen as the gold standard of evidence in EBM. Systematic review and meta-analysis are closely related EBP methods to evaluate evidence from a body of sometimes contradictory research to assess treatment quality. There is a great deal of overlap between methods, stemming in part from their different origins. The systematic review arises from EBM and the early work of Cochrane and more recently Sackett and colleagues (2000). Metaanalysis originated in the 1970s, with the contributions of Glass (1976) in education and Rosenthal (1984) in psychology being central. Meta-analyses are a particular type of systematic review. In a meta-analysis, measures of the size of treatment effects are obtained from individual studies. The effect sizes from multiple studies are then combined using a variety of techniques to provide a measure of the overall effect of the treatment across all of the participants in all of the studies included in the analysis. Both approaches use explicit methods to systematically search, critically appraise for quality and validity, and synthesize the literature on a given issue. Thus, a key aspect is the quality of the individual studies and the journals in which they appear. Searches ideally include unpublished reports as well as published reports to counteract the “file drawer” phenomenon: Published findings as a group may be less reliable than they seem because studies with statistically nonsignificant findings are less likely to be published (Rosenthal, 1984; Sackett et al., 2000). The principal difference between systematic review and meta-analysis is the latter includes a statistical method for combining results of individual studies that produces a larger sample size, reduces random error, and has greater power to determine the true size of the intervention effect. Not only do these methods compensate for the limited power of individual studies resulting in a Type I error, the failure to detect an actual effect when one is present, they can also reconcile conflicting results (Rosenthal, 1984; Sackett et al., 2000). The criticisms of meta-analysis mostly relate to the methodological decisions made during the process of conducting a meta-analysis, 8 Overview and Foundational Issues c01 20 April 2012; 12:43:29 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. often reducing the reliability of findings. This can result in meta-analyses of the same topic yielding different effect sizes (i.e., summary statistic), with these differences stemming from differences in the meta-analytic method and not just differences in study findings (Flather, Farkouh, Pogue, & Yusuf, 1997). Methodological decisions that can influence the reliability of the summary statistic produced from a meta-analysis include the coding system used to analyze studies, how inclusive the study selection process was, the outcome measures used or accepted, and the use of raw effect sizes or adjusted sample sizes (Flather et al., 1997). Some meta-analyses are conducted without using a rigorous systematic review approach, and this is much more likely to produce a mathematically valid but clinically invalid conclusion (Kazdin & Weisz, 1998; Rosenthal, 1984). There are also some criticisms of the meta-analysis approach, specifically from child and adolescent psychology literature. Meta-analyses can obscure qualitative differences in treatment execution such as investigator allegiance (Chambless & Hollon, 1998; Kazdin & Weisz, 1998) and are limited by confounding among independent variables such as target problems, which tend to be more evident with certain treatment methods (Kazdin & Weisz, 1998). Evidence-based clinical practice guidelines are also included in this top level of evidence, with the caveat that they must be primarily based on systematic reviews of RCTs. The purpose of these practice guidelines is to provide an easy-to-follow tool to assist clinicians in making decisions about the treatment that is most appropriate for their patients (Straus, Richardson, Glasziou, & Haynes, 2005). There are some issues with the publication and use of evidence-based clinical practice guidelines. One problem is that different groups of experts can review the same data and arrive at different conclusions and recommendations (Hadorn, Baker, Hodges, & Hicks, 1996). Some guidelines are also criticized for not being translated into tools for everyday use. One of the major drawbacks of clinical guidelines is that they are often not updated frequently to consider new evidence (Lohr, Eleazer, & Mauskopf, 1998). In addition to individual researchers and groups of researchers, there are a large number of organizations that conduct systematic reviews, publish clinical practice guidelines, and publish their findings in journals and in databases on the Internet, including the Cochrane Collaboration (www.cochrane.org/), the National Institute for Clinical Evidence (NICE; www.nice.org.uk/), the Joanna Briggs Institute (www.joannabriggs.edu.au/about /home.php), and the TRIP database (www .tripdatabase.com/index.html). For example, the Cochrane Collaboration, an organization named after Archie Cochrane, is an international network of researchers who conduct systematic reviews and meta-analyses and provide the results of these to the research, practice, and policy community via the Cochrane Library. The overall strengths of systematic reviews, meta-analyses, and clinical practice guidelines relate partly to the nature of the methods used and partly to the nature of RCTs, which were discussed earlier. The ultimate goal of scientific research is to contribute toward a body of knowledge about a topic (Bowling, 1997). Systematic reviews and clinical practice guidelines are essentially trying to summarize the body of knowledge around a particular treatment, which is something that would otherwise take a clinician a very long time to do on their own, so it is easy to see the value in a process that does this in a rigorous and systematic way. There are a number of overall weaknesses for this top level of evidence for EBM. For example, the reliability of findings from systematic reviews has been found to be sensitive to many factors, including the intercoder reliability procedures adopted by the reviewers (Yeaton & Wortman, 1993). It is possible for different people using the same coding system Rationale and Standards of Evidence in Evidence-Based Practice 9 c01 20 April 2012; 12:43:29 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. to come up with different conclusions about the research evidence for a particular treatment. The selection of comparable studies is also a major issue, and especially in matching comparison groups and populations. The control and treatment groups in RCTs and other studies are usually not identical and this raises issues, particularly in the use of meta-analyses, where the treatment effects are being combined (Eysenck, 1994). The mixing of populations can also cause problems, where the treatment may only be effective in one or more specific populations but the review has included studies from other populations where the treatment is not effective. The overall effect of mixing these populations is to diminish the apparent effectiveness of the treatment under review (Eysenck, 1994). Other issues with this top level of evidence relate to the information not included. Once a review or guideline is completed it must be constantly updated to check for new evidence that may change the conclusions. There are also issues around the information not included in the review process, including nonRCT studies, gray literature, such as technical reports and commissioned reports, and unpublished studies. Studies that do not get into peer-reviewed journals, and therefore usually get excluded from the review process, are often those that did not have significant results. The effect of this is that reviews will often exaggerate the effectiveness of a treatment through the exclusion of studies where the treatment in question was found to be ineffective. For this reason there has been a recent move to introduce what is called the “failsafe N” statistic. This is the hypothetical number of unpublished (or hidden) studies showing, on average, no effect that would be required to overturn the statistically significant effects found from review of the published (or located) studies results (Becker, 2006). Quasi-experimental designs are the same as RCTs but the groups are not randomly assigned, there is no control group, or they lack one or more of the other characteristics of an RCT. In many situations randomization of participants or having a control group is not practically and/or ethically possible. The strengths of quasi-experimental designs come from the degree of control that the research has over the groups in the study, and over possible confounding variables. A well-designed quasiexperimental design has the important characteristic of being able to infer some degree of causation from differences in outcomes between study groups. Case control studies identify a population with the outcome of interest (cases) and a population without that outcome (controls), then collects retrospective data to try to determine their relative exposure to factors of interest (Grimes & Schulz, 2002). There are numerous strengths of case control studies. They have more ecological validity than experimental studies and they are good for health conditions that are very uncommon (Grimes & Schulz, 2002). There is a relatively clear temporal sequence, compared to lower level evidence, which allows some degree of causality to be inferred (Grimes & Schulz, 2002). They are also relatively quick to do, relatively inexpensive, and can look at multiple potential causes at one time (Grimes & Schulz, 2002). The weaknesses of case control studies include the inability to control potentially confounding variables except through statistical manipulations and the reliance on participants to recall information or retrospectively collating information from existing data; the choice of control participants is also difficult (Grimes & Schulz, 2002; Wacholder, McLaughlin, Silverman, & Mandel, 1992). All of this often leads to a lot of difficulty in correctly interpreting the results of case control studies (Grimes & Schulz, 2002). Cohort studies are longitudinal studies where groups are divided in terms of whether they receive or do not receive a treatment or exposure of interest, and are followed over time to assess the outcomes of interest (Roberts & Yeager, 2006). The strengths of cohort studies include the relatively clear 10 Overview and Foundational Issues c01 20 April 2012; 12:43:29 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. temporal sequence that can be established between the introduction of an intervention and any subsequent changes in outcome variables, making the establishment of causation possible, at least to some degree. The limitations of cohort studies include the difficulty in controlling extraneous variables, leading to a relatively limited ability to infer causality. They are also extremely resource intensive and are not good where there is a large gap between treatment and outcome (Grimes & Schulz, 2002). Descriptive studies involve the description of data obtained relating to the characteristics of phenomena or variables of interest in a particular sample from a population of interest (Melnyk & Fineout-Overholt, 2005). Correlational studies are descriptive studies where the relationship between variables is explored. Qualitative studies collect nonnumeric data, such as interviews and focus groups, with the analysis usually involving some attempt at describing or summarizing certain phenomena in a sample from a population of interest (Melnyk & Fineout-Overholt, 2005). Descriptive studies in general sit low down on the ladder of evidence quality due to their lack of control by the researcher and therefore their inability to infer causation between treatment and effect. On the other hand, much research is only feasible to conduct using this methodology. Qualitative research in health services mostly arose from a desire to get a deeper understanding of what the quantitative research finding meant for the patient and provider (Mays & Pope, 1996). It asks questions such as “How do patients perceive . . . ?” and “How do patients value the options that are offered?” Expert opinion is material written by recognized authorities on a particular topic. Evidence from these sources has the least weight in EBM, although to many clinicians the views of experts in the field may hold more weight than the higher level evidence outlined above. Despite its criticisms and methodological complexity, EBM is seen by most clinicians as a valid and novel way of reasoning and decision making. Its methods are widely disseminated through current clinical education programs at all levels throughout the world. As proposed by Gordon Guyatt and colleagues in 1992, EBM has led to a paradigm shift in clinical practice (Sackett et al., 2000). CURRENT STATUS OF EBP MOVEMENTS ACROSS HEALTH AND EDUCATION PROFESSIONS Evidence-based practice was initially in the domain of medicine, but now most human service disciplines subscribe to the principles of EBP. For example, EBP or the use of empirically supported treatment is recommended by the APA, Behavior Analyst Certi- fication Board (BACB), American Psychiatric Association, National Association of Social Workers, and General Teaching Council for England. For U.S. education professionals, EBP has been mandated by law. The No Child Left Behind Act of 2001 (NCLB; U.S. Department of Education, 2002) was designed to make the states, school districts, principals, and teachers more answerable for the performances shown by the students for whom they were providing education services. Along with an increase in accountability, the NCLB requires “schoolwide reform and ensuring the access of children to effective, scientifically based instructional strategies and challenging academic content” (p. 1440). The advent of the NCLB and the resulting move toward EBP occurred because, despite there being research conducted on effective and efficient teaching techniques (e.g., Project Follow Through: Bereiter & Kurland, 1981; Gersten, 1984) and a growing push for accountability from the public (Hess, 2006), this seldom was translated into actual practice. Education appears to have been particularly susceptible to implementing programs that were fads, based on little more than personal ideologies and good marketing. Much money and time has been lost by school districts adopting programs that have no Rationale and Standards of Evidence in Evidence-Based Practice 11 c01 20 April 2012; 12:43:30 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. empirical support for their effectiveness, such as the adoption of known ineffective substance abuse prevention programs (Ringwalt et al., 2002) or programs that are harmful for some students and their families, such as facilitated communication (Jacobson, Mulick, & Schwartz, 1995). This leads not only to resource wastage, but an even greater cost in lost opportunities for the students involved. It seems like common sense for EBP to be adopted by other disciplines, as it is difficult to understand why reputable practitioners, service providers, and organizations would not want to provide interventions that have been shown to be the most effective and efficient. Yet, despite this commonsense feeling, the codes of conduct and ethical requirements, and mandated laws, many disciplines (including medicine) have found it difficult to bridge the gap between traditional knowledge-based practice and the EBP framework (Greenhalgh, 2001; Stout & Hayes, 2004). Professions such as social work (Roberts & Yeager, 2006; Zlotnick & Solt, 2006), speech language therapy (Enderby & Emerson, 1995, 1996), occupational therapy (Bennett & Bennett, 2000), and education (Shavelson & Towne, 2002; Thomas & Pring, 2004) have all attempted to overcome the difficulties that have arisen as they try to move EBP from a theoretical concept to an actual usable tool for everyday practitioners. Although some of these disciplines have unique challenges to face, many are common to all. Many human services disciplines have reported that one of the major barriers to the implementation of EBP is the lack of sound research. For example, speech language therapy reports difficulties with regard to the quality and dissemination of research. A review of the status of speech language therapy literature by Enderby and Emerson (1995, 1996) found that there was insufficient quality research available in most areas of speech language therapy; however, they did find that those areas of speech therapy that were associated with the medical profession (e.g., dysphasia and cleft palate) were more likely to have research conducted than those associated with education (e.g., children with speech and language disorders and populations with learning disabilities). There continues to be a lack of agreement on speech language therapies effectiveness (Almost & Rosenbaum, 1998; Glogowska, Roulstone, Enderby, & Peters, 2000; Robertson & Weismer, 1999). In addition to the lack of research, Enderby and Emerson were also concerned about the manner in which health resources were being allocated for speech language therapists. They found that of the resources allocated to speech language therapy approximately 70% were being used with children with language impairments, despite there being limited quality evidence of the effectiveness of speech language therapy with the population at that time. They also identified that dysarthria was the most commonly acquired disorder, but had very little research outside of the Parkinson’s disease population. So, again, many resources have been allocated to programs that have no evidence of effectiveness. This questionable allocation of resources is seen in other fields also. For example, a number of studies have found that people seeking mental health treatments are unlikely to receive an intervention that would be classified as EBP and many will receive interventions that are ineffective (Addis & Krasnow, 2000; Goisman, Warshaw, & Keller, 1999). A number of organizations and government agencies within a variety of fields try to facilitate the much-needed research. Examples in education include the United States Department for Children, Schools, and Families (previously Department for Education and Skills), the National Research Council (United Kingdom), and National Teacher Research Panel (United Kingdom). Similarly, in social work the National Association of Social Workers (United States) and the Society for Social Work and Research (United States) both facilitate the gathering of evidence to support the use of EBP within their field; 12 Overview and Foundational Issues c01 20 April 2012; 12:43:30 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. however, even if these organizations generate and gather research demonstrating the effectiveness of interventions, this does not always translate into the implementation of EBP. As mentioned previously, resource allocation does not necessarily go to those treatments with evidence to support them due to a lack of effective dissemination on the relevant topics. Despite medicine being the profession with the longest history of EBP, Guyatt et al. (2000) stated that many clinicians did not want to use original research or they fail to do so because of time constraints and/or lack of understanding on how to interpret the information. Rosen, Proctor, Morrow-Howell, and Staudt (1995) found that social workers also fail to consider empirical research when making research decisions. They found that less than 1% of the practice decisions were justified by research. Although the lack of research-based decision making is concerning, many professions are attempting to disseminate information in a manner that is more user friendly to its practitioners. Practice or treatment guidelines have been created to disseminate research in a manner that will facilitate EBP. These guidelines draw on the empirical evidence and expert opinion to provide specific best-practice recommendations on what interventions or practices are the most effective/efficient for specific populations (Stout & Hayes, 2004). There are a number of guideline clearinghouses (National Guidelines Clearinghouse [www.guideline.gov/] and What Works Clearinghouse [ies.ed.gov/ncee/wwc/]). Associations and organizations may also offer practice guidelines (e.g., American Psychiatric Association). Although there appears to have been emphasis placed on EBP, there is still much work to be done in most human service fields, including medicine, to ensure that there is appropriate research being conducted, that this research is disseminated to the appropriate people, and that the practitioners then put it into practice. Reilly, Oates, and Douglas (2004) outlined a number of areas that the National Health Service Research and Development Center for Evidence-Based Medicine had identified for future development. They suggest that there is a need for a better understanding of how practitioners seek information to inform their decisions, what factors influence the inclusion of this evidence into their practice, and the value placed, both by patients and practitioners, on EBP. In addition, there is a need to develop information systems that facilitate the integration of evidence into the decision-making processes for practitioners and patients. They also suggest that there is a need to provide effective and efficient training for frontline professionals in evidence-based patient care. Finally, they suggest that there is simply a need for more research. EVIDENCE IN PSYCHOLOGY Although RCT research is still held up as the gold standard of evidence in medical science, in other clinical sciences, especially psychology, best research evidence has been less focused on RCTs as being the only scientifically valid approach. Randomized controlled trials are still held in high regard in clinical psychology, but it has long been recognized that alternative designs may be preferred and still provide strong evidence, depending on the type of intervention, population studied, and patient characteristics (APA Presidential Task Force on Evidence-Based Practice, 2006; Chambless & Hollon, 1998). Research Methods Contributing to Evidence-Based Practice in Psychology We have described and discussed RCTs in the previous section on EBM. The issues concerning RCTs and their contribution to the clinical psychology evidence base are similar. The inclusion of evidence of treatment efficacy and utility from research paradigms other than RCT methods and approximations thereto has Rationale and Standards of Evidence in Evidence-Based Practice 13 c01 20 April 2012; 12:43:30 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. been recommended for clinical psychology since the EBPP movement commenced (Chambless & Hollon, 1998). The APA Presidential Task Force on Evidence-Based Practice (2006) findings support the use of qualitative studies, single-case or small-N experimental designs, and process-outcome studies or evaluations, in addition to the RCT and systematic review techniques employed in EBM. A more in-depth discussion of some of these methods will be provided. Psychologists and members of other professions that are more influenced by social science research methods than medicine may consider that findings from qualitative research add to their EBP database. Although qualitative research is not accepted as legitimate scientific study by many psychologists, and not usually taught to clinical psychologists in training, it may well have its place in assessing variables regarding clinical expertise and client preferences (Kazdin, 2008). Research questions addressable through qualitative research relevant to EBPP are similar to those mentioned for EBM. Another important source of scientific evidence for psychologists, educators, and other nonmedical professionals is that derived from small-N research designs, also known as single-case, single-subject, or N ¼ 1 designs. These alternative labels can confuse, especially since some single-case designs include more than one subject (e.g., multiple baseline across subjects usually include three or more participants). The most familiar small-N designs are ABAB, multiple baseline, alternating treatments, and changing criterion (Barlow & Hersen, 1984; Hayes, Barlow, & Nelson-Gray, 1999; Kazdin, 1982). Data from small-N studies have been included as sources of evidence, sometimes apparently equivalent to RCTs (Chambless & Hollon, 1998), or at a somewhat lower level of strength than RCTs (APA, 2002). Strength of evidence from small-N designs. Small-N designs can be robust in terms of controlling threats to internal validity; however, external validity has often been viewed as problematic. This is because participants in small-N studies are not a randomly selected sample from the whole population of interest. The generality of findings from smallN studies is established by replication across more and more members of the population of interest in further small-N studies. A hypothetical example of the process of determining the generality of an intervention goes like this: (a) A researcher shows that a treatment works for a single individual with a particular type of diagnosis or problem; (b) Either the same or another researcher finds the same beneficial effects with three further individuals; (c) Another researcher reports the same findings with another small set of individuals; and so on. At some point, sufficient numbers of individuals have been successfully treated using the intervention that generality can be claimed. Within a field such as a particular treatment for a particular disorder, small-N studies can be designed so results can be pooled to contribute to an evidence base larger than N ¼ 1 to 3 or 4 (Lord et al., 2005). Even if there is an evidence base for an intervention from a series of small-N studies, every time another individual receives the same treatment, the clinician in scientistpractitioner role evaluates the effects of the intervention using small-N design methods. Thus, every new case is clinical research to determine the efficacy and effectiveness of this treatment for that individual. Clinical psychologists can be cautioned that physicians and medical researchers have a similar-sounding name for an experimental design with their “N of 1 trials.” The definition of N of 1 trials can vary somewhat but typically they are described as “randomised, double blind multiple crossover comparisons of an active drug against placebo in a single patient” (Mahon, Laupacis, Donner, & Wood, 1996, p. 1069). These N of 1 trials are continued until the intervention, usually a drug, in question shows consistently better effects than its comparison treatment or control condition. 14 Overview and Foundational Issues c01 20 April 2012; 12:43:30 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. Then the intervention is continued, or discontinued if there were insufficient beneficial effects. The N of 1 trials can be considered top of the hierarchy of evidence in judging strength of evidence for individual clinical decisions, higher even than systematic reviews of RCTs (Guyatt et al., 2000). This is because clinicians do not have to generalize findings of beneficial effects of the treatment to this patient from previous researched patients. Consistent beneficial findings from many N of 1 trials add up to an “N of many” RCT, thus providing evidence of the generality of findings. An N of 1 trial is quite similar to the alternating treatments small-N design employed in clinical psychology, especially applied behavior analysis; however, they are not identical and the differences may be overcome for some psychosocial interventions only with careful planning, but many interventions could not be assessed by fitting them into an N of 1 trial format. Further details are beyond the scope of this chapter; however, close study of the methodological requirements of both N of 1 designs and small-N designs can show the similarities and differences (Barlow & Hersen, 1984; Guyatt et al., 2000; Hayes et al., 1999; Kazdin, 1982; Mahon et al., 1996). Some state that small-N designs may be most suitable for evaluating new treatments (Lord et al., 2005), others that single-case studies are most suitable for determining treatment effects for individuals (APA Presidential Task Force on Evidence-Based Practice, 2006). We do not disagree with either, except to point out that it has been argued that small-N studies can contribute much more to EBPP than these two advantages. In the following section, we review how ESTs can be determined from an evidence base consisting entirely of small-N studies. Criteria for Assessing Efficacy Lonigan, Elbert, and Johnson (1998) tabulated criteria for determining whether an intervention for childhood disorders could be considered well-established (i.e., efficacious) or probably efficacious (i.e., promising). For the former, they recommended at least two well-conducted RCT standard studies by independent research teams or a series of independent well-designed small-N studies with at least nine participants carefully classified to the diagnostic category of interest showing that the intervention was better than alternative interventions. The availability of treatment manuals was recommended. For “promising” treatments, the criteria were relaxed to allow nonindependent RCTs, comparing treatment to notreatment, or a minimum of three small-N studies. These criteria followed those established at the time for psychological therapies in general (Chambless & Hollon, 1998). We have discussed the determination of empirical support from RCTs already. The next sections will examine how evidence is derived from small-N studies: First, how evidence is assessed from individual research reports; and second, how the evidence can be combined from a group of research articles addressing the same topic of interest. Evaluating Evidence From Small-N Research Designs How do those assessing the efficacy of a treatment from small-N designs measure the strength of the design for the research purpose and whether, or to what extent, a beneficial effect has been demonstrated? Chambless and Hollon (1998) recommended reviewers to rate single-case studies on the stability of their baselines, use of acceptable experimental designs, such as ABAB or multiple baseline designs, and visually estimated effects. Baselines need not always be stable to provide an adequate control phase; there are other valid designs (e.g., changing criterion, multielement experimental designs); and visual estimates of effects are not necessarily reliable (Cooper, Heron, & Heward, 2007). Validity Rationale and Standards of Evidence in Evidence-Based Practice 15 c01 20 April 2012; 12:43:30 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. and, probably, reliability of reviews using only Chambless and Hollon’s (1998) criteria could not be assured. Attempts to improve reliability and validity of judgments about the value of small-N design studies have included establishing more welldefined criteria. Quite detailed methods for evaluating small-N studies have been published. For example, Kratochwill and Stoiber (2002) describe the basis of a method endorsed by the National Association of School Psychologists (NASP) designed for evaluating multiple research reports that used single-case designs for establishing evidence-based recommendations for interventions in educational settings. The system is more inclusive of design variations beyond those recommended by Chambless and Hollon (1998), and reviewers are instructed to code multiple variables, including calculating effects sizes from graphical data. The coding form extended over 28 pages. Shernoff, Kratochwill, and Stoiber (2002) illustrated the assessment procedure and reported that, following extensive training and familiarity with the coding manual, they achieved acceptable agreement among themselves. They stated that the process took 2 hours for a single study, although, so our own graduate students report, it takes much longer if multiple dependent and independent variables or hundreds of data points had been features of the research article with which they chose to illustrate the NASP procedure. The NASP method was constructively criticized by Levin (2002), and is apparently being revised and expanded further (Kratochwill, 2005). Meanwhile, others have developed what appear to be even more labor intensive methods for attempting to evaluate objectively the strength of evidence from a series of smallN designs. As an example of a more detailed method, Campbell (2003) measured every data point shown on published graphs from 117 research articles on procedures to reduce problem behaviors among persons with autism. Each point was measured by using dividers to determine the distance between the point and zero on the vertical axis. Campbell calculated effect sizes for three variables: mean baseline reduction, percentage of zero data points, and percentage of nonoverlapping data points. Use of these statistical methods may have been appropriate considering the types of data Campbell examined; nonzero baselines of levels of problem behavior followed by intervention phases in which the researchers’ goal was to produce reduction to zero (see Jensen, Clark, Kircher, & Kristjansson, 2007, for critical review of meta-analytic tools for small-N designs). Nevertheless, the seemingly arduous nature of the task and the lack of generalizability of the computational methods to reviewing interventions designed to increase behaviors are likely to mitigate wider acceptance of Campbell’s (2003) method. A final example of methods to evaluate the strength of an evidence base for interventions is that outlined by Wilczynski and Christian (2008). They describe the National Standards Project (NSP), which was designed to determine the benefits or lack thereof of a wide range of approaches for changing the behaviors of people with autism spectrum disorders (ASD) aged up to 21 years. Their methods of quantitative review enabled the evidence from group and small-N studies to be integrated. Briefly, and to describe their method for evaluating small-N studies only, their review rated research articles based on their scientific merit first. Articles were assessed to determine whether they were sufficiently welldesigned in terms of experimental design, measurement of the dependent variables, assessment of treatment fidelity, the ability to detect generalization and maintenance effects, and the quality of the ASD classifi- cation of participants. If the article exceeded minimum criteria on scientific merit, the treatment effects were assessed as being beneficial, ineffective, adverse, or that the data were not sufficiently interpretable to decide on effects. Experienced trained reviewers were able to complete a review 16 Overview and Foundational Issues c01 20 April 2012; 12:43:30 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. for a research article in 1 hour or less with interreviewer agreement of 80% or more. Inevitable problems for all these systems for review of single case designs arise from the necessity of creating “one size fits all” rules. For example, the evaluative methods of both the NASP (Kratochwill & Stoiber, 2002) and NSP projects (Wilczynski & Christian, 2008) include consideration of level of interobserver agreement for determining partly the quality of measurement of the dependent variables. The minimum acceptable level of agreement is specified at, say, 70% or 80%, which makes it relatively convenient for reviewers to determine from reading the published research article that they are rating; however, it has been known for more than 30 years that an agreement percentage is rather meaningless without examination of how behaviors were measured, what interobserver agreement algorithm was employed, and the relative frequency or duration of the behaviors measured (Hawkins & Dotson, 1975). Another example concerns coding rules for evaluating the adequacy of baseline measures of behavior. Whether the criterion for a baseline phase of highest scientific merit is a minimum of 3 points (NASP) or 5 points (NSP), there will be occasions when the rule should be inapplicable. For example, in Najdowski, Wallace, Ellsworth, MacAleese, and Cleveland (2008), after more than 20 observational sessions during intervention showing zero severe problem behavior, a return to baseline in an ABAB design raised the rate of problem behavior to more than four per minute, which was higher than any points in the first A-phase. To continue with the baseline to meet NASP or NSP evaluative criteria would have been unnecessary to show experimental control and dangerous for the participant. These examples indicate why more detailed and codified methods for reviewing small-N studies quantitatively are, as yet, not firmly established. Although there may be satisfactory methods for fitting RCTs’ scientific merit and size of effects into databases for metaanalyses, that appears to be more problematic with small-N designs, given their flexibility in use (Hayes et al., 1999). Volume of Evidence From Small-N Studies Required to Claim That an Intervention Is Evidence Based Having rated the strength of evidence from individual research articles, the next step is to determine whether a group of studies on the same topic between them constitute sufficient evidence to declare that an intervention is an empirically supported treatment or a promising or emerging intervention. We discuss only empirically supported treatment criteria here. Consensus on what minimum criterion should apply has yet to be reached. Chambless and Hollon (1998) originally recommended a minimum of two independent studies with three or more participants (N $ 3) showing good effects for a total of N $ 6 participants. Lonigan et al. (1998) required three studies with N $ 3; that is, beneficial effects shown for N $ 9 participants. Since then, the bar has been raised. For instance, Horner et al. (2005) proposed that the criteria for determining that an intervention is evidence-based included a minimum of five small-N studies, from three or more separate research groups, with at least 20 participants in total. Wilczynski and Christian (2008) used similar criteria with $ 6 studies of strongest scientific merit totaling N $ 18 participants with no conflicting results from other studies of adequate design. Others have recommended similar standards: Reichow, Volkmar, and Cicchetti (2008) described a method for evaluating research evidence from both group and small-N designs, as had Wilczynski and Christian (2008). Finally, Reichow et al. (2008) set the criterion for an established EBP at $ 10 small-N studies of at least “adequate report strength” across three different locations and three different research teams with a total of 30 participants, or, if at least five studies of “strong report strength” with a total of 15 or more participants existed, that could substitute for the 10-study criterion. Rationale and Standards of Evidence in Evidence-Based Practice 17 c01 20 April 2012; 12:43:30 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. The rationale for selecting the numerical criteria is usually not stated. Thus, all we can state is that some systems (e.g., Reichow et al., 2008) are more conservative than others (e.g., Lonigan et al., 1998). Sometimes conservative criteria may be appropriate, for example, when the costs of treatment are high, and/or the intervention is exceedingly complex requiring highly skilled intervention agents, and/or the benefits for the intervention are less than ideal (i.e., it reduces problems to a more manageable level, but does not eliminate them), and/or some negative side effects have been observed. On the other hand, if relatively few resources are required to implement an effective and rapid intervention without unwanted side effects, fewer well-conducted studies may be needed to persuade consumers that the intervention is empirically supported, and therefore worth evaluating with the individual patient. This brief discussion should advise readers to examine and consider criteria carefully when reviewers claim that a particular intervention is an empirically supported treatment for a particular disorder or problem for a particular population. The discussions on evidence from small-N designs have left much out. For instance, Reichow et al. (2008) and Wilczynski and Christian (2008) developed algorithms for assessing the strength of evidence at different levels, although we have outlined only the highest levels. Both groups have also reported algorithms for determining ESTs from mixed methods (e.g., RCTs and small-Ns). Wilczynski and Christian (2008) report rules for incorporating conflicting results into the decision-making process about overall strength of evidence (De Los Reyes & Kazdin, 2008). Level of Specificity of Empirically Supported Treatments The issue to be discussed next concerns the unit of analysis of the research evidence. We illustrate by examining an example provided by Horner et al. (2005) in which they assessed the level of support for functional communication training (FCT; Carr & Durand, 1985). Functional communication training is an approach to reducing problem behaviors that teaches an appropriate nonproblematic way for individuals to access reinforcers for the problem behavior that have been identified through functional assessment. Horner and colleagues cited eight published research reports that included 42 participants who had benefited in FCT studies across five research groups. The evidence was sufficient in quantity for it to be concluded that FCT is an empirically supported treatment, exceeding all criteria reviewed earlier except that two more studies would have been needed to reach the number of studies $ 10 by the criteria of Reichow et al. (2008). It might reasonably be asked: “For which population is FCT beneficial?” Perusal of the original papers cited by Horner et al. (2005) shows that 21/42 participants’ data were reported in one of the eight cited studies (Hagopian, Fisher, Sullivan, Acquisto, & LeBlanc, 1998), with the oldest participant being 16 years old, and none reported to have a diagnosis of autism. Thus, applying Horner et al.’s criteria, it cannot be concluded from the studies cited that FCT is an empirically supported treatment for participants with autism or for participants older than 16, regardless of diagnosis. As an aside, eight participants across the other seven studies were reported to have autism, and three participants in total were aged over 20 years; however, the literature on FCT that Horner et al. (2005) included did not appear to have been obtained from a systematic search, so it is possible that there has been sufficient research to show that FCT is an empirically supported treatment for subgroups, and perhaps autism and adults are two of those. TREATMENT GUIDELINES Treatment guidelines specifically recommend ESTs to practitioners and consumers. Alternative descriptors are clinical practice and 18 Overview and Foundational Issues c01 20 April 2012; 12:43:30 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. best practices guidelines (Barlow, Levitt, & Bufka, 1999). The view of APA was that guidelines are not intended to be mandatory, exhaustive, or definitive . . . and are not intended to take precedence over the judgment of psychologists. APA’s official approach to guidelines strongly emphasizes professional judgment in individual patient encounters and is therefore at variance with that of more ardent adherents to evidence-based practice. (Reed et al., 2002, p. 1042) It is apparent that many health-care organizations, insurance companies, and states in the United States interpret the purpose of lists of ESTs and treatment guidelines differently (Gotham, 2006; Reed & Eisman, 2006). They can interpret guidelines as defining what treatments can be offered to patients and, via manualization, exactly how treatment is to be administered, by whom, and for how long. The requirement for manualization allows funders of treatment to specify a standard reimbursement for the treatment provider. Thus, the empirically supported treatment movement was embraced by governments and health-care companies as it was anticipated to be a major contributor to controlling escalating health-care costs. Many practicing psychologists were less enthusiastic about the empirically supported treatment and EBPP movements (see contributions by clinicians in Goodheart et al., 2006; Norcross et al., 2005). General concerns included that requirements to use only ESTs restrict professionalism by reframing psychologists as technicians going by the book mechanically; restricting client choice to effective interventions that have been granted empirically supported treatment status higher than others, only because they, like drugs, are relatively easy to evaluate in the RCT format. Prescription of one-size-fits-all ESTs may further disadvantage minorities, and people with severe and multiple disorders for whom there is scant evidence available. There were also concerns that the acknowledged importance of clinical expertise, such as interpersonal skills to engage the client (child) and significant others (family) in a therapeutic relationship, would be ignored. Contrary to the pronouncements from the APA (2002, 2006), guidelines have been interpreted or developed that “assume the force of law” in prescribing some interventions and proscribing others (Barlow et al., 1999, p. 155). Compulsion of psychologists in practice to follow treatment guidelines has been reported to occur in the United States by giving immunity from malpractice lawsuits to those who use only ESTs, and increasing the vulnerability of those who do not to litigation and increased professional indemnity insurance (Barlow et al., 1999; Reed et al., 2002). Some guidelines, especially those produced by agencies or companies employing psychologists, have been viewed as thinly veiled cost-cutting devices justified with a scientistic gloss. Ethical Requirements For many human service professional organizations, EBP and ESTs have become an ethical requirement. The APA’s Ethical Principles of Psychologists and Code of Conduct document mentions the obligation to use some elements of EBPP; for example, “Psychologists’ work is based upon established scientific and professional knowledge of the discipline” (American Psychological Association, 2010, p. 5). Other professional groups appear to be more prescriptive with regard to EBP. The BACB’s Code for Responsible Conduct, for example, recommends EBP with statements such as, “Behavior analysts rely on scientifically and professionally derived knowledge when making scientific or professional judgments in human service provision” (Behavior Analyst Certification Board, 2004, p. 1). The BACB also require the use of ESTs: a. The behavior analyst always has the responsibility to recommend scientifically Rationale and Standards of Evidence in Evidence-Based Practice 19 c01 20 April 2012; 12:43:30 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. supported most effective treatment procedures. Effective treatment procedures have been validated as having both long-term and short-term benefits to clients and society. b. Clients have a right to effective treatment (i.e., based on the research literature and adapted to the individual client). c. Behavior analysts are responsible for review and appraisal of likely effects of all alternative treatments, including those provided by other disciplines and no intervention. (Behavior Analyst Certification Board, 2004, p. 4) As shown by the previous examples, each of these organizations have variations in how they have included EBP and empirically supported treatment into their codes of conduct and ethical statements; however, they both include the basic tenets of EBP: combining the best research information with clinical knowledge and the preferences of the individuals involved. CHILDREN, ADOLESCENTS, AND EVIDENCE-BASED PRACTICE IN PSYCHOLOGY Despite there being ESTs for a number of disorders and problem behaviors manifesting in children and adolescents, there are more than 500 treatments in use with children (Kazdin, 2008), most of which are unresearched. Chambless and Ollendick (2001) listed 108 ESTs for adults, compared with 37 for children, suggesting that research on therapies for young people is lacking relative to adults. Further research with child and adolescent populations has been prioritized by APA (APA Presidential Task Force on Evidence-Based Practice, 2006). Researching the effects of treatments for children brings special difficulties (Kazdin & Weisz, 1998). Regarding practice, children do not typically selfrefer for mental, psychological, or behavioral disorders, nor are they active seekers of ESTs or preferred interventions. Parents or guardians tend to take those roles, either independently or following recommendations from family, or health or education professionals. Children and youth cannot legally provide informed consent for treatments or for participation in research studies. These are among the developmental, ethical, and legal factors that affect consideration of EBPP with children. Lord et al. (2005) discussed the challenges of acquiring rigorous evidence regarding efficacy of treatments for children with complex, potentially chronic behavioral/psychological disorders. Contributors to the article were researchers from multiple disciplines assembled by the National Institutes of Health in 2002. They wrote about autism spectrum disorders specifically, but acknowledged that the issues may have relevance to other child and youth problems. Parents may be unwilling to consent to randomization studies in case their child is assigned to what parents perceive to be a less preferred treatment alternative, particularly when the intervention is long term and early intervention is, or is perceived to be, critical, such as early intensive behavioral intervention for pervasive developmental disorders. Lord et al. (2005) noted that ethical concerns may prohibit RCTs of promising interventions when randomization to no treatment or treatment of unknown effects is required by the evaluation protocol. Additional factors that reduce the internal validity of group comparison studies of psychosocial interventions include that parental blindness to the intervention allocated to their children is nigh on impossible, diffusion of treatment through parent support groups is likely, parents may choose to withdraw their children from no treatment or treatment as usual groups and obtain the experimental intervention or an approximation to it from outside the study, and children with severe disorders will often be subjected to multiple interventions of unknown benefit, provided with varying 20 Overview and Foundational Issues c01 20 April 2012; 12:43:30 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. fidelity that may interact with one another to produce uninterpretable beneficial, neutral, or adverse effects (Smith & Antolovich, 2000). Parents seek out information on their child’s disorder and intervention recommendations through Internet or parent organizations, sometimes accepting advice from professionals. Mackintosh, Meyers, and Goin-Kochel (2006) received 498 responses to a Web-based survey of parents with children with autism spectrum disorders and found that the most oft-cited sources of information were books (88%), Web pages (86%), other parents (72%), and autism newsletters (69%). Lagging somewhat as sources of advice were professionals other than educators or physicians (57%). Physicians, education professionals, and family members were cited as sources of information by fewer than half the parents who responded to the survey. Multiple fad treatments have been recommended for childhood onset disorders, and their adoption by families and some professionals, including psychologists, wastes resources and time that could have been spent profitably by employing ESTs (Jacobson, Foxx, & Mulick, 2005). Although some of our examples of supported and unsupported treatments have related to children with autism spectrum disorders (Romanczyk, Arnstein, Soorya, & Gillis, 2003), the problem of treatment selection uninformed by research affects children with other difficulties and their families. Some interventions for children with developmental and other disabilities have been found ineffective or harmful (Jacobson et al., 2005), and the same occurs for children with ADHD (Waschbusch & Hill, 2003). (See also Chapter 2 of this volume for a further discussion of this point by Waschbush, Fabiano, and Pelham.) We believe that clinical psychologists working with young people ought to have a professional ethical obligation to inform themselves and others about empirically unsupportable treatments as well as ESTs for their clients. LIMITATIONS OF THE EVIDENCE BASE REGARDING EVIDENCE-BASED PRACTICE IN PSYCHOLOGY There is a relatively large body of evidence concerning efficacy of treatments (Kazdin & Weisz, 1998, 2003), but far less on treatment utility, effectiveness, and efficiency. There is evidence that the utility of some efficacious treatments has been demonstrated, but further study is needed before the general statements can be made about the similarity or difference between outcomes from controlled research and clinical practice (Barlow et al., 1999; Hunsley, 2007). There is less evidence concerning dimensions of clinical expertise and client characteristics, culture, and preferences that are relevant to beneficial treatment outcomes (Kazdin, 2008). Employment of qualitative research methods may help us to understand clients’ experiences of psychological treatments. The APA Task Force has suggested that clinical expertise is made up of at least eight components, including assessment and treatment planning, delivery, interpersonal skills, self-reflection, scientific skills in evaluating research, awareness of individual and social factors, the ability to seek additional resources where necessary, and having a convincing rationale for treatment strategies (APA Presidential Task Force on Evidence-Based Practice, 2006). Qualitative methods may provide evidence regarding clinical expertise also. Improvement of two-way communication between psychologists who are primarily researchers and those who identify more as practitioners would assist dissemination of ESTs, collaboration in clinical utility studies of efficacious treatments, and facilitate research into clinical expertise and barriers to the adoption of ESTs by clinicians (Kazdin, 2008). Lilienfeld (2005) and McLennan, Wathen, MacMillan, and Lavis (2006) recommended further research on interventions in child Rationale and Standards of Evidence in Evidence-Based Practice 21 c01 20 April 2012; 12:43:31 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. psychopathology on two fronts: (1) increased research on promising or new but plausible interventions; and (2) on research to combat questionable, but potentially harmful, interventions. Attention is needed to increase training of clinical psychologists in ESTs and cessation of training to use treatments that have been shown to be harmful, ineffective, or less effective. Decreasing the demand and use of treatments that have not been evaluated scientifically or have been found ineffective or harmful may be another strategy for helping clinical psychologists orient more to treatments that do work (i.e., ESTs). Woody, Weisz, and McLean (2005) report that APA accredited clinical psychologist training programs taught and supervised interns in fewer ESTs in 2003 than they had in 1993. The training of clinical psychologists should include sufficient study of research methodology so that career-long learning, and contribution to research in practice concerning ESTs can be enhanced (Bauer, 2007; Kazdin, 2008). Considering the APA definition of EBPP, trainees need supervised practice in incorporating clients’ preferences, values, and cultural considerations as well as to develop clinical expertise (Collins, Leffingwell, & Belar, 2007). Further training for universitybased psychologist training faculty should be studied. Psychologists in practice may wish or feel forced to adapt to employing EBPP, but willingness or compulsion to do so is not the same as becoming immediately competent to use an empirically supported treatment effectively. The typical workshop format for introducing new techniques is as ineffective for professionals (Gotham, 2006), as it is for direct care staff (Reid, 2004). Skill-based training can be effective when trainees practice an intervention method in the natural clinical environment with differential feedback from the trainer on their performance of the skill. This should occur after workshops that introduce the rationale and method, and include in vivo or videotaped demonstration of the skill by the trainer (i.e., modeling). Frequent follow-up observations by the trainer, again with objective feedback to the trainee, can facilitate maintenance of the newly acquired skills (Reid, 2004). Gotham (2006) identified barriers to implementation of EBPP, and provided an example of how to implement an empirically supported treatment statewide despite obstacles. McCabe (2004) wrote quite optimistically for clinicians about the challenges of EBPP, offering advice in a step-by-step form to psychologists. Reorienting and training clinicians is an area for further clinicianresearcher collaboration that requires emphasis. We have not included discussion of an area of EBPP that has, to date, received less attention than evidence-based treatment, which is “evidence-based assessment” (Kazdin, 2005; Mash & Hunsley, 2005). An initial assessment with clinical utility will identify what is the disorder or problem behavior so that it points the psychologist in the right direction for identifying the range of available ESTs for this client. Evidence-based assessment includes identifying reliable and valid ongoing measures that show the effects of an intervention with the individual client (Kazdin, 2005). Typically, empirically supported treatment reviews identify treatments for DSM-type classifications, the nosological approach; however, an idiographic functional approach to assessment may lead to better problem-EST match (Sturmey, 2007). It was mentioned earlier that some professional groups are concerned at the lack of research on the outcomes of their assessment and treatment methods. Because clinical and other psychologists have extensive training in research methods, we can assist other professions to assess the evidence for their intervention methods. Interdisciplinary collaborations also may help elucidate the interaction effects of behavioral or psychological ESTs with interventions with presently unknown effects delivered by other professions. 22 Overview and Foundational Issues c01 20 April 2012; 12:43:31 Hersen, Michel, and Peter Sturmey. Handbook of Evidence-Based Practice in Clinical Psychology, Child and Adolescent Disorders, John Wiley & Sons, Incorporated, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=817355. Created from ashford-ebooks on 2017-11-07 11:30:44. Copyright © 2012. John Wiley & Sons, Incorporated. All rights reserved. CONCLUDING REMARKS The issues concerning evidence-based clinical practice of psychology for children and adolescents are arguably even more complex than for adult clients. All readers would welcome the day when, for any referred psychological, behavioral, or mental problem, psychologists with clinical expertise were able to offer a wide range of effective and safe interventions. Then the young person, with more or less help from those who care for them, could select the effective treatment that suited their culture, preferences, and resources. The literature reviewed for this chapter suggests that, generally speaking, the health care, including clinical psychology, and education professions are starting on the road to addressing the aspirational targets of the EBP movement. The lack of unanimous agreement within professions that the evidence-based practice movement is desirable and what constitutes evidence for beneficial interventions is unsurprising; however, with more evidence for safe, acceptable, and effective interventions such as contained in the present volume and more education for consumers, professionals, and the public, eventually the naysayers and peddlers of unsupportable treatments may find they have no raison d’eˆtre (and also no income!).