Document 1: Program Evaluation
Table of Contents
Program Evaluation
Please no plagiarism and make sure you are able to access all resources on your own before you bid. Main references come from Balkin, R. S., & Kleist, D. M. (2017) and/or American Psychological Association (2014). Assignments should adhere to graduate-level writing and be free from writing errors. I have also attached the resources given to complete the assignment. Please follow the instructions to get full credit. I need this completed by 10/26/19 at 7pm. You will complete the program evaluation worksheet attached.
Assignment – Week 9
In this case study, the counselor, Steven, has collected data from his clients after participation in a psychoeducation therapy group. Steven wants to know whether his program effectively met the needs of the clients. He is interested in learning whether the clients met their program outcomes and whether the program evaluation data supports his desire to expand the program.
For this Assignment, you will review the data and determine whether the service contributed to client success.
To Prepare
· Review the Program Evaluation audio recording and the Program Evaluation Worksheet found in the Learning Resources and consider the requirements for this Assignment.
· You may work independently or form small groups of no more than three people. If you choose to work in small groups, you may use the Blackboard Collaborate Ultra “Live Meetings” tool found in the left-hand navigation of the classroom to collaborate with your group in a synchronous way. Once you access the “Live Meetings” tool, use the “Sessions Help” feature in the top right-hand corner to guide you through setting up your session with your group if you choose to do so.
· Review the Program Evaluation Worksheet and consider the requirements for this Assignment. Specifically:
o Review the case study.
o Analyze the Program Evaluation dataset to determine whether the counseling intervention worked or did not work.
o Consider a recommendation you might make regarding the future of the program and why.
Assignment
Imagine you are a task force or part of a task force charged with evaluating the effectiveness of a new counseling program. Your job is to complete a Program Evaluation Worksheet that will help you determine the effectiveness of the program.
- As an individual part of a task force or in your small group task force, complete the Program Evaluation Worksheet.
Astramovich, R. L., & Coker, J. K. (2007 ). Program evaluation: The accountability bridge model for counselors. Journal of Counseling & Development, 85, 162–172.
Note: You will access this article from the Walden Library databases.
Balkin, R. S., & Kleist, D. M. (2017). Counseling research: A practitioner-scholar approach. Alexandria, VA: American Counseling Association.
- Chapter 13, “Program Evaluation”
Note: You will access this article from the Walden Library databases.
Lamis, D. A., Underwood, M., & D’Amore, N. (2017). Outcomes of a suicide prevention gatekeeper training program among school personnel. Crisis, 38(2), 89-99. doi: 10.1027/0227-5910/a000414
Note: You will access this article from the Walden Library databases.
Note: Select the “Program Evaluation” tab on this web page to access the Program Evaluation Tutorial.
Document: Program Evaluation Worksheet
Required Media
Laureate Education (Producer). (2017). Program evaluation [Video file]. Baltimore, MD: Author.
Note: The approximate length of this media piece is 1 minute.
Accessible player –Downloads– Download Video w/CC Download Audio Download Transcript
Laureate Education (Producer). (2017). Program evaluation [Video file]. Baltimore, MD: Author.
Note: The approximate length of this media piece is 18 minutes.
Accessible player –Downloads– Download Video w/CC Download Audio Download Transcript
Credit: Provided courtesy of the Laureate International Network of Universities.
–Downloads– Download Video w/CC Download Audio Download Transcript
Here is some food for thought!
A while back I wrote a book chapter on assessment in counseling and included information about program evaluations. It’s really interesting because program evaluation can be considered research and assessment and just plain program evaluation
Here is a case study that was included. This may help you think about some of the aspects of program evaluation:
The agency needs to “prove its worth”!
You were recently hired at a non-profit counseling center as a quality assurance counselor and as part of your role it is your responsibility to conduct a program evaluation. You are vaguely aware that you need to collect some data but where to begin is the question. The goal of program evaluation is to create a systematic assessment which will work to improve the quality of services or the programs of the agency. The first step involves determining your goal and then creating a plan to collect objective and subjective information. There are several questions you must ask as you create the program evaluation. Who are your stakeholders? Are you focusing on specific programs with the agency? Are you going to utilize a formative or summative evaluation? Would using test results be helpful? Will you utilize surveys, interviews, observations, or focus groups? Questions such as these guide the evaluation with the goal of accountability for the counseling profession.
References:
Foster, L.H. (2020). Assessment practices in counseling. In D. Capuzzi and D. Gross (Eds.), Introduction to the counseling profession (8th ed.). Boston: Allyn and Bacon.
Seven steps to guide a focused evaluation
1. Solicit input from stakeholders and identify program goals;
2. Design a plan for examining program implementation to achieve program goals;
3. Design a plan for evaluating program progress toward program goals ;
4. Create a consolidated data collection plan to assess progress towards program goals ;
5. Plan the data analysis to examine program goals ;
6. Estimate the financial and time costs of the evaluation needed to meet program goals;
7. Come to a final agreement about services, costs, and responsibilities with relevant stakeholders needed to reach program goals.
Do these seven steps fit in with these 4 steps?
COUN 6626: Research Methodology and Program Evaluation
| Name of Student: | Names of Group Members: |
Read the following case study, review the data sets, and answer the subsequent questions.
Case Study
Steven is a Licensed Professional Counselor who works for a community mental health center providing a psychoeducation group on stress management and anxiety reduction. The goals of the program are for members to increase coping strategies for daily life stressors and reduce their current level of anxiety. Steven holds weekly psychotherapy group meetings on Wednesdays at noon in a conference room at the mental health center. The therapy group is a closed group that runs continuously in 8-week intervals. Clients are referred to the group by therapists, physicians, and clergy. Inclusion criteria to participate in the group include formal referral and a Beck Anxiety Inventory Score of 22 or higher.
In an effort to determine whether the services are meeting the needs of the clients, Steven has conducted a Program Evaluation and collected pre- and post-test program data from three sets of group participants. His findings will be used to determine renewal of his program. Analyze the following quantitative and qualitative data and determine whether enough evidence exists to rule the program as effective and make one recommendation regarding the future of the program.
Pre-Test Data
Beck Anxiety Inventory (BAI) Pre-Intervention Scores
| Group 1 | Group 2 | Group 3 | |
| 1 | 25 | 22 | 31 |
| 2 | 24 | 24 | 30 |
| 3 | 22 | 24 | 30 |
| 4 | 22 | 27 | 29 |
| 5 | 27 | 22 | 28 |
| 6 | 28 | 33 | 24 |
| 7 | 25 | 36 | 24 |
| 8 | 25 | 33 | 22 |
| 9 | 29 | 32 | 27 |
| 10 | 24 | 27 | 28 |
Post-Test Data
Beck Anxiety Inventory (BAI) Post Intervention Scores
| Group 1 | Group 2 | Group 3 | |
| 1 | 25 | 21 | 30 |
| 2 | 16 | 20 | 20 |
| 3 | 21 | 22 | 19 |
| 4 | 20 | 18 | 24 |
| 5 | 21 | 20 | 22 |
| 6 | 19 | 23 | 21 |
| 7 | 22 | 34 | 21 |
| 8 | 24 | 33 | 20 |
| 9 | 28 | 30 | 20 |
| 10 | 24 | 22 | 18 |
Post-Intervention Qualitative Survey Responses
Question: Do you feel that participating in this group helped you feel better prepared to cope with daily life stressors?
| Group 1 | Group 2 | Group 3 | |
| 1 | Y | Y | Y |
| 2 | N | N | Y |
| 3 | Y | Y | Y |
| 4 | Y | Y | Y |
| 5 | Y | Y | Y |
| 6 | N | Y | N |
| 7 | N | Y | Y |
| 8 | Y | Y | N |
| 9 | Y | N | N |
| 10 | Y | Y | Y |
| What is the purpose of Steven’s Program Evaluation? | ||||||
| What is the population of interest and who are the stakeholders for this Program Evaluation? | ||||||
| Compute the mean BAI scores pre- and post-intervention. | Pre-Intervention | Post-Intervention | ||||
| G 1 | G2 | G3 | G1 | G2 | G3 | |
| Total Pre-Test Mean | Total Post-Test Mean | |||||
| What is the median BAI score pre- and post-intervention? | Pre-Intervention Median | Post-Intervention Median | ||||
| G1 | G2 | G3 | G1 | G2 | G3 | |
| Total Pre Test Median | Total Post Test Median | |||||
| 1. Quantitative: According to the scoring criteria for the BAI, a score of 21 or below indicates very low anxiety. What percentage of each group’s scores falls below that clinical cutoff?2. Qualitative: Based on the qualitative responses, what percentage of the participants articulated a feeling of improvement? | Group 1 | Group 2 | Group 3 | |||
| Pre | Post | Pre | Post | Pre | Post | |
| 0% | 0% | 0% | ||||
| Mean of Total Percentage of Improvement for all 3 Groups | ||||||
| 1 | 2 | 3 | ||||
| What trends to you observe when you compare the quantitative data with the qualitative data? | ||||||
| Based on the information provided, was the intervention effective for reducing anxiety? Why or why not? What was the most critical data that informed your decision? | ||||||
| Describe one recommendation that you would make for the assessing program effectiveness going forward and explain your rationale.(An improvement for the evaluation, not the program.) | ||||||
| Based on your data analysis, describe one recommendation that you would make for the design of the intervention program going forward and explain your rationale.(An improvement for the therapy group, not the program.) |
Assignment – Week 9
Top of Form
In this case study, the counselor, Steven, has collected data from his clients after participation in a psychoeducation therapy group. Steven wants to know whether his program effectively met the needs of the clients. He is interested in learning whether the clients met their program outcomes and whether the program evaluation data supports his desire to expand the program.
For this Assignment, you will review the data and determine whether the service contributed to client success.
To Prepare
· Review the Program Evaluation audio recording and the Program Evaluation Worksheet found in the Learning Resources and consider the requirements for this Assignment.
· You may work independently or form small groups of no more than three people. If you choose to work in small groups, you may use the Blackboard Collaborate Ultra “Live Meetings” tool found in the left-hand navigation of the classroom to collaborate with your group in a synchronous way. Once you access the “Live Meetings” tool, use the “Sessions Help” feature in the top right-hand corner to guide you through setting up your session with your group if you choose to do so.
· Review the Program Evaluation Worksheet and consider the requirements for this Assignment. Specifically:
· Review the case study.
· Analyze the Program Evaluation dataset to determine whether the counseling intervention worked or did not work.
· Consider a recommendation you might make regarding the future of the program and why.
Assignment
Imagine you are a task force or part of a task force charged with evaluating the effectiveness of a new counseling program. Your job is to complete a Program Evaluation Worksheet that will help you determine the effectiveness of the program.
· As an individual part of a task force or in your small group task force, complete the Program Evaluation Worksheet.
Required Resources
Astramovich, R. L., & Coker, J. K. (2007 ). Program evaluation: The accountability bridge model for counselors. Journal of Counseling & Development, 85, 162–172.
Note: You will access this article from the Walden Library databases.
Balkin, R. S., & Kleist, D. M. (2017). Counseling research: A practitioner-scholar approach. Alexandria, VA: American Counseling Association.
· Chapter 13, “Program Evaluation”
Note: You will access this article from the Walden Library databases.
Lamis, D. A., Underwood, M., & D’Amore, N. (2017). Outcomes of a suicide prevention gatekeeper training program among school personnel. Crisis, 38(2), 89-99. doi: 10.1027/0227-5910/a000414
Note: You will access this article from the Walden Library databases.
Neilson, T. (2015). Practice-based research: Meeting the demands of program evaluation through the single-case design. Journal of Mental Health Counseling, 37(4), 364–376. https://doi.org/10.17744/mehc.37.4.07
Walden University, Center for Research Quality. (n.d.-b). Research resources: Research design & analysis: Program evaluation tutorial . Retrieved from http://academicguides.waldenu.edu/researchcenter/resources/Design
Note: Select the “Program Evaluation” tab on this web page to access the Program Evaluation Tutorial.
Document: Program Evaluation Worksheet
Required Media
Laureate Education (Producer). (2017). Program evaluation [Video file]. Baltimore, MD: Author.
Note: The approximate length of this media piece is 1 minute.
Accessible player
Laureate Education (Producer). (2017). Program evaluation [Video file]. Baltimore, MD: Author.
Note: The approximate length of this media piece is 18 minutes.
Accessible player
Credit: Provided courtesy of the Laureate International Network of Universities.
Bottom of Form
Week 9 means Program Evaluation~
Program evaluation is a systematic assessment of a program or intervention that is designed to evaluate and improve programs that are offered to whole communities. Programs may be offered in agencies, schools, work settings and can include a wide variety of mental health issues. Program evaluation helps to find out what works and what does not work and provide future direction for programs.
Be sure to check out the Program Evaluation Worksheet which is found in the weekly resources.
Research Trends
Outcomes of a Suicide Prevention Gatekeeper Training Program Among School Personnel Dorian A. Lamis1, Maureen Underwood2, and Nicole D’Amore2
1Department of Psychiatry and Behavioral Sciences, Emory School of Medicine, Atlanta, GA, USA 2Society for the Prevention of Teen Suicide, Freehold, NJ, USA
Abstract. Background: Gatekeeper suicide prevention programs train staff to increase the identification and referral of suicidal individuals to the appropriate resources. Aims: We evaluated Act on FACTS: Making Educators Partners in Youth Suicide Prevention (MEP), which is an online training program designed to enhance the knowledge of suicide risk factors and warning signs as well as improve participants’ attitudes and self-efficacy/confidence. Method: School personnel (N = 700) completed a survey administered before and immediately after the training to assess gains in training outcomes and to evaluate participants’ satisfaction with the training. Results: Results indicated that MEP participants demonstrated significant increases in suicide knowledge, attitudes, and self-efficacy. Moreover, exploratory analyses revealed moderating effects of professional role on pre-/posttest changes in self-efficacy, but not suicide knowledge or attitudes. Specifically, guidance counse- lors demonstrated significantly smaller increases in self-efficacy/confidence compared with teachers and classroom aids, whereas teachers demonstrated significantly larger increases in self-efficacy/confidence compared with administrators. The majority of school personnel who completed the MEP program were satisfied with the training content and experience. Conclusion: Although the current findings are promising, more rigorous evaluations employing randomized controlled research designs are warranted to adequately determine the effectiveness of the MEP program.
Keywords: suicide prevention, gatekeeper training, youth, school personnel
Suicide is the second leading cause of death among youth 10–24 years of age, and the prevalence of suicide significantly increases from childhood to the end of ad- olescence (Centers for Disease Control and Prevention [CDC], 2015). Moreover, according to the 2013 Youth Risk Behavior Surveillance survey (Kann et al., 2014), 17% of high school students seriously considered su- icide in the previous 12 months, 13.6% made a suicide plan, and 8% attempted suicide. However, the majori- ty of youths with mental health problems do not receive services (Carlton & Deane, 2000; Gould, Munfakh, Lu- bell, Kleinman, & Parker, 2002). Thus, prevention efforts identifying, assessing, and treating suicidal adolescents are warranted. As such, federal organizations, such as the Substance Abuse and Mental Health Services Adminis- tration, have continually invested in preventing suicide among youth. Moreover, the National Action Alliance for Suicide Prevention’s Research Prioritization Task Force (RPTF, 2014) recently revised and implemented the Na- tional Strategy for Suicide Prevention through best-prac- tice recommendations and a prioritized research agenda to obtain the goal of reducing suicides by 20% in 5 years and by 40% in 10 years.
In an attempt to reduce youth suicide attempts and deaths, a number of effective strategies have been devel- oped, implemented, and evaluated. The goals of most suicide prevention programs are to reduce risk factors, enhance protective factors, and increase the knowledge, skills, and confidence of individuals who may encoun- ter at-risk youth in order to identify and refer potentially suicidal adolescents (Bean & Baber, 2011; Cusimano & Sameem, 2011; York et al., 2013). Notably, the theory of planned behavior (Ajzen, 1985; Ajzen & Fishbein, 2008; Armitage & Conner, 2001) posits that human behavior is guided by (a) beliefs about likely outcomes and evaluations of behaviors (i.e., behavioral beliefs), (b) beliefs about nor- mative expectations of others and motivation to comply (i.e., normative beliefs), and (c) beliefs about the capabil- ity of factors to facilitate or impede behavior performance (i.e., control beliefs). Thus, increasing awareness of facts about suicide, risk factors, and prevention strategies may enhance knowledge, which in turn, will boost confidence in suicide prevention skills through behavioral, norma- tive, and control beliefs (Smith, Silva, Covington, & Join- er, 2014). Consequently, in keeping with the theory of planned behavior, prevention efforts (e.g., identification,
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
.
D. A. Lamis et al.: Outcomes of a Suicide Gatekeeper Training Program90
© 2016 Hogrefe PublishingCrisis (2017), 38(2), 89–99
risk assessment, referrals) will improve and suicidal be- haviors will be reduced owing to this increase in knowl- edge and confidence in skills.
Gatekeeper training is one such programmatic strategy designed to enhance early identification of youth at high risk for suicide and to facilitate timely mental health refer- rals (Condron et al., 2015; Gould & Kramer, 2001; Isaac et al., 2009). Specifically, these programs train staff to in- crease their knowledge of risk factors and warning signs, which in turn, increases confidence and likelihood that personnel will effectively identify and refer suicidal youth to the appropriate resources (Indelicato, Mirsu-Paun, & Griffin, 2011; Rodi et al., 2012; Schilling, Lawless, Bu- chanan, & Aseltine, 2014). In a systematic review con- ducted by Mann and colleagues (2005), the researchers concluded that gatekeeper training programs are one of the most promising strategies to effectively prepare those who are in contact with young people (e.g., school person- nel) to understand the risk and protective factors associat- ed with suicide, to identify at-risk youth, to be aware of ap- propriate resources, and to make referrals when necessary. Gatekeeper training also decreases stigma by highlighting the importance of increasing help-seeking behaviors and raising awareness of effective mental health treatment (Bean & Baber, 2011).
Moreover, recent studies have provided evidence sup- porting the feasibility of implementing and delivering gatekeeper training programs using web-based media (Lancaster et al., 2014; Sueki & Ito, 2015), which decreas- es training costs, improves administration flexibility, and increases the control that the learner has with regard to the training process (Long, DuBois, & Faley, 2008). As such, a recently developed and implemented online gatekeeper training suicide prevention program, Act on FACTS: Mak- ing Educators Partners in Youth Suicide Prevention (MEP; http://www.sptsuniversity.org), was designed to meet the general awareness and knowledge needs of school facul- ty and staff and its content reflects current knowledge in the field of youth suicide prevention from the perspective of school personnel. Rather than adopting a mental health perspective or medical model of youth suicide prevention, the MEP program focuses on addressing prevention and creating a safe learning environment for students.
Listed on the Suicide Prevention Resource Center’s (SPRC) Best Practice registry, MEP’s program objectives include (a) increasing awareness and understanding of youth suicide, (b) increasing recognition of warning signs, and (a) improving the confidence of school personnel to provide an effective initial response and refer to appro- priate resources. The online training program allows par- ticipants to complete the modules at their own pace, with embedded interactive activities that reinforce content. Specifically, the MEP program content addresses benefits
of suicide-awareness training for the entire school com- munity; relevant information about youth suicide; warn- ing signs; risk factors and protective buffers; the educator’s role in identifying suicidal risk in students and making appropriate referrals; interaction strategies with students; and, resources. Designed in a series of five modules, MEP addresses the critical but limited responsibilities of educa- tors in the process of identification and referral of poten- tially suicidal youth by focusing on the practical realities and challenges inherent in the school setting through a variety of training formats, which include lecture, question and answer, and virtual vignettes. Experts in mental health and suicide prevention provide informed commentary and survivors of suicide speak eloquently about the important role that school-based suicide awareness programs can play in the prevention process. The training also includes the ability to leave feedback on the Society for the Preven- tion of Teen Suicide (SPTS) website (http://www.sptsusa. org).
Given that suicidal behaviors occur infrequently, it is difficult to document the impact of prevention efforts on suicide (Mann & Currier, 2011). However, evaluating sui- cide prevention programs is critical. When considering the effectiveness of any particular study or training program, it is necessary to evaluate the evidence presented in support of the program. Moreover, researchers (e.g., Musal et al., 2008) have suggested that program evaluations empha- size both educational processes and outcomes. Kirkpat- rick’s (1959; 1996) four-level model is the most widely used evaluation framework, which includes four distinct levels of evaluation: reaction, learning, behavior, and re- sults. The first level, reaction, measures participants’ per- ceptions of program experiences; the second level, learn- ing, measures the impact of training on knowledge, skills, and attitudes (Kirkpatrick, 1996). The third level, behavior, examines the extent to which training transfers to on-the- job behavior; the fourth level, results, is a measure of the final outcome of training, such as improved well-being or reduced psychological symptoms. Given that Kirkpatrick suggests beginning evaluation with the first two levels, we evaluate the MEPs program according to Kirkpatrick’s levels of learning (knowledge, skills, attitudes, confidence) and reaction (satisfaction).
Accordingly, in the current study, we evaluated quan- titative data collected from participants who completed the MEP online training program. Specifically, we as- sessed participants’ changes in knowledge, attitudes, and self-efficacy/confidence, all of which are important deter- minants of behavior (Armitage & Conner, 2001) and es- sential aspects of program evaluation (Ingvarson, Meiers, & Beavis, 2005). Knowledge refers to content taught by the training, attitudes describe views toward identifica- tion and help-seeking behaviors, and self-efficacy/confi-
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
.
D. A. Lamis et al.: Outcomes of a Suicide Gatekeeper Training Program 91
© 2016 Hogrefe Publishing Crisis (2017), 38(2), 89–99
dence refers to one’s belief in his or her ability to identify, intervene, and refer suicidal youth (Thompson, Eggert, Randell, & Pike, 2001). Although the primary target audi- ence for the MEP training was classroom-based teachers, individuals in other professional roles participated in the training. Previous research (Reis & Cornell, 2008; Smith et al., 2014; Wyman et al., 2008) has found that school counselors and administrators demonstrate higher levels of suicide knowledge and self-efficacy/confidence than school personnel in other job roles. Moreover, studies have shown that age (Tompkins, Witt, & Abriabesh, 2010), gen- der (Hamilton & Klimes-Dougan, 2015), and prior suicide training (King & Smith, 2000; Tompkins, Witt, & Abria- besh, 2009) have influenced suicide training gatekeeper outcomes. As a result, these variables were included as co- variates in all analyses evaluating training effects.
On the basis of existing literature, we expected that guidance counselors would score significantly higher on suicide knowledge and self-efficacy/confidence at base- line than teachers, administrators, and classroom aids (Hypothesis 1). We did not anticipate any significant differ- ences among professional role groups with regards to atti- tudes. Moreover, we hypothesized that all participants who complete the online MEP program would report significant increases in (a) knowledge, (b) attitudes, and (c) self-effi- cacy/confidence (Hypothesis 2). In addition to testing the hypothesized main effects of MEP, we examined the par- ticipants’ professional roles as a potential moderating fac- tor of training effects on suicide knowledge, attitudes, and self-efficacy/confidence. No specific a priori hypotheses were made regarding differences in pre-/posttest chang- es among the professional roles. Finally, we hypothesized that participants would report satisfaction with the MEP training content and experience (Hypothesis 3).
Method
Participants
Many states require educators to complete a minimum number of training hours in youth suicide prevention for a specific duration of time for continuing education cred- its and professional development. For example, in New Jersey, where the SPTS was founded, the state mandate requires 2 hrs of youth suicide prevention training every 5 years. Given that the SPTS is included on a list of approved providers of youth suicide prevention training, school per- sonnel who participated in the free 2-hr online training of- ten did so to fulfill a state requirement for professional de- velopment hours. From February to June 2015, data were collected from 700 school personnel between the ages of
18 and 70 years (M = 40.24, SD = 12.03), all of whom completed the MEP training. Of the participants, 79.6% (n = 557) were female and 20.4% (n = 143) were male. The majority described their race/ethnicity as Caucasian (n = 609, 87.0%), followed by Hispanic/Latino (n = 36, 5.1%), African American (n = 30, 4.3%), Asian American/ Pacific Islander (n = 5, 0.7%), and an additional 2.9% (n = 20) of the sample indicated other for race/ethnicity. The school personnel reported that they were current- ly working in middle or junior high school (grades 6–8; n = 420; 60.0%) or in high school (grades 9–12; n = 280; 40.0%), with the majority being teachers (n = 620; 88.6%), followed by administrators (n = 35; 5.0%), classroom aids (n = 26; 3.7%), and guidance counselors (n = 19; 2.7%).
The majority of respondents (n = 565; 80.7%) report- ed that they had not worked with a youth who ended his or her life by suicide, 12.0% (n = 84) indicated that they had worked with at least one person who had ended his or her life by suicide, and 6.6% (n = 46) did not know. Sim- ilarly, 70.3% (n = 492) reported they had never thought a student’s behavior indicated he or she was considering suicide within the past year; 23.3% (n =163) of school per- sonnel reported thinking one or two times that a student was considering suicide, 4.0% (n = 28) reported thinking three to five times that a student was considering suicide, while 0.9% (n = 6) reported six or more times, and 1.6% (n = 11) did not wish to answer. Participants were asked whether they had or had not received any past suicide awareness or prevention training and more specifically the MEP training. In all, 38% (n = 267) self-reported that they had any suicide prevention training, whereas 6.9% (n = 48) reported that they had previously participated in the MEP training program.
Procedure
Data collection assessing suicide knowledge, attitudes, and self-efficacy/confidence was conducted through a sur- vey administered before and immediately after the train- ing. Moreover, items assessing relevant demographic (e.g., age, gender) and background (e.g., prior training, profes- sional role) characteristics were administered with the pre- tests and training program satisfaction evaluations accom- panied the posttests. Participants were asked to answer the items truthfully and were informed that the questionnaires would remain confidential. Institutional Review Board ap- proval was obtained and participants completed the survey online via a secure website.
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
.
D. A. Lamis et al.: Outcomes of a Suicide Gatekeeper Training Program92
© 2016 Hogrefe PublishingCrisis (2017), 38(2), 89–99
Measures
Knowledge To assess knowledge, quantitative pre- and posttests were administered, with higher scores indicating more content knowledge. Specifically, 15 items (eight multiple choice, seven true/false items) were used to assess knowledge of key curriculum components, with each item being scored 0 if incorrect and 1 for correct, and total scores ranging from 0 to 15. Sample items include: “What is one of the most significant protective factor for suicide in youth?” and “Risk factors and warning signs for suicide are essen- tially the same things (True/False).”
Attitudes and Self-Efficacy/Confidence The mean of two Likert scale items was computed to as- sess attitudes (1 = strongly disagree, 2 = disagree, 3 = nei- ther disagree nor agree, 4 = agree, 5 = strongly agree), with question one being reverse scored: (1) “If I think a student is at risk for suicide, there is nothing I can do to help” and (2) “School teachers and staff should be responsible for identifying behaviors that increase the risk of suicide in students.” Self-efficacy/confidence was evaluated by cal- culating the average of seven questions, four of which were assessed on the same Likert scale (1 = strongly disagree, 2 = disagree, 3 = neither disagree nor agree, 4 = agree, 5 = strong- ly agree): (1) “I can recognize warning signs that a student is distressed and possibly at risk for suicide,” (2) “I know how to respond when I am concerned about a student who may be suicidal,” (3) “I can recognize students who may be at risk for suicide by the way they behave,” and (4) “I have the necessary knowledge and skills to intervene with students at risk for suicide.” Two Likert scale questions: (5) “How prepared do you feel you are to identify a student at risk for suicide?” and (6) “How prepared do you feel you are to refer a student at risk for suicide to the proper resource?” were assessed using the following response options: 1 = not at all prepared, 2 = minimally prepared, 3 = adequate- ly prepared, 4 = well prepared, and 5 = extremely prepared. The seventh question was: “Overall, how confident are you in your ability to identify and refer a student at risk for su- icide?” (1 = not at all confident, 2 = minimally confident, 3 = moderately confident, 4 = quite confident, 5 = extreme-
ly confident). The internal consistency reliability estimates for attitudes at pre- and posttest were 0.73 and 0.75, re- spectively; the reliability coefficient was 0.89 at both the pre- and posttest for self-efficacy/confidence.
Satisfaction With Training Participants were asked to complete an evaluation at the conclusion of the MEP program to measure their satisfac- tion with the training content and experience. Specifically, a 5-item self-report instrument using 5-point Likert scales was administered. Questions included: (1) “How would you rate the overall learning experience of this course?” (poor to very good), (2) “How well did this training meet its objectives?” (poor to very good), (3) “I found this training to be culturally competent for a school setting” (strongly disa- gree to strongly agree), (4) “How useful was the training con- tent to your current job responsibilities?” (not at all useful to extremely useful), and (5) “How likely are you to recom- mend this training to others?” (very unlikely to very likely). The internal consistency reliability estimate was 0.87 for the scale assessing satisfaction with the MEP training.
Results
An analysis of variance (ANOVA) was conducted to ex- amine potential differences in baseline variables of inter- est among school personnel in various professional roles. There was a significant main effect of professional role on pretest scores of suicide knowledge, F(3,696) = 8.73, p < .001, of attitudes, F(3,696) = 3.42, p = .017, and of self-efficacy/confidence, F(3,696) = 12.12, p < .001 (see Table 1). Post hoc tests with the Bonferroni correction for multiple comparisons revealed that the mean score on su- icide knowledge for guidance counselors was significantly higher than the mean scores for teachers and classroom aids; however, there was no difference found between guidance counselors and administrators. Moreover, re- sults indicated that administrators scored significantly higher than teachers on the measure assessing pretest at- titudes toward identification and help-seeking behaviors. With regard to self-efficacy/confidence, guidance coun-
Table 1. MEP participants’ average baseline scores by professional role
Teachers Administrators Classroom aids Guidance counselors
Variable (n = 620) (n = 35) (n = 26) (n = 19)
Suicide knowledge 8.22(1.71)a 9.00(2.12)ab 8.39(2.53)a 10.22(2.01)b
Attitudes 4.05(0.63)a 4.34(0.50)b 4.14(0.69)ab 4.26(0.45)ab
Self-Efficacy/Confidence 2.77(0.65)a 3.12(0.69)b 2.85(0.80)ab 3.72(0.45)
Note. Means that have no superscripts within rows in common are significantly different from each other based on post hoc pairwise comparisons with Bonferroni correction. MEP = Making Educators Partners in Youth Suicide Prevention.
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
.
D. A. Lamis et al.: Outcomes of a Suicide Gatekeeper Training Program 93
© 2016 Hogrefe Publishing Crisis (2017), 38(2), 89–99
Table 2. MEP participants’ scores from pretest to posttest (N = 700)
Pretest Posttest Range t d
Variable M (SD) M (SD)
Suicide knowledge 8.32 (1.81) 11.27 (2.09) 3–15 37.08* 1.51
Attitudes 4.07 (0.62) 4.25 (0.61) 1.5–5.0 6.90* 0.29
Self-Efficacy/Confidence 2.81 (0.67) 3.81 (0.53) 1–5 43.51* 1.66
Note. MEP = Making Educators Partners in Youth Suicide Prevention. *p < .001.
selors scored higher than teachers, administrators, and classroom aids; however, administrators demonstrated significantly more baseline self-efficacy/confidence than teachers (Table 1).
Paired t tests were used to examine changes in suicide knowledge, attitudes, and self-efficacy/confidence from the pretraining survey to the posttraining survey. Addi- tionally, Cohen’s d effect sizes were calculated for each variable to assess the strength of the effects from pre- to posttest. Specifically, Cohen (1988) defined effect sizes as: small, d = .2; medium, d = .5; and large, d = .8. There were significant increases from pre- to posttest in suicide knowledge, attitudes, and self-efficacy/confidence, with effect sizes ranging from small to large (see Table 2). When broken down by professional roles, significant differences between pre- and posttests were found in all domains with the exception of attitudes, where only teachers demon- strated significant gains (p < .01).
Next, a series of repeated measures analyses of co- variance (ANCOVA) were used to evaluate the possible moderating influence of professional role on pre/posttest changes on suicide knowledge, attitudes, and self-effica- cy/confidence, while controlling for age, gender, and any previous suicide training. Additionally, if the overall F test was found to be significant, post hoc comparisons with the Bonferroni correction were examined to determine which group means were significantly different from each other. Results indicated that there was not a significant Role Time interaction for suicide knowledge, F(3,696) = 2.54, p = .06 (see Figure 1), suggesting that school personnel who com- pleted the MEP training increased their suicide knowledge regardless of professional role. Similarly, professional role
did not moderate training effects on attitudes, F(3,696) = 1.21, p = .30 (see Figure 2). However, there was a signif- icant interaction between professional role and time on self-efficacy/confidence, F(3,696) = 5.61, p < .001 (see Figure 3). Guidance counselors demonstrated significant- ly smaller increases in self-efficacy/confidence compared with teachers and classroom aids. Moreover, teachers demonstrated significantly larger increases in self-effica- cy/confidence compared with administrators (Table 3). In general, the MEP program demonstrated its effectiveness in increasing participants’ suicide knowledge, attitudes, and self-efficacy/confidence.
Means and standard deviations were calculated for the global satisfaction score (with higher scores reflecting greater training satisfaction) and each Likert scale item (1–5) separately in order to determine which aspects of the MEP program participants liked or disliked (see Table 4). Training evaluations were positive with overall satisfaction being high (M = 3.99, SD = 0.65, range = 1–5). The ma- jority of participants (76.7%) endorsed good or very good to describe their overall experience with the MEP training and how well the training met its objectives (85.6%). Moreover, most of the respondents (86.3%) agreed or strongly agreed that the training was culturally competent for a school set- ting, and 64.7% found the training content to be very useful or extremely useful to their job responsibilities, while 29.7% found the training to be somewhat useful. Finally, 72.6% of the participants said they were either likely or very likely to recommend the MEP training to others. In sum, the online MEP training was well received by school personnel, who were generally very satisfied with the training and its over- all goals of reducing suicidality among youth.
Table 3.MEP participants’ average change in scores from pretest to posttest
Teachers Administrators Classroom aids Guidance counselors
Variable (n = 620) (n = 35) (n = 26) (n = 19)
Suicide knowledge 2.97(2.09) 3.06(1.98) 3.46(2.70) 1.63(1.89)
Attitudes 0.19(0.68) 0.03(0.57) 0.04(0.69) 0.05(0.52)b
Self-Efficacy/Confidence 1.03(0.60)a 0.74(0.54)b 0.99(0.66)ab 0.42(0.52)
Note. Analyses include age, gender, and prior suicide training as covariates. MEP = Making Educators Partners in Youth Suicide Prevention. Means that have no superscripts within rows in common are significantly different from each other based on post hoc pairwise comparisons with the Bonferroni correction.
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
.
D. A. Lamis et al.: Outcomes of a Suicide Gatekeeper Training Program94
© 2016 Hogrefe PublishingCrisis (2017), 38(2), 89–99
Figure 1. Changes in suicide knowledge from pre- to posttest for different groups of school personnel.
Figure 2. Changes in attitudes from pre- to posttest for different groups of school personnel.
Figure 3. Changes in self-efficacy/confidence from pre- to posttest for different groups of school personnel.
Teacher Administrators 1 8,22 1 8,97 2 11,19 2 12,06
Classroom 1 8,44 2 11,92
Guidance 1 10,15 2 11,93
8
8,5
9
9,5
10
10,5
11
11,5
12
12,5
K no
w le
dg e
Teachers
Administrators
Classroom Aids
Guidance Counselors
Post-MEP Pre-MEP
Teacher Adminstrators 1 4,05 1 4,35 2 4,23 2 4,37 Classroom 1 4,12 2 4,18 Guidance 1 4,27 2 4,33
4
4,1
4,2
4,3
4,4
4,5
A tti
tu de
s
Teachers
Administrators
Classroom Aids
Guidance Counselors
Post-MEP Pre-MEP
Teacher Adminstrators 1 2,77 1 3,09 2 3,8 2 3,87 Classroom 1 2,83 2 3,82 Guidance 1 3,55 2 4,13
2,5
3
3,5
4
4,5
Se lf
Ef fic
ac y
Teachers
Administrators
Classroom Aids
Guidance Counselors
Post-MEP Pre-MEP
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
.
D. A. Lamis et al.: Outcomes of a Suicide Gatekeeper Training Program 95
© 2016 Hogrefe Publishing Crisis (2017), 38(2), 89–99
Discussion
Although there have been significant advancements in suicide prevention efforts for youth, program evaluation is relatively limited. Unfortunately, implementing and evaluating novel online suicide interventions is an even rarer practice. Thus, the purpose of the present study was to evaluate the MEP online gatekeeper training in a large sample of school personnel. Specifically, this is the first study to examine whether participants who completed the MEP program reported increases in suicide knowledge, attitudes, and self-efficacy/confidence. Additionally, we investigated professional role as a potential moderating influence on training effectiveness. Lastly, participants’ satisfaction with the MEP program was also assessed fol- lowing training completion. In general, significant gains in suicide knowledge, attitudes, and self-efficacy/confidence were demonstrated from pre- to posttest and MEP was positively evaluated. As such, results suggest that imple- menting the MEP school-wide training in order to increase the number of gatekeepers who can recognize and refer at- risk youth may be an effective strategy to reduce suicidal behaviors.
In line with Hypothesis 1, results indicated that guid- ance counselors scored significantly higher on suicide knowledge at baseline than teachers and classroom aids; however, they did not score higher than administrators. This finding suggests that guidance counselors are more knowledgeable with regard to youth suicide than are other school personnel, which is not surprising given that they typically have formal training in psychology and/or coun- seling (Pérusse & Goodnough, 2001). Furthermore, as anticipated, there were no differences found among roles regarding attitudes, suggesting that all school personnel believe that they play a role in suicide prevention. Finally, guidance counselors reported higher levels of self-efficacy/ confidence than teachers, administrators, and classroom aids, with administrators reporting more baseline self-ef- ficacy/confidence than teachers. Overall, these findings are consistent with previous research (King, Price, Telljo- hann, & Wahl, 2000; Reis & Cornell, 2008; Tompkins et al., 2010; Wyman et al., 2008) and suggest that guidance
counselors are more knowledgeable and confident in their skills to identify and refer suicidal youth as compared with other professional roles, including teachers and classroom aids. In a relatively recent investigation of school staff per- spectives (Nadeem et al., 2011), counselors, social work- ers, and administrators indicated that they relied on teach- ers to inform them about the suicidal behaviors of students. Indeed, teachers are in a unique position to recognize sui- cide risk factors and warning signs, identify at-risk youth, and refer them to appropriate resources (Freedenthal & Breslin, 2010; Tompkins et al., 2010); however, they need to be properly trained. Thus, our results highlight the im- portance of providing suicide prevention training to school personnel in general and among teachers in particular.
In accordance with Hypothesis 2, all participants who completed the MEP training demonstrated significant in- creases in suicide knowledge, attitudes, and self-efficacy/ confidence. Although these results are consistent with previous findings evaluating other gatekeeper programs (e.g., Cross, Matthieu, Lezine, & Knox, 2010; Schilling et al., 2014; Smith et al., 2014), this is the first evaluation of the MEP program, with results showing a positive impact on effectiveness in training outcomes. While no specific a priori hypotheses were made regarding the potential im- pact of professional role on training effects, exploratory analyses revealed a significant interaction between role and self-efficacy/confidence, but not suicide knowledge or attitudes. Specifically, guidance counselors demonstrated smaller increases in self-efficacy than all other profession- al roles; however, it is important to note that they also re- ported high levels of self-efficacy/confidence at baseline. Indeed, other studies (e.g., Cross et al., 2011) have shown that guidance counselors have higher baseline levels of learning outcomes than teachers in school-based gate- keeper training programs, which may create a ceiling effect and makes increasing these scores at posttest following the training more of a challenge.
Further, results indicated that there were no significant differences found among professional roles in suicide knowledge, highlighting overall program effectiveness in this domain. While we cannot be certain that this null finding is indicative of the program effectiveness across
Table 4. Training evaluation results
About the MEP training M (SD)
How would you rate the overall learning experience of this course? 3.97 (.81)
How well did this training meet its objectives? 4.16 (.76)
I found this training to be culturally competent for a school setting. 4.10 (.70)
How useful was the training content to your current job responsibilities? 3.78 (.85)
How likely are you to recommend this training to others? 3.92 (.88)
Note. N = 700. MEP = Making Educators Partners in Youth Suicide Prevention.
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
.
D. A. Lamis et al.: Outcomes of a Suicide Gatekeeper Training Program96
© 2016 Hogrefe PublishingCrisis (2017), 38(2), 89–99
all professional roles, our study is adequately powered to detect such an effect and suggests that school personnel who completed the MEP training program gained knowl- edge regardless of their profession. It is important to note that although there were no significant differences found among the professional roles on suicide knowledge, all groups demonstrated significant increases. Similarly, there were no significant differences found among profes- sional roles on attitudes; however, in contrast to suicide knowledge, only teachers demonstrated significantly pos- itive changes in attitudes from pre- to posttest. Given that teachers are considered the primary audience for the MEP program, this finding reveals the advantage of teacher par- ticipation in school-based training regarding successfully changing their views toward identification and help-seek- ing behaviors with suicidal youth. In sum, our findings pro- vide preliminary evidence for the effectiveness of the MEP school-based suicide prevention gatekeeper training with regard to increasing participants’ knowledge, attitudes, and self-efficacy/confidence.
An additional aim of this study was to evaluate partici- pants’ overall satisfaction with the training. Specifically, we assessed how well they thought the program obtained its objectives, was culturally competent, and relevant to their job responsibilities. Consistent with Hypothesis 3, results indicated that the majority of school personnel who com- pleted the MEP program were satisfied with the training and would recommend it to others. Moreover, most respond- ents believed that the training was successful in meeting its objectives, culturally competent, and useful with regard to their current responsibilities. Overall, the unanimous- ly positive reactions to the web-based gatekeeper training provide initial support for offering the online MEP program to school personnel. Given the challenges of providing training in suicide prevention to all school staff (see Walsh, Hoover, & Kronick, 2013), reaching these individuals via an online system has shown to be feasible (King et al., 2015; Lancaster et al., 2014) and this universal approach has the potential to reduce suicidal behaviors in youth.
Although this study is the first to evaluate the MEP su- icide prevention program and demonstrate positive re- sults in the school setting, several limitations should be acknowledged. First, we did not include a control group in which to compare the participants who completed the MEP training. Future evaluation studies should prioritize more rigorous controlled trials employing random assignment to adequately determine the effectiveness of the MEP in- tervention (Brown, Wyman, Brinales, & Gibbons, 2007). Moreover, it is important to note that significant changes in suicide-related variables, such as the ones assessed (i.e., knowledge, attitudes, and self-efficacy/confidence), do not necessarily translate into effective suicide prevention (i.e., reductions in ideation, attempts, and deaths; Klimes-Dou-
gan, Klingbeil, & Meller, 2013). As suggested by the pe- nultimate and last levels of Kirkpatrick’s (1996) evaluation model, subsequent investigations of effectiveness should examine the extent to which the MEP training transfers to on-the-job behavior and ultimately reduce suicidality. Future researchers should collect longitudinal follow-up data, including identification and referral practices as well as maintenance of the significant participant posttest gains and monitoring of youth suicidal behaviors in the school systems. Prospective designs assessing participants across multiple time points are needed to examine long-term ef- fects of the MEP program.
Additionally, the school personnel who participated in the MEP training were primarily Caucasian and female, which may limit generalizability to male school staff and individuals from different racial/ethnic backgrounds. Rep- lication of these results across diverse samples is warrant- ed. Another limitation is the self-report data collection method, which introduces concerns related to social desir- ability bias. Moreover, given that the data were collected through an online survey and in some cases in return for continuing education credits, participants may not have been representative of all school personnel. Direct meas- ures of school staff members’ interactions with students using a multimodal data collection strategy should be em- ployed in future investigations. Furthermore, other possi- ble moderating and/or mediating third variables were not assessed, but may have affected the results. Variables to consider might include behavioral intentions and social norms, which the theory of planned behavior suggests may play a critical role in whether or not trained behaviors are adopted and implemented. Finally, although the overall sample size was large, the majority of participants were teachers and the remaining sample sizes of school per- sonnel in other professional roles were relatively small. As such, the findings regarding moderation must be consid- ered preliminary until they are either replicated or refuted across larger subsamples.
Despite these limitations, the present study provides support for the effectiveness of the MEP program. Consist- ent with recent research (e.g., Ghoncheh, Koot, & Kerkhof, 2014), the results are encouraging for the delivery of gate- keeper training through web-based media. Participants who completed the online MEP program made significant gains in suicide knowledge, attitudes, and self-efficacy/ confidence. Given that previous research and theory (e.g., Ajzen & Fishbein, 2008; Cusimano & Sameem, 2011; Walsh et al., 2013) have shown that a lack of knowledge, skills, and confidence are primary barriers for school per- sonnel who potentially come in contact with suicidal stu- dents, enhancing these abilities should improve partici- pants’ capacity to successfully recognize, intervene, and refer at-risk youth to the appropriate resources.
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
.
D. A. Lamis et al.: Outcomes of a Suicide Gatekeeper Training Program 97
© 2016 Hogrefe Publishing Crisis (2017), 38(2), 89–99
Overall, the results presented show promise for the MEP online gatekeeper training and its potential impact on the schools that adopt and implement it as a suicide preven- tion program. However, more research and evaluation em- ploying randomized controlled trials are needed to test the benefits of offering the MEP program to schools across the country. Moreover, in line with the 2014 National Strate- gy for Suicide Prevention, gatekeeper training programs, such as MEP, should be used along with other strategies in schools as a comprehensive and multimodal approach (Adelman & Taylor, 2012; Godoy Garraza, Walrath, Gold- ston, Reid, & McKeon, 2015; Wilkins et al., 2013). For ex- ample, given that suicide prevention is not only the respon- sibility of schools, other agencies, groups, and people in the community who may potentially come into contact with suicidal youth also have an important role to play. Along these lines, MEP makes it clear that although prevention may begin by identifying at-risk students in the school, easily accessible referral resources should be available in the community to address suicide risk among youth.
Our findings underscore the value of offering the online MEP program in schools to increase the number of trained gatekeepers who can effectively identify suicidal students and refer them to the appropriate services. However, the positive results of the present study only represent one effective strategy of what should be a multimodal com- prehensive approach to preventing suicide. Thus, it is im- portant for school personnel, researchers, mental health providers, and policy makers to work together to establish prevention programs that successfully reduce the inci- dence of suicidality. Given the importance of preventing suicide in young people, expanded investigations of effec- tiveness across schools in the United States are warranted to continue the systematic evaluation of the MEP program.
References
Adelman, H. S., & Taylor, L. (2012). Mental health in schools: Moving in new directions. Contemporary School Psychology, 16, 9–18.
Ajzen, I. (1985). From intentions to actions: A theory of planned behavior. In J. Kuhl & J. Beckmann (Eds.), Action-control: From cognition to behavior (pp. 11–39). Heidelberg, Germany: Spring- er. doi:10.1007/978-3-642-69746-3_2
Ajzen, I., & Fishbein, M. (2008). Attitudinal and normative variables as predictors of specific behaviors. In R. H. Fazio, R. E. Petty, R. H. Fazio, & R. E. Petty (Eds.), Attitudes: Their structure, function, and consequences (pp. 425–443). New York, NY: Psychology Press.
Armitage, C. J., & Conner, M. (2001). Efficacy of the theory of planned behavior: A meta- analytic review. British Journal of So- cial Psychology, 40, 471–499. doi:10.1348/014466601164939
Bean, G., & Baber, K. M. (2011). Connect: An effective commu- nity-based youth suicide prevention program. Suicide and Life-Threatening Behavior, 41, 87–97. doi:10.1111/j.1943-278X. 2010.00006.x
Brown, C. H., Wyman, P. A., Brinales, J. M., & Gibbons, R. D. (2007). The role of randomized trials in testing interventions for the prevention of youth suicide. International Review of Psychiatry, 19, 617–631. doi:10.1080/09540260701797779
Carlton, P. A., & Deane, F. P. (2000). Impact of attitudes and suicidal ideation on adolescents’ intentions to seek professional psy- chological help. Journal of Adolescence, 23, 35–45. doi:10.1006/ jado.1999.0299
Centers for Disease Control and Prevention, National Center for In- jury Prevention and Control. (2015). Web-based injury statistics query and reporting system (WISQARS). Retrieved from http:// www.cdc.gov/injury/wisqars/fatal.html
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Condron, D. S., Garraza, L. G., Walrath, C. M., McKeon, R., Goldston, D. B., & Heilbron, N. S. (2015). Identifying and referring youths at risk for suicide following participation in school-based gatekeeper training. Suicide and Life-Threatening Behavior, 45, 461–476.
Cross, W. F., Matthieu, M. M., Lezine, D., & Knox, K. L. (2010). Does a brief suicide prevention gatekeeper training program enhance observed skills? Crisis, 31, 149–159. doi:10.1027/0227-5910/ a000014
Cross, W. F., Seaburn, D., Gibbs, D., Schmeelk-Cone, K., White, A., & Caine, E. (2011). Does practice make perfect? A randomized control trial of behavioral rehearsal on suicide prevention gatekeeper skills. Journal of Primary Prevention, 32, 195–211. doi:10.1007/s10935-011-0250-z
Cusimano, M. D., & Sameem, M. (2011). The effectiveness of mid- dle and high school-based suicide prevention programmes for adolescents: A systematic review. Injury Prevention, 17, 43–49. doi:10.1136/ip.2009.025502
Freedenthal, S., & Breslin, L. (2010). High school teachers’ expe- riences with suicidal students: A descriptive study. Journal of Loss And Trauma, 15, 83–92. doi:10.1080/15325020902928625
Ghoncheh, R., Koot, H. M., & Kerkhof, A. M. (2014). Suicide preven- tion e-learning modules designed for gatekeepers: A descriptive review. Crisis, 35, 176–185. doi:10.1027/0227-5910/a000249
Godoy Garraza, L., Walrath, C., Goldston, D. B., Reid, H., & McKeon, R. (2015). Effect of the Garrett Lee Smith Memorial Suicide Pre- vention Program on suicide attempts among youths. JAMA Psy- chiatry, 72, 1143–1149. doi:10.1001/jamapsychiatry.2015.1933
Gould, M. S., & Kramer, R. A. (2001). Youth suicide prevention. Su- icide and Life-Threatening Behavior, 31, 6–31. doi:10.1521/ suli.31.1.5.6.24219
Gould, M. S., Munfakh, J. H., Lubell, K., Kleinman, M., & Parker, S. (2002). Seeking help from the Internet during adolescence. Jour- nal of The American Academy of Child & Adolescent Psychiatry, 41(10), 1182–1189. doi:10.1097/00004583-200210000-00007
Hamilton, E., & Klimes-Dougan, B. (2015). Gender differences in su- icide prevention responses: Implications for adolescents based on an illustrative review of the literature. International Journal of Environmental Research in Public Health, 12, 2359–2372.
Indelicato, N. A., Mirsu-Paun, A., & Griffin, W. D. (2011). Outcomes of a suicide prevention gatekeeper training on a university cam- pus. Journal of College Student Development, 52, 350–361.
Ingvarson, L., Meiers, M., & Beavis, A. (2005). Factors affecting the impact of professional development programs on teach- ers’ knowledge, practice, student outcomes, and efficacy. Ed- ucation Policy Analysis Archives, 13, 1–26. doi: 10.14507/epaa. v13n10.2005
Isaac, M., Elias, B., Katz, L. Y., Belik, S., Deane, F. P., Enns, M. W., & Sareen, J. (2009). Gatekeeper training as a preventative inter- vention for suicide: A systematic review. The Canadian Journal of Psychiatry, 54, 260–268.
Kann, L., Kinchen, S., Shanklin, S. L., Flint, K. H., Kawkins, J., Har- ris, W. A., … Centers for Disease Prevention and Control. (2014).
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
.
D. A. Lamis et al.: Outcomes of a Suicide Gatekeeper Training Program98
© 2016 Hogrefe PublishingCrisis (2017), 38(2), 89–99
Youth risk behavior surveillance United States. Morbidity and Mortality Weekly Report (MMWR) Surveillance Summaries, 63, 1–168.
King, C. A., Eisenberg, D., Zheng, K., Czyz, E., Kramer, A., Horwitz, A., & Chermack, S. (2015). Online suicide risk screening and in- tervention with college students: A pilot randomized controlled trial. Journal of Consulting and Clinical Psychology, 83, 630–636. doi:10.1037/a0038805
King, K. A., Price, J. H., Telljohann, S. K., & Wahl, J. (2000). Prevent- ing adolescent suicide: Do high school counselors know the risk factors? Professional School Counseling, 3, 255–263.
King, K. A., & Smith, J. (2000). Project SOAR: A training program to increase school counselor’s knowledge and confidence re- garding suicide prevention and intervention. Journal of School Health, 70, 402.
Kirkpatrick, D. L. (1959). Techniques for evaluating training pro- grams. Journal of the American Society of Training Directors, 13, 3–9.
Kirkpatrick, D. L. (1996). Great ideas revisited: Techniques for eval- uating training programs. Training and Development, 50, 54–59.
Klimes-Dougan, B., Klingbeil, D. A., & Meller, S. J. (2013). The impact of universal suicide- prevention programs on the help-seek- ing attitudes and behaviors of youths. Crisis, 34, 82–97. doi:10.1027/0227-5910/a000178
Lancaster, P. G., Moore, J. T., Putter, S. E., Chen, P. Y., Cigularov, K. P., Baker, A., & Quinnett, P. (2014). Feasibility of a web-based gatekeeper training: Implications for suicide prevention. Sui- cide and Life-Threatening Behavior, 44, 510–523. doi:10.1111/ sltb.12086
Long, L. K., DuBois, C. Z., & Faley, R. H. (2008). Online training: The value of capturing trainee reactions. Journal of Workplace Learning, 20, 21–37. doi:10.1108/13665620810843629
Mann, J. J., Apter, A., Bertolote, J., Beautrais, A., Currier, D., Haas, A., … Hendin, H. (2005). Suicide prevention strategies: A system- atic review. Journal of the American Medical Association, 294, 2064–2074. doi:10.1001/jama.294.16.2064
Mann, J. J., & Currier, D. (2011). Evidence-based suicide prevention strategies: An overview. In M. Pompili, R. Tatarelli, & R. Tatarelli (Eds.), Evidence-based practice in suicidology: A source book (pp. 67–87). Cambridge, MA: Hogrefe Publishing.
Musal, B., Taskiran, C., Gursel, Y., Ozan, S., Timbil, S., & Velipasao- glu, S. (2008). An example of program evaluation project in un- dergraduate medical education. Education for Health: Change in Learning & Practice, 21, 1–7.
Nadeem, E., Kataoka, S. H., Chang, V. Y., Vona, P., Wong, M., & Stein, B. D. (2011). The role of teachers in school-based suicide pre- vention: A qualitative study of school staff perspectives. School Mental Health, 3, 209–221. doi:10.1007/s12310-011-9056-7
National Action Alliance for Suicide Prevention: Research Prior- itization Task Force. (2014). A prioritized research agenda for suicide prevention: An action plan to save lives. Rockville, MD: National Institute of Mental Health and the Research Prioriti- zation Task Force.
Pérusse, R., & Goodnough, G. E. (2001). A comparison of existing school counselor program content with the education trust ini- tiatives. Counselor Education & Supervision, 41, 100–110.
Reis, C., & Cornell, D. (2008). An evaluation of suicide gatekeeper training for school counselors and teachers. Professional School Counseling, 11, 386–394. doi:10.5330/PSC.n.2010-11.386
Rodi, M. S., Garraza, L. G., Walrath, C., Stephens, R. L., Condron, D. S., Hicks, B. B., & McKeon, R. (2012). Referral patterns for youths identified at risk for suicide by trained gatekeepers. Crisis, 33, 113–119. doi:10.1027/0227-5910/a000114
Schilling, E. A., Lawless, M., Buchanan, L., & Aseltine, R. J. (2014). ‘Signs of Suicide’ shows promise as a middle school suicide prevention program. Suicide and Life-Threatening Behavior, 44, 653–667. doi:10.1111/sltb.12097
Smith, A. R., Silva, C., Covington, D. W., & Joiner, T. J. (2014). An as- sessment of suicide- related knowledge and skills among health professionals. Health Psychology, 33, 110–119. doi:10.1037/ a0031062
Sueki, H., & Ito, J. (2015). Suicide prevention through online gate- keeping using search advertising techniques: A feasibility study. Crisis, 36, 267–273. doi:10.1027/0227-5910/a000322
Thompson, E. A., Eggert, L. L., Randell, B. P., & Pike, K. C. (2001). Evaluation of indicated suicide risk prevention approaches for potential high school dropouts. American Journal of Public Health, 91, 742–752. doi:10.2105/AJPH.91.5.742
Tompkins, T. L., Witt, J., & Ajbraibesh, N. (2009). Does a gatekeep- er suicide prevention program work in a school setting? Eval- uating training outcome and moderators of effectiveness. Su- icide and Life-Threatening Behavior, 39, 671–681. doi:10.1521/ suli.2009.39.6.671
Tompkins, T. L., Witt, J., & Abraibesh, N. (2010). Does a gatekeep- er suicide prevention program work in a school setting? Eval- uating training outcome and moderators of effectiveness. Su- icide and Life-Threatening Behavior, 40, 506–515. doi:10.1521/ suli.2010.40.5.506
Walsh, E., Hooven, C., & Kronick, B. (2013). School-wide staff and faculty training in suicide risk awareness: Successes and chal- lenges. Journal of Child and Adolescent Psychiatric Nursing, 26, 53–61. doi:10.1111/jcap.12011
Wilkins, N., Thigpen, S., Lockman, J., Mackin, J., Madden, M., Per- kins, T., … Donovan, J. (2013). Putting program evaluation to work: A framework for creating actionable knowledge for sui- cide prevention practice. Translational Behavioral Medicine, 3, 149– 161. doi:10.1007/s13142-012-0175-y
Wyman, P. A., Brown, C. H., Inman, J., Cross, W., Schmeelk-Cone, K., Guo, J., & Pena, J. B. (2008). Randomized trial of a gatekeeper program for suicide prevention: 1-year impact on secondary school staff. Journal of Consulting and Clinical Psychology, 76, 104–115. doi:10.1037/0022-006X.76.1.104
York, J., Lamis, D. A., Friedman, L., Berman, A. L., Joiner, T. E., McIn- tosh, J. L., … Pearson, J. (2013). A systematic review process to evaluate suicide prevention programs: A sample case of com- munity-based programs. Journal of Community Psychology, 41, 35–51. doi:10.1002/jcop.21509
Received September 26, 2015 Revision received March 15, 2016 Accepted March 16, 2016 Published online August 26, 2016
About the authors
Dorian A. Lamis, PhD, is a licensed clinical psychologist and an assis- tant professor in the Department of Psychiatry and Behavioral Scienc- es at the Emory University School of Medicine, Atlanta, GA. His research focuses on mood disorders, substance use, and suicidal behaviors in a variety of populations. He has published over 100 peer-reviewed arti- cles and book chapters on these topics.
Maureen Underwood, LCSW, is a licensed clinical social worker, cer- tified group psychotherapist, and a nationally recognized expert on youth suicide prevention, with a focus on schools and best-practice programs that enhance overall student outcomes. She is co-developer of LIFELINES: A Suicide Prevention Program, LIFELINES Postvention, and LIFELINES Intervention.
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
.
D. A. Lamis et al.: Outcomes of a Suicide Gatekeeper Training Program 99
© 2016 Hogrefe Publishing Crisis (2017), 38(2), 89–99
Nicole D’Amore, MA, is a nonprofit professional, having received her master’s degree in public policy with honors from Monmouth Univer- sity in 2014, focusing on social and political philosophies and program evaluation research. Currently, Nicole is pursuing a master’s degree in social work through Monmouth University, West Long Branch, NJ, USA.
Dorian A. Lamis Department of Psychiatry and Behavioral Sciences Emory School of Medicine Grady Hospital, 10 Park Place Atlanta, GA 30303 USA dalamis@gmail.com
Th is
d oc
um en
t i s c
op yr
ig ht
ed b
y th
e A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. Th
is a
rti cl
e is
in te
nd ed
so le
ly fo
r t he
p er
so na
l u se
o f t
he in
di vi
du al
u se
r a nd
is n
ot to
b e
di ss
em in
at ed
b ro
ad ly
