Saint Louis County Project RESTORE

State: MO Type: Model Practice Year: 2020

The Saint Louis County Department of Public Health (DPH) is a large local health department serving approximately one million residents of St. Louis County. The issue of concern in our region is youth violence. From 2013 to 2014, the rate of youth (15-24) homicides caused by firearms in target zip codes more than doubled (from 14.48 to 31.87 per 100,000 population). The north St. Louis County area ranks ninth nationally in the rate of youth who are killed by gun violence, and St. Louis County had the highest number of violent teen deaths in the state 2010-2014. In 2016, Saint Louis County Police reported that six youth were victims of homicide within target zip codes alone. With 50 youth gunshot deaths per 100,000 people, the relative social, financial, and physical burden of violence in the area is one of the highest in the U.S. To address this public health challenge, DPH partnered with a multidisciplinary team of institutions. This effort, which currently serves over 450 students, is called Project RESTORE (Reconciliation and Empowerment to Support Tolerance and Race Equity).

The partnership leverages existing and emerging research, established partnerships, adapted youth programs, and structured community policing to build a cohesive, cross-sector ecosystem of violence prevention, academic support and behavioral health promotion interventions to address violence among youth in north St. Louis County. Interventions include: peer mentorship program called TeenAge Health Consultants (TAHC) where students from high school deliver a structured life-skills lessons, some of which was developed by students from the area, to middle school students; a summer program and year-round after-school program operated by the Police Athletic League (PAL), which involves athletic activities such as swimming, basketball, boxing, field trips to local and regional attractions such as the National Civil Right Museum in Memphis, lessons by both civilian and uniformed law-enforcement personnel; tutoring delivered by electronic software as well as teachers; parent engagement activities involving learning about mental health and coping techniques; and professional development focusing on cultural competency. Project RESTORE set specific objectives to address this major public health problem: (1) increase cultural competence among those working with or serving at-risk youth; (2) protective / resiliency factors; (3) improve academic outcomes; (4) decrease disciplinary actions; (5) arrests, court referrals, crimes, gun violence; (6) reduce stress and increase behavioral health; and (7) increase family engagement.

As a result of Project RESTORE, we have been able to build capacity of staff at the different institutions that will ensure sustainability of the program. For instance, a total of 283 teachers received cultural competency training. Similarly, the implementation of Restorative Justice practices has now been enhanced in these schools based on initial assessment and an action plan provided to the schools as part of Project RESTORE. We have conducted assessment of the implicit bias training that officers in St. Louis County receive and will provide feedback to law-enforcement leadership on the impact of the training. We also increased parental engagement. In year two, we hosted parent engagement activities at the schools that were attended by over 400 parents. Moreover, we have made minimal improvements in academic outcomes at the intervention school; the average GPA of the cohort students improved from 2.28 in year one to 2.33 in year two. Similarly, in year two, the proportion of students in the program who received disciplinary actions decreased from 32.5% in year one to 23.7% in year two. In addition to fostering positive outcomes for the cohort student, peer mentors have also benefited from the program. In year one we trained 52 peer mentors, most of whom were upper class students and they all received community service credits that they applied toward their graduation. In year two, we trained a similar number and we are currently in the process of training the third batch of peer educators who will continue delivering the life skills program in the spring semester of 2020. Additional, results of the project are presented in the evidence-based section of this application. We have met most of the objectives of the project. However, we were not able to measure two of the objectives and we cannot say whether they were met or not. These will be assessed in the next cycle.  Our success can be attributed to the strong partnership and the peer mentorship aspect of the life skills program. Understanding that a multidisciplinary approach is required for effective violence prevention strategies, its successful implementation gives public health professionals another tool to address youth violence and will have important public health implications.

Target Population

Starting with the secession of St. Louis City from St. Louis County in 1876, the St. Louis region has been exponentially fragmented by the interests of businesses, land-owners and policy makers; resulting in the creation of over 89 distinct municipalities. St. Louis continues to be one of the most racially and economically segregated regions in the country; ranking as the 5th most racially segregated of 50 large metropolitan areas in the United States.[1] Resources and median incomes vary sharply by zip code, with some zip codes in north St. Louis County having median household incomes of only $15,313 compared to the median incomes for St. Louis City and County of $34,384 and $58,485 respectively. St. Louis ranks near the bottom (42 of 50 metropolitan areas) in economic mobility, defined as a person or group's ability to improve their economic status by moving up in income. Residents of zip codes separated by only few miles have up to an 18-year difference in life expectancy.[2] This residential segregation and income inequality has created school districts with a large majority of either White or Black students, as well as drastic disparities in school resources. This systemic, structural racism of St. Louis segregation fueled a confluence of factors to create what we now call the school to prison pipeline.” In August 2014, when a White police officer shot and killed Michael Brown, an unarmed African American man in Ferguson, the resulting protests and civil unrest over the next 12 months represented a tipping point of decades of intentional policy and practices that have created significant health and social inequities and generations of community- and individual-level trauma. With 60 police departments and 32 municipalities contracting with another department for services, fragmented policing exacerbates resident confusion, community distrust, and high crime rates.

Therefore, target population for Project RESTORE were students enrolled in 7th grade (at least 12 years of age) at the start of the 2017-2018 school year in three target Middle Schools in the Hazelwood, Normandy, and University City districts. Partner school districts are within close proximity to one another Students enrolled in these schools are predominantly minorities and environmentally disadvantaged, and many are also economically disadvantaged. The three school districts had a combined student enrollment of 8,360 in the spring of 2017, with 92% of the students being African American, 5.4% White, and 1% Hispanic/Latino. Approximately 67% of the students qualified for free/reduced lunch at school. In the target community, 24% of children under 18 years lived in poverty. Ultimately, Project RESTORE enrolled 587 students in the intervention group from the three school districts. This represents 7% of the entire student population in the three school districts.  A group of 327 students were enrolled in the comparison group from another Middle school in Hazelwood School District.


The RESTORE program improves outcomes compared to strategies aimed to improve academic outcomes and reduce youth violence because it is supported by a multi-sector collaborative team of evaluation and public health experts. Project RESTORE includes new innovative programs and tests modifications to evidence-informed programs. Implementation of a Restorative Justice Framework where all educators at the intervention schools are trained in Restorative Justice Practices provides systems-level changes that create a protective school environment and strengthen protective factors—skills to deal with conflict. Further, peer-led life skills curriculum (TAHC) was modified by adopting a more structured program delivery that adheres to best practices for violence prevention; in year 2, TAHC implemented a culturally tailored, curriculum designed by students in the target area. Rigorous local evaluation allowed researchers to determine whether this peer education format improves known protective factors for youth violence and associated high risk behaviors. In addition, project RESTORE included a unique integration of a year-round community policing component (PAL) to reduce minority youth violence. Specifically, through PAL, County Police Department engaged at-risk youth in a variety of athletics activities during school year as well as summer camps and serve as role models and mentors to the students. An overall coordinated approach to RESTORE programs and the evaluation design drives improvement of outcomes and impacts on the RESTORE Cohort youth. Table 1 below demonstrates how the levels of social ecological influence align with the evidence basis and how the adaptations and innovative aspects of the project combine to form an integrated youth violence prevention program that incorporates public health, education, cultural competence and community policing.

Table 1: Social Ecological Model, Evidence-Based Interventions and Innovations


Proven Outcomes

Innovation & Improvement


Restorative Justice

(Conflict Resolution)

Reduces recidivism (offenders); Reduces suspensions, expulsions, disciplinary infractions, misbehavior, deviance; Students report fewer victimizations, better academic achievement, decreased absenteeism.

Expanding to a whole school strategy including initial formal assessment, tailored action plan and summative evaluation of intervention fidelity; Focus on proactive skill building school-wide as well as resolving immediate conflicts.

Teacher Training (Cultural Competence)

Reduces disparities in disciplinary actions; Promotes retention in school and academic success for African American males; Fosters positive relationships with teachers.

Promoting cultural competence, trauma-informed practices, and conflict management skills in Restorative Justice context; Partnering teacher education with direct student interventions.

Fair and Impartial Policing (Implicit Bias)

Implicit bias training is a new requirement in Missouri, but has been found to reduce implicit bias by raising participants' concern about and awareness of personal bias.

Implementing in highly segregated urban area with history of civil unrest and tensions; Evaluator will assess the efficacy of the training in the context of the broader RESTORE initiative.


Police Athletic League – PAL (Community Policing)

PAL is a form of community policing, which includes fostering partnerships between police and citizens to promote public safety; Connections with adult mentors is a youth violence protective factor; Coming together to participate in common interests is a key factor in the success of mentoring programs.

Adaptation includes expansion of PAL from an academic year, after-school activity to a year-round program including Summer Program and expanded to new schools; PAL is widely implemented though no robust evaluations utilizing experimental design have been published to demonstrate the specific programs' efficacy or impact.

Individual / Relationship

Teen Age Health Consultant – TAHC (Peer Education and Life Skills Training)

Helps students develop positive group norms and make healthy decisions; Based on USAID Evidence-Based Guidelines for Youth Peer Education and CDC Health Education Curriculum Standards; Life and social skills development is a promising strategy for preventing youth violence; Programs designed by peers are viewed by students as more concrete” than those designed by adults; Similar to School-Connect in NREPP database.

TAHC peer educators (H.S. upper classman) will adopt a more structured curriculum that adheres to best practices for violence prevention; In year 2 TAHC will implement a curriculum designed by students in target area; Popular program implemented by St. Louis DPH for 30+ years; Rigorous local evaluation will allow researchers to determine whether this peer education format improves known protective factors for youth violence and associated high risk behaviors; Design and delivery by students is a unique aspect of adaptation.


Academic Tutoring

Academic tutoring has been found to promote high academic achievement; Partner districts have identified Mathematics and English as priorities.

Tailored tutoring based on individual needs and priorities of students coordinated schools; Comprehensive evaluation will allow researchers to identify interaction of strategies to assess impact on academic achievement.

[1] Ihnen, A. (2013). Lies, damn lies, racial integration and segregation in St. Louis, and statistics. NextSTL. Retrieved from: 

[2] Purnell, J., G. Camberos, and R. Fields. (2014). "For the sake of all: A report on the health and well-being of African Americans in St. Louis and why it matters for everyone."

Goals and Objectives of the practice:

The goal of Project RESTORE was to plan, implement and evaluate a series of coordinated, multi-disciplinary interventions tailored for minority, at-risk youth that improve academic, disciplinary and health-related protective factors and reduce risk factors for violent behaviors among participating youth. To achieve this goal, the project had specific objectives: (1) increase cultural competence among those working with or serving at-risk youth; (2) protective / resiliency factors; (3) improve academic outcomes; (4) decrease disciplinary actions; (5) arrests, court referrals, crimes, gun violence; (6) reduce stress and increase behavioral health; and (7) increase family engagement.

What did you do to achieve the goals and objectives?

Project RESTORE staff at DPH worked with partner school administrators to identify staff members at the intervention schools as well as the high school from which peer mentors would be recruited. These staff members then worked with Project RESTORE team from DPH to coordinate the implementation of the programs at the intervention schools. In consultation with school administrators, a class period during which the peer mentorship program (TAHC) would be delivered was identified at each of the schools. The project RESTORE coordinator then worked with the three school districts to: (1) identify and sustain a TAHC sponsor (s) in each partner high school who are responsible for recruiting peer educators (10-30 students); and (2) identify and maintain contacts in intervention schools (e.g., Principals, Counselors, Health or Family and Consumer Studies teachers) to assist with scheduling TAHC presentations.  Once the peer educators were identified, DPH staff then delivered a one-half day training (annually) to peer educators for them to learn about the history of the program, understand their role as peer educators and role models, be trained on presentation formats and tips, and practice participating in, and leading lessons. The TAHC curriculum offers grade- and age-appropriate lessons, and is delivered in a highly structured and prescriptive sequence of sessions for the first time as part of the Project RESTORE program. Specifically, in Year One of the program, the peer educators were trained to deliver a curriculum consisting of eight 50-minute lessons. These lessons covered a variety of topics including: (1) every choice counts; (2) dealing with conflict; (3) alcohol and tobacco and other drugs; and (4) dating issues.  In Year Two the peer mentors were again trained for one-half day and the lessons the curriculum included eight 50-minute lessons focusing on: (1) positive outlook on life; (2) forming a positive self-concept; (3) decision making and problem solving; (4) managing stress; (5) coping with depression; (6) substance abuse; and (7) bullying. During scheduled TAHC lesson day and class period, trained peer educators were transported from high school in the same district to the intervention middle school to deliver lessons to the cohort students. During the designated class time, either TAHC sponsor (teacher) or the teacher for the specific class period was present in the classroom only to observe and ensure order. For providing peer mentorship and education, as an incentive, the peer educator received credit for volunteer hours that they applied toward their graduation requirements. 

The summer program was delivered by the Police Athletic League (PAL). PAL planed, implemented and reported data for the summer program that offers opportunities to participate in activities on the school campus, involves sports and physical activity, provides positive and productive individuals to serve as mentors (law enforcement officers) and involves parents/ caregivers in activities to promote their understanding of risk and protective factors and foster interaction and communication with their children. The summer program also includes field trips at the end of each week. Field trip destinations in Year One included trips to Busch Stadium for a baseball game, Brunswick Zone, City Museum, Redman Pool, Six Flags and a culminating day-long trip to the Civil Rights Museum in Memphis, Tennessee. In Year Two, a six-week summer program was delivered again from May 15, 2019 to June 30, 2019. During this time, PAL participants went on several field trips, including, Six Flags of Mid America, camping, movies, fishing, swimming and several other activities.  Specialized officers in the different police units including crime analysis lab and tactical unit teams, also, came to speak to the students about their work.

Then, beginning in the 2018 – 2019 school year, as part of the four-year implementation plan for Project RESTORE, PAL transitioned to a year round after-school program. Before the transition, all partners worked closely to ensure that PAL's after-school activities aligned with existing after-school programs within the schools. The after school program therefore leveraged existing after-school activities such as basketball and other extracurricular activities to implement the year-round PAL after-school program. As part of the after-school program, PAL officers and other volunteer officers enlisted by the program help with different activities including homework and academic tutoring. Additionally, to better facilitate PAL's after-school program, the school administrators provided office space at each of the participating high schools where the cohort students transitioned. Using funds from Project RESTORE, these offices were turned into Club Houses” fully equipped with computers, video games, and other equipment that PAL participants can utilize after school. In these Club Houses” officers interact with cohort participants to play games, or help the students with their homework.  

An important aspect of this project was parent engagement. In this regard, we worked with school administrators and teachers to identify suitable event and time to bring together parents and students to talk to them about the project. On the recommendation of parents on the advisory board, we organized fun and engaging activities. Specifically, at the beginning of the 2018 – 2019 school year, we worked with school principals to organize a barbecue events at the three intervention schools that brought parents, teachers and students together to talk about the project and how their children can benefit from it. The schools brought in Disk Jockeys (DJs) to play music and entertain the students and parents. We also attended parent teacher conferences where we talked about the project.  Additionally, we brought in and expert to talk to both parents and teachers about how to deal with mental health issues, specifically focusing on how to cope with depression and stress, and how to identify these problems in their children. Two of the school districts were also able to bring in local artists that have become famous nationally (Koran Bolden, nationally acclaimed motivational speaker and author of the book Rock, Paper, Scissors; and Gerald Fulton, Jr. better known as Hitman Holla a battle rapper and star of the show Wild N Out with Nick Cannon), to talk to them about how to avoid getting in trouble and making good choices. 

An important component of the project is engaging adults who work with the cohort students. These included teachers, school staff as well as the police officers and other volunteers. To help these adults improve their engagement with the students, Project RESTORE provided professional development trainings focusing on cultural competence. Specifically, in Year One, we worked with all the schools to engage a company called Educational Equity Consultants (EEC), a local entity that provides diversity training that identifies and eradicates systems of oppression that damage our workplaces, communities and schools” (EEC, 2017). In addition, through project RESTORE funding, schools sent staff and school leaders to national training centers to be trained as trainers on Restorative Justice practices so that they can then come back and train their colleagues to ensure sustainability of these practices and changing the culture in their schools.

Another important component of this project is evaluation. To this end, it was important to have a strong evaluation team whose responsibility was to evaluate the program to ensure that the program is delivered with fidelity. The evaluation team evaluated three different components of the project: (1) the implicit bias training taken by St. Louis County Police officers; (2) the delivery of TAHC and PAL programs; and (3) Restorative Justice Practices at the schools. As described in the evaluation section, the evaluation teams worked with the Project RESTORE implementation team as well as administrators at intervention schools to provide feedback and guidance on different aspects of the project. For example, one of the major issues we learned during the planning stages of the project was that the schools were at different stages of implementing Restorative Justice practices. As part of this project, the evaluation team conducted an initial assessment in Year One and provided each of the schools with an assessment report on where they were, what they were doing, and an individualized action plan for moving forward with implementation of Restorative Justice practices. 

Criteria and Timeframe

The selection criteria of schools invited to participate in the project was based on requirement by the Office of Minority Health to include participants from schools with a high percentage of racial minority, and environmentally and/or economically disadvantaged students, as measured by the percentage of students receiving free/reduced lunch. Additionally, during the planning phase school administrators voiced concern that most of the violent behavior often started as cyber bullying. Because research indicates that electronic aggression peaks around the end of Middle School and the beginning of High School (~ 8th grade),[1]  we decided to enroll a cohort starting in the 7th grade. These students are being followed for four years until they complete 10th grade and all partners are involved in all aspects of the project until the end.

LHD and Partners' Roles

The DPH, as the lead agency with already established relationships with many entities in the county, convened all the partners and led in the planning and implementation of the project. DPH continues to provide overall leadership and coordination of the project. In this role, DPH plans and hosts all regular and ad hock partner meetings and ensures that the project adheres to the four-year implementation plan developed by all stakeholders. Further, as the cornerstone program (TAHC) is run by DPH, it coordinates with the schools to provide this training to peer educators and students in the cohort. Moreover, DPH reports benchmark data and progress to OMH; ensure financial and programmatic compliance with all HHS and OMB regulations. The other stakeholders play crucial roles as well. The three school partners provide school administrative data for project cohort; coordinate academic tutoring at least four hours per week; facilitate student and parent recruitment for Advisory Board and in-school interventions; facilitate teacher participation in: cultural competence and/or conflict management training, Restorative Justice assessments, seeking parental consent for cohort students; promote engagement in program evaluation; ensure availability of school campus facilities year-round (summer and academic year). Partners from institutions of higher learning provide evaluation for all aspects of the project. Law-enforcement agency approves officer participation and encourages full engagement in PAL and Implicit Bias training; provides access to law enforcement officers for evaluation of training and PAL; recruits cohort students to attend PAL after-school activities and Summer Program; plans, manages, implements and reports on PAL activities per project requirements and evaluation team requests; promotes program participation (students and law enforcement officers); provides training, orientation, guidance and supervision of PAL officers.


Project RESTORE is implemented through a 4-year $1.7 million funding from the Office of Minority Health, in the U.S. Department of Health and Human Services. This amount is spread out over 4 four years ($425,000 per year). Additionally, partners including DPH and St. Louis County Police Department have made staff available to help with the implementation of the project. For example, at DPH, the time that the Program Director, a Health Educator, and three members of the chronic disease epidemiology team is provided as in-kind service to the grant. The St. Louis County Police Department has assigned three officers and a commanding Sargent to the Police Athletic League. While these officers work full-time on the implementation of Project RESTORE at the three partner schools and other related projects, their salaries are covered by the St. Louis County Police Department.

[1] Williams, KR et al. (2007). Prevalence & predictors of internet bullying. Journal of Adolescent Health. 41:S14-21.

Project RESTORE Evaluation

The project goal for Project RESTORE is to plan, implement and evaluate a series of coordinated, multi-disciplinary interventions tailored for minority, at-risk youth that improve academic, disciplinary and health-related protective factors and reduce risk factors for violent behaviors among participating youth. In order to achieve this goal, program objectives include: 1) increasing teacher understanding of implicit bias and cultural competence, 2) increasing police officer understanding of implicit bias and cultural competence, 3) increasing peer, parent and/or adult connections and student social skills, 4) increasing academic achievement, 5) decreasing disciplinary actions, 6) increasing fidelity and integration of Restorative Justice practices, 7) decreasing encounters with criminal justice system, 8) decreasing aggression and peer violence, and 9) increasing parent engagement and understanding of protective factors.  Performance and outcome measures are presented in Table 2 below.

In order to measure achievement of these objectives, the St. Louis County Department of Public Health (DPH) recognizes the need for a robust evaluation design. To achieve this the St. Louis County DPH collaborates with a multi-disciplinary evaluation team. The evaluation team consists of evaluators from the University of Missouri St. Louis (UMSL) Department of Criminology and Criminal Justice, Southern Illinois University (SIU) Carbondale Department of Criminology and Criminal Justice, and the University of Missouri's Institute of Public Policy.

The evaluation team created primary data collection systems to evaluate and analyze the work of Project RESTORE. The first component of the evaluation are pre-post program surveys for trainings conducted with project personnel. The first set of surveys are for implicit bias trainings conducted with law enforcement personnel. The next set of surveys are for program staff and teachers at participating schools on school climate, cultural competency and implicit bias professional development. These surveys are collected online through Qualtrics online survey software. They aim to measure the change in cultural competency skills of school personnel and law enforcement officers working with minority, at-risk youth while identifying opportunities to refine or improve upon implicit bias training for law enforcement and school personnel. Analyses of these data include pre-post means test comparison for law enforcement implicit bias training and annual difference in mean comparison for school personnel.

The second component of the evaluation are surveys for youth at the intervention and control schools. These surveys are conducted annually with in-person paper surveys. They aim to measure the change in disciplinary actions, suspensions and expulsions among youth participants, the change in youth violence protective factors (i.e. positive, healthy relationships), the change in stress and behavior health, change in family engagement and environment, change in youth arrests, court referrals, crimes, homicides, and gun violence, and the change in academic outcomes of youth participants. Specifically, Police Athletic League (PAL) after-school and summer programming, TeenAge Health Consultant (TAHC) curriculum and participation, and academic tutoring offered through Project RESTORE. Analyses of these data include annual difference in mean comparison between treatment and control groups utilizing Difference-in-Difference regression methodology.

The final component of the evaluation are qualitative interviews with school administrators and project personnel, TAHC peer mentors and youth participants, and a survey of stakeholders.  This component aims to measure Restorative Justice fidelity and integration in participating school districts. Phone interviews are conducted with intervention and control school administrators and Project RESTORE personnel. Online surveys are conducted with community stakeholders and key personnel. Additionally, in-person focus groups are conducted with TAHC peer mentors and youth participants. Findings are analyzed through qualitative themes assessment methodology where participant quotes are utilized in reporting.

Secondary data are collected from school districts, the Missouri Department of Elementary and Secondary Education (DESE), and the Census Bureau's American Community Survey (ACS) for district and regional-level data comparisons. These include regional demographic profiles, income levels, district level free and reduced lunch status, and graduation rates.

Following data collection and analyses, the evaluation of law enforcement implicit bias training found that trainees indicated they recognize their own bias, felt confident in their knowledge about implicit bias, and their ability to address it prior to taking course. Results were similar across gender, race, and rank. The evaluation observed the most change in participant understanding of subjective culture and how it interacts and intersects with law enforcement culture. Addition changes were seen in participants understanding of their own values and biases. Results indicate implicit bias awareness exists for the law enforcement personnel and that the next phase of trainings should focus on reactions and treatment of implicit biases in the field. Evaluation results have been shared with the training team for future law enforcement implicit bias trainings.

The evaluation of school personnel implicit bias and cultural competency professional development found statistically significant differences in the cultural competency of administrators with the control school administrators indicating more adeptness in this area. Moreover, the control school administrators seemed to be viewed as more willing or capable in speaking about racism/sexism/classism. Finally, there were statistically significant differences between the two groups in terms of disciplinary procedures. Control school personnel indicated that verbal reprimands, conferences with students, notification of a parent, and detention were utilized more often than in the treatment schools. Results highlight that the schools with the highest need for intervention are the treatment schools. Evaluation findings are shared with school administrators to inform district-level discipline policy and guide future professional development opportunities.

The youth participant evaluation is in phase three of four and final data collection and analysis are ongoing. Initial review of youth survey results compare changes between phases one and two. The evaluation reviewed results for healthy habits, school commitment and problems, resilience, and disciplinary actions. Evaluators found significant differences between control and treatment schools in these areas and findings have been shared with project personnel to inform programmatic changes like an increase in survey participation and school district reporting. These findings are communicated annually with youth issue briefs in order to inform quality improvement efforts.

The qualitative Restorative Justice evaluation analysis found programming themes that the program is building stronger relationships, addressing disruptive behavior with appropriate interventions, and creating a safe and supportive environment. This assessment is conducted on a biennial basis and results presented here are for the first year of Restorative Justice implementation. Schools were categorized into one of four integration stages: ownership, expansion, initiation, and development. The ownership stage exemplifies complete knowledge and leadership in restorative practices. The expansion stage exemplifies considerable knowledge and leadership in restorative practices. The initiation stage exemplifies basic knowledge and leadership involvement in restorative practices. The development stage exemplifies minimal to some knowledge of restorative practices and leadership acknowledges issues related to restorative practices. The majority of schools were in the development stage and one school was found to be in the initiation stage. All schools were provided with an inventory of the qualitative assessment and an action plan for their stage of restorative practices. Additional technical assistance was provided for integration of restorative justice in participating schools on moving between implementation stages.

The evaluation team has a strong partnership with the St. Louis County DPH and participating school districts. Results are presented to program stakeholders on a regular basis in order to influence programmatic changes and quality improvement. Implementation of the third phase of the evaluation is underway and results will continue to be shared with all stakeholders. The robustness of the evaluation will aid in the dissemination of results and the research impact of program findings on communities implementing interventions tailored for minority, at-risk youth. While no modifications have yet been made based on the results, we are continuing to evaluate and adjustments will be made accordingly.

Table 2. Project Logic Model with showing Outcome Measures and Performace Metrics.

Contributing Factors

Strategies and Practices

Short Term Impact (Project Period)

Long Term project Impact

Measurable Outcomes

Performance Metrics

  • High rates of bullying and electronic aggression peaking in approx. 8th grade

  • Stark disparities in household incomes and school resources 
  • Racial segregation
  • Lack of economic mobility
  • Fragmented municipalities and policing jurisdictions
  • Strained relationships between community and police
  • High incarceration rate
  • Evidence of racial bias and disparities in education and criminal justice systems
  • Cycles of poverty and family / community violence
  • Poor health outcomes for racial minorities
  • Transient youth population

  1. Implement Whole-School Restorative Justice Framework in three partner school districts.
  2. Evaluate the efficacy and impact of Implicit Bias training delivered to St. Louis County Police Department officers.
  3. Actively engage, support and foster parental involvement and understanding across all project activities and protective factors for preventing youth violence and criminal behavior.
  4. Expand, promote and evaluate Police Athletic League programming to include academic year and Summer Program activities for MYVP Cohort youth.
  5. Implement, supervise and evaluate a structured curriculum of life skills / resilience education delivered by TeenAge Health Consultants (Peer Educators).
  6. Ensure provision of academic tutoring for a minimum of four hours per week for MVYP Cohort youth.

  • Increase the cultural competency skills of school personnel and law enforcement officers working with at-risk youth.
  • Decrease the number of disciplinary actions, suspensions and expulsions among MYVP Cohort youth.
  • Strengthen family engagement to create a positive / healthy home environment.
  • Identify opportunities to refine or improve upon Implicit Bias training for law enforcement personnel.
  • Increase youth violence protective factors for MYVP Cohort youth (i.e., positive, healthy relationships).
  • Reduce stress and improve behavioral health for MYVP Cohort youth.
  • Reduce youth arrests, court referrals, crimes, homicides and gun violence.
  • Improve academic outcomes of MYVP Cohort youth.

# students retained in RESTORE Cohort

# students retained in Comparison Group

# of arrests reported in RESTORE Cohort

# of arrests reported in Comparison Group

#/rate of arrests in school district

# of disciplinary actions in RESTORE Cohort

# of disciplinary actions in Comparison Group

#/rate of disciplinary actions in district

Change in responses to YRBSS questions for RESTORE Cohort and Comparison (C&C)

District and local graduation rate

Promotion rate to next grade in RESTORE C&C

Promotion to next grade in district/locally



# TAHC peer educators

# TAHC lessons delivered

# students receive TAHC

# MYVP cohort enrolled in PAL

# PAL after-school activities offered

# PAL Contact hours

# students enrolled in PAL summer program

# Engaged adults by program

# Academic Tutoring Contact hours

# teachers / school staff trained

Self-report surveys of risk / protective factors

Aggression, Peer Violence scale assessments

Child and Youth Resilience Measures

Implicit Bias pre/post test

  • Create protective school and community environments in north St. Louis County.

  • Increase recognition and awareness of implicit bias among adults who work with at-risk youth.

  • Promote family environments that support healthy youth development.

  • Strengthen youth life skills  (e.g., conflict management) and bolster youth resiliency, self-efficacy and health literacy.

  • Connect north St. Louis County youth to caring adults and prosocial activities.

  • Support and foster academic achievement of students attending north St. Louis County schools.

  • Foster positive peer connections among target youth.

Lessons Learned

The implementation of the program was not smooth, especially in the first year. The first import lesson learned was that bringing different institutions together to work on such a big project was difficult and time consuming. In this process, we learned that understanding the capacity level of the different institutions is important in order to set expectations for what can be achieved and what cannot be accomplished by the different organizations. We also learned that transparency and open communications among the partner organizations, not only at the leadership level, but at the staff level is crucial. In this regard, regular meetings with detailed minutes with clearly identified action items, and responsibilities become very important documents for implantation of the project. Additionally, it is important to understand that not everyone in the partnership may have the full grasp of the granular details of the project. This may be especially important where there is staff or leadership turnover at the different partner organizations. In this regard it becomes important to periodically visit the overarching objectives of the project, and perhaps have a cheat sheet that everyone can refer to from time to time to have the same understanding of the project objectives.      

Cost/benefit analysis

Although we did not conduct a cost benefit analysis for this project, we know the social and economic cost of violence, particularly youth violence are substantial. Recent reports have found that school-based violence prevention programs help reduce violent risky behaviors at all grade levels (CDC, School-Based Violence Prevention). From this perspective, although the investment of $1.7 million from the Office of Minority Health in the U.S. department of Health and Human Services for the implementation of Project RESTORE is substantial, this will eventually pay great dividend overtime. This is because the capacity that has been built through this funding at the partner institution will continue to implement these violence prevention strategies that will ensure sustainability of the project. Although there is no direct way to measure the cost-savings in terms of how many violent incidences will be averted, we anticipate that the conducive environment that is being created for students to learn violence prevention skills, stay in school longer and succeed, form partnership with law-enforcement will have substantial benefits for the target community and the society in general.  

Stakeholder commitment

The St. Louis County Department of Public Health considers violence as a major public health concern and is committed to addressing this issue using novel public health approaches. Indeed, this commitment is what lead the department to partner with the Department of Criminology and Criminal Justice Department at the University of Missouri- St. Louis (UMSL) even before the formal start of the Project RESTORE partnership. The two agencies are committed to working together and learning from each other about the different approaches to address violence. This partnership is what lead to the creation of the interventions being implemented as part of Project RESTORE. The schools are also committed for very obvious reasons. In fact, when the St. Louis County department of Public Health sent out invitations to the various schools in the area, other schools did not answer the call. The schools that ultimately remained are the ones that have shown serious commitment to address the issue of youth violence in schools. Further, at the beginning of the project, each partner's leadership signed a commitment letter committing their organization to this partnership clearly outlining each partners' responsibilities in the partnership. Additionally, during the funding period, each partners signs a contract with the department of Public Health that further outlines the responsibilities and expectations for the Project.  


As we developed this proposal, one of the questions we had as partners as well as a requirement from the funding agency was sustainability of the project once funding was over. We therefore designed the intervention programs with this objective in mind. As a result, we made a determination that the central component of the project would be the peer mentorship program called TeenAge Health Consultants (TAHC). This is a peer led health education program supported by the Saint Louis County Department of Public Health (DPH) in partnership with local schools that has been delivered in this area for over 30 years. There is a fulltime health education coordinator, whose responsibility is to train students at the various high schools so that they can deliver this program to their peers. Even if this particular health educator was to leave the health department, this is a position will always be there and replacement to do exactly the same job will be hired. Other staff in the chronic disease epidemiology program have also learned the different aspects of the implementation of the program and this capacity building will ensure the sustainability of this program. In addition, the school districts and the Police Athletic League (PAL) were working together even before the Project RESTORE partnership started and have a commitment to work together in the future. Moreover, the PAL program is a program that is delivered as part of the community policing imitative of the St. Louis County Police Department and there are four full-time officers and a sergeant assigned to this unity, with potentially more officers to be added to the program as the program's success has become well known. Because of this, the PAL component of this project will be sustainable as additional resources in terms of officer-salaries will not be required. Moreover, PAL also relies on volunteer uniformed officers as well as civilian members for the police force as well as members of the community who pass a background check. Additionally, an important aspect of this project was the evaluation component so that we demonstrate what and how it works. With this evidence-based information, we will be able to replicate this process and will freely share any data collection tools, lessons learned from this process with anyone interested in implementing this program.  More importantly, as the St. Louis County Department of Public Health considers violence as an important public health problem and is one of the driving organizations to address violence prevention in the region, discussions are currently underway to create a permanent violence prevention coordinator position. This will help in continuing to implement the programs currently being implemented as part of Project RESTORE ensuring the sustainability of the program.

Colleague in my LHD