Comparing Effective Prevention Programs

As a returning college student at Columbia University, I am amazed by the number of volunteer groups, programs, and organizations that are interested in prevention. One might assume that most, if not all of these programs affiliated with the university, are effectively preventing what ever it is they say they are. But is this a safe assumption? Does the sexual violence prevention program actually prevent sexual violence? A practitioner, granting agency, or even a student wishing to volunteer in a prevention program is likely to be interested in evidence that a program is achieving its goal. Besides ensuring that funds and grants are allocated properly, private and public funders as well as eager volunteers are also interested in the most important bottom line: How effective is the program?
Nation et al.’s (2003) article “What works in prevention: Principles of effective prevention programs” points out that not all prevention programs are replicated from science-based research models demonstrating positive effects. Some prevention programs were created or adapted by local agencies and showed marginal effects. By using a “review-of-reviews approach” to identify general principles of effective prevention programs Nation et al. found characteristics that were generalizable across areas of substance abuse, risky sexual behavior, school failure, and juvenile delinquency and violence. Then they ranked each characteristic from strong to moderate based on the percentage of reviews endorsing the characteristic. This approach yielded 9 characteristics that were associated with effective prevention programs and listed the characteristics from strong to moderate endorsement: comprehensive, varied teaching methods, sufficient dosage, theory driven, positive relationships appropriately timed, socio-culturally relevant, outcome evaluation, and well-trained staff.
The purpose of this article is to compare an actual prevention program at Columbia University with current research on “What works in prevention” (Nation Et al. 2003). I compared Nation et al.’s findings with observable characteristics in the program Peace by Peace (PxP). By interviewing staff members and volunteers, participating in training and teaching as a volunteer I gathered specific characteristics that matched Nation et al.’s general principles of effective prevention programs. Although Nation et al.’s article focused on substance abuse, risky sexual behavior, school failure, and juvenile delinquency and violence; for the purpose of this comparison, I will only focus on aspects associated with juvenile delinquency and violence (the target population of Peace by Peace).
Peace by Peace (PxP) began originally as a conflict resolution program sponsored by Yale University’s undergraduate public service organization in 1992. At that time the program was called “Peace Games at Yale” and taught middle school students ways to find creative solutions to conflicts. In 1993, community impact, a community service organization of Columbia University, established a similar conflict resolution program in New York City (Peace Games New York). In 1998 the program was renamed to Peace by PEACE International (Playful Explorations in Active Conflict-resolution Education) and is now established in Atlanta, Baltimore, New Haven, New York, and Toronto. According to the PxP website, program directors and volunteers, the program is highly effective in resolving conflict and reducing student violence in school. To test this claim, I evaluated PxP’s program by using Nation et al.’s principles of effective programs.
According to Nation et al, a prevention program is effective if it is comprehensive, has varied teaching methods, provides sufficient dosage, is theory driven, promotes positive relationships, is appropriately timed, socio-culturally relevant, outcome evaluative, and has well-trained staff.
A comprehensive program provides multiple interventions such as increasing information and awareness or promoting skill development. The program should also have multiple settings that address school norms and combine parent, peer, and school interventions (Nation 2003). PxP increases information and awareness by defining conflict in the classroom and showing students how a conflict develops and escalates. The “students examine their body language, tone-of-voice, words and action” (Peace by Peace). By using games, skits, discussions, and journal writing students develop skills such as apologizing, learning how to use body language, and getting help from others. Unfortunately, PxP serves only 5 schools on the Upper West Side, Washington Heights and Harlem. Although the target population is “at risk” fifth graders, not all students in the designated schools attend the program. This is due to scheduling conflicts and volunteer accessibility and involvement. The lack of multiple setting would reduce the program’s comprehensiveness.
Nation et al. (2003) noted that a majority of effective prevention programs included varied teaching methods, which included interactive instruction and provided active, hands-on experience that increased the participant’s verbal, language and social skills. The teaching method of PxP uses all of these aspects in there teaching curriculum. Students are given interactive instructions by discussing important rules of behavior and participation before each class. Then the students are encouraged to participate in a game, skit, or discussion with various themes. One game is called “shocker” in which students stand in a circle holding hands. The volunteer begins the game by squeezing a student’s hand and starting the timer. The object of the game is to beat the clock and improve the time by working together. The goal of the game is to show positive results of working together as a group.
The next characteristic was the amount of intervention provided. Nation et al. (2003) noted that the amount of problems or deficits should be in direct correlation to the amount of interventions available. The targeted schools of PxP are in high risk neighborhoods; however, not all schools are considered high risk. One school, I.S. 223 Mott Hall school, is a public school for gifted student. By interviewing volunteer students I found out that the class environment was very different at Mott Hall as compared to the other schools. The volunteers at Mott claim that the kids are easier to teach, pay attention more, and follow directions better. One possible explanation is that the admission process is extremely selective for Mott Hall. Students are admitted on the basis of their test scores, teacher recommendations, a written exam, and an interview. One volunteer questioned the differences in effectiveness between the schools stating that the Mott Hall students were less at-risk than the students at the other schools. PxP would be more effective in producing the desired preventative effects if they provided at least minimal prevention at all schools and increasing prevention involvement in schools that are considered high risk.
The next characteristic found in effective prevention programs is “scientific justification”. According to Nation et al. (2003) theories that focus on the causes of the problem and theories that focus on changing or reducing these causes should be identified in the prevention program. “Once the causes are identified, effective prevention programs are then based on empirically tested intervention theories shown to produce the desired changes…” (p.453). The PxP website offers statistics on demographics, crime, and students’ performance. They also state “the problem”: violence, aggression, and unsafe schools. PxP gives research data showing that “schools that implemented a conflict resolution program saw a 29 percent decline in hitting and fighting and a 20 percent decline in verbal aggression. But the cause of violence, aggression, or even simple conflict in schools is not clearly stated unless you consider not having a conflict resolution program as a cause. PxP should explicitly state the risk (i.e. violence and aggression in each elementary school) and give empirical data showing the number of incidents in each school. They should also add empirical research showing the causes of violence and aggression (i.e. verbal ability or anger management). They could also use
Another important principle associated with effective prevention programs was the amount of exposure to positive relationships. Nation et al. noted that strong relationships with positive adult models were very important in areas of school failure and dropout. The training of PxP volunteers encourages a positive relationship between the students and volunteers by empowering the student to control his or her behavior and by allowing students to give feedback. Volunteers and students build a strong rapport by mutually respecting the learning environment and each other. Volunteers are asked to respond to children with sentences like “I understand that…” or “I see your side…” Unlike the normal teacher-student relationship, students and volunteers interact as peers. This serves as a bridge between controlled behavior and self-motivation and gives students an advantage for transitional success into middle school.
The time a program is initiated also had an impact on the effectiveness of the program. Dryfoos (1990, as sited in Nation, 2003) stated that “programs should be timed to focus on changeable precursor behaviors prior to the full-blown problem behavior being prevented” (p. 453). Although I could not find information from PxP pertaining to age appropriateness for prevention in 5th graders, research has shown that early adolescence was the beginning of a downward pattern of problems in some individuals (Eccles et al. 1993). The 6th and 7th grade adolescence begin a general phase of transition, both physically and emotionally. Variables such as puberty, a change in school, and a desire for independence are especially problematic for adolescents who do not have close relationships with adults outside of their homes (Eccles, 1993).
Another important characteristic that was found in many effective prevention programs was the socio-cultural relevance of the planning and implementation of the program to include the targeted group (Nation, 2003). Student participation is a major function of PxP’s curriculum. Although the class activities are coordinated in advance, students use their personal experiences, cultural norms, beliefs, and attitudes in PxP games, skits, and discussion. Rather than focusing on beliefs or practices that inadequate, students focus on finding alternative solutions to violence and anger while retaining their pride in culture, family, and school.
Characteristics that were moderately endorsed by Nation et al.’s (2003) study included outcome evaluation and involved well-trained staff. Studies have shown that some programs using anecdotal or case study evidence may actually not be effective. Studies have also shown that prevention programs are enhanced by staff members who are adequately trained, supported, and receive sufficient supervision. As mentioned earlier, PxP does not use empirical research theories for possible causes or theories showing successful treatments for school violence and aggression. Unfortunately, research-based outcome evaluations of the program are non-existent.
At present, PxP gives a two hour training and orientation session at the beginning of each semester. Volunteers also meet once a week in smaller groups to plan the week’s curriculum and discuss any problems or concerns from the previous meeting. PxP uses a “learn-as-you-go” method of training. Each volunteer is given a curriculum booklet (see attachment) which includes the activities for each session. The games, skits, and class discussions for each session are reviewed in the weekly group meetings.


Discussion
Before drawing any conclusions from this comparison, it is important to point out that PxP has an enormous of support in the elementary schools that are involved in the program. I observed 6 different classrooms in 3 New York City schools and received a massive amount of positive feedback. Both teachers and students look forward to each session. Volunteers are also enriched by the experience. Nicole Safranek, the co-director of Peace by Peace at Columbia University, has been involved in the program for 3 years. She stated that she has witnessed positive changes in the students’ behavior and attitudes. “One child explained how she used one activity at home when her mother was very angry”. Although PxP were unable to supply research information, research has shown the importance of various levels of the child’s environment.
As Bronfenbrenner (1986) pointed out, the child’s environment is extremely influential in fostering a healthy development and that this environment consisted of varying levels or models, such as the mesosystem model and exosystem model. Bronfenbrenner’s ecological model also shows which system has the greatest impact on the child, such as the direct relations between the child and the immediate environment (microsystem) vs. the network of interrelationships (mesosystem). But the majority of studies only focused on the influence of the family on the child’s performance in school rather than looking at the parent-child, student-teacher, or student-peer relationships.
Epstein’s research on “Longitudinal effects of family-school-person interactions on student outcomes” (1983a, 1983b, as cited in Bronfenbrenner,1986) showed that greater opportunities for communication and decision-making increased the child’s initiative and independence and led to higher grades. Epstein focused mainly on the processes involved in the family and classroom that impact the change in pupil’s attitudes and academic achievement. Although her research showed that family processes were more powerful than interactions in school, the processes in school were still effective in changing attitudes and academic achievements.
In conclusion, a practitioner, granting agency, or even a student wishing to volunteer in a prevention program, would have a difficult time evaluating PxP according to the principles of effective programs that were endorsed in Nation et al.’s (2003) article. This, by no means, shows that PxP is ineffective as a prevention program; rather, this comparison illustrates the difficulty in presenting and finding characteristics that empirically show program efficacy. Although many of the volunteers, teachers, and students are convinced that the program is highly successful in reducing violence and aggression in schools, anecdotal evidence and case-study interviews may not enough. Prevention programs need to use science-based research as evidence that the program is effective including using most, if not all of, the characteristics found in effective prevention programs. Efficacy information should be easy to locate and contain concise data allowing practitioners, granting agencies, and perspective volunteers to make educated appraisals of the prevention program.














Reference:
Bronfenbrenner, U. (1986). Ecology of the family as a context for human development: Research perspectives. Developmental Psychology, 22, 6, 723-742.
Eccles, J., Midgley, C., Wigfield, A., Miller-Buchanan, C., Reuman, D., Flanagan, C., Mac Iver, C. (1993). Development during Adolescence: The impact of stage-environment fit on young adolescents’ experiences in school and in families. American Psychologist, 48, 2, 90-101.
Nation, M., Crusto, C., Wandersman, A., Kumpfer, K., Seybolt, D., Morrissey-Kane, E., Davino, K. (2003) What works in prevention: principles of effective prevention programs. American Psychologist, 58, 6-7, 449-456.
Peace by peace. (2004). http://www.peacebypeace.org/
Safranek, N. (Feb. 7, 2005). Personal Interview. Co-director: Peace by Peace. Columbia University. New York
Surapaneni, P. (Feb. 13, 2005). Personal Interview. Group leader: Peace by Peace. Columbia University. New York