A Validation Study of a Prosocial Reasoning Intervention for Juveniles Under Probation Supervision

Norbert Ralph
Private Practice, San Leandro, CA

[Sexual Offender Treatment, Volume 11 (2016), Issue 2]

Abstract

Aim/Background: A validation study was conducted of a prosocial intervention for juveniles under probation supervision, 97.4% who had sexually offended using a workbook format.
Material/Methods: The study sample consisted of 39 youth under probation supervision and all but one for sexual offenses, and all but one were males. The average age was 15.7 years. White youth were 53.85% of the sample and 46.15% were other ethnic groups. Two performance measures, three counselor rating scales, and one youth self-report were used. A pre/post test design with one sample was used. The workbook intervention was used as part of individual counseling and completed in 10 weeks.
Results: A multivariate procedure was used to assess all six outcome variables which indicated that there were significant pre/post changes. Univariate analyses showed significant prosocial changes related to the intervention on 5 of 6 variables. The three counselor ratings all showed large or very large effect sizes.
Conclusions: Study results were consistent with the hypothesis that the intervention, a workbook, were related to positive changes in prosocial reasoning and behavior. Replication of the results to rule out rival hypotheses is appropriate.

Keywords: Adolescent, sex offending, treatment, outcomes, prosocial

Introduction and Literature Review

Juveniles on probation for sexual and nonsexual offenses are routinely referred for counseling services. An increasing trend is to choose evidence-based methods to promote prosocial outcomes. These outcomes include reduced sexual and nonsexual recidivism, but also reducing substance abuse, educational failure, psychiatric symptoms, and deficits in social reasoning and judgment. A number of related treatment methods have targeted specifically prosocial reasoning and social skills for juveniles on probation (Ralph, 2012; Ralph & Wong, 2013). The rationale for these types of programs is that delinquent and criminal behavior for juveniles is in part due to deficits in prosocial reasoning skills and related areas that contributed to harmful behaviors, along with other factors. These methods share the goals of teaching cognitive skills, social skills, and prosocial competence. The rationale for this approach is that juveniles on probation at some point have exercised choices that had a delinquent outcome which likely reflects deficits in these areas. Likewise, increasing prosocial reasoning to improve decision-making skills is justified based on current research, and common sense understanding of these problems. Presumably if youth are better able to understand social situations, they will be more likely to avoid future problems with probation and behave prosocially. A treatment model for teens who have made harmful choices in behavior would presumably include a developmentally based theory of moral or prosocial reasoning and factors that can promote less harmful functioning. The present paper describes an outcome study of a newly developed prosocial intervention in workbook format, Being a Pro, based on research regarding promoting prosocial development in adolescents.

Best Practices in Treatment

Best practices for interventions with probation youth have several characteristics which influenced the intervention being studied here, Being a Pro. Fidelity to the treatment model is associated with improved outcomes. One example is from the Washington State Policy Institute (2004). They note in one study, that with competent implementation of ART, the cost/benefit is +$11.66, and for services not competently delivered it is +$2.77. Lipsey (2009) using meta-analysis methodology found that programs with the highest treatment fidelity were the most effective. They also found that those that had multiple services and/or multisystemic approaches or counseling/ skill-building methods had the highest effectiveness. A related observation was that "get tough" programs were not effective, because they do not teach prosocial thinking or behavioral skills to reduce delinquent behaviors. Tennyson (2009) analyzed factors associated with program effectiveness, which included the fidelity with which the program was implemented. Four criteria were used to measure the level of fidelity which were: 1. whether a treatment manual was used, 2. whether training was provided to practitioners, 3. whether there was supervision of therapists, and 4. whether adherence checks were used. Those using three of the criteria showed the highest treatment effect, those with two showed a still positive but lower treatment effect, and those with none showed a negative treatment effect. JSO populations were not identified as being part of study populations. Also Lipsey, Wilson, and Cothern (2000) reviewed the characteristics of effective treatment programs for noninstitutionalized probation youth. Effective treatment programs used individual counseling, interpersonal skills, and had behavioral programs.

Being a Pro is also based on research which has shown the importance of the therapy relationship in therapeutic change (Norcross & Lambert, 2011; Leversee & Powell, 2012), and also the developmental literature that looks at the importance of identification, modeling, and scaffolding in child development (Watson, 2002). ART is based on similar research. Training for use of the Being a Pro model has the counselor model a prosocial relationship with the youth in counseling to give the youth a real life example of prosocial behavior and increase their affinity to the model. Information about how to have prosocial and empathic relations can best be taught by someone who behaves prosocially and demonstrates accurate empathy, for example. Lessons are more likely to be retained if taught this way by someone who the youth has a therapeutic alliance with.

The Being a Pro workbook is based on neuropsychological theory regarding cognitive development, including that learning takes place by stages in which basic concepts and skills are acquired and mastered, then combined with others into larger behavioral schemas, elaborated by practice in the real world, and become part of the "firmware" of the nervous system, and eventually habits the youth continues to use because they are beneficial (Bruner & Anglin, 1981; Watson, 2002). Through the workbook exercises, the youth's current level of prosocial reasoning is challenged by options and strategies at more sophisticated levels. This is similar to ART in its moral reasoning module, which does not correct or criticize the youth's current level of thinking, but presents them with more sophisticated prosocial alternatives. This approach simulates the natural mechanism whereby prosocial development occurs. The Being a Pro model as part of its design and using concepts like the proximal zone and pacers, uses a set of exercises or challenges, that will likely promote the youth to develop higher levels of prosocial reasoning.

Prosocial Research and Probation Juveniles

Significant research exists describing various approaches to increasing prosocial reasoning in delinquent youth. One such approach for increasing prosocial thinking and behavioral skills is the Reasoning & Rehabilitation (R&R) program developed in Canada which has had outcome studies (Antonowicz, 2005). A version was developed for youth who are under the supervision of social services or juvenile justice agencies, the R&R2 Short Version for Youth (Ross & Hilborn, 2003). The group-based intervention takes 18 hours and is done in 12 sessions. A handbook is provided for the counselors. Cognitive behavioral approaches are supplemented by prosocial neurodevelopmental training, and the developers note:

Neuroscience has established that adolescence is a period during which youths are experiencing extraordinarily rapid brain development. Based on the neurocriminology model, the youths are engaged in prosocial simulation training and prosocial role-taking throughout the program in order to stimulate their development of prosocial neuronal connections - the foundation of a prosocial identity (Cognitive Centre of Canada, n.d.).

Initial field testing was done in Estonia, and subsequently it was implemented in the United Kingdom. An evaluation with adolescent males in residential care in Scotland found that youth who completed the program made significant improvement in their risk of general reoffending, problem-solving ability, behavior, and antisocial attitudes (Curran, 2006).

Two other approaches that promote prosocial reasoning are Aggression Replacement Training (ART) (Goldstein, Glick, & Gibbs, 1998), and Moral Reconation Therapy (MRT) (Little & Robinson, 1988). These models while similar in several ways, were developed independently and use different approaches. Both have significant research validating their use with probation youth, and demonstrate a reduction in recidivism and other positive outcomes. They have been used with a wide spectrum of probation youth, including those with more severe offenses. Both are group based models, and both used Kohlberg's view of moral development as a key concept in developing their treatment methods. Both were influenced in their development by research on moral reasoning and delinquency. For example, in one article (Stams et al., 2006) in a meta-analysis of 50 studies found lower levels of moral judgment in delinquent youth than nondelinquents and effect size in the medium to large range (d=.76). Effect sizes were large for male offenders, late adolescents, those with low intelligence, and incarcerated delinquents. They concluded that developmentally delayed moral judgment was strongly associated with juvenile delinquency, even after controlling for socioeconomic status, gender, age and intelligence.

ART is a group based intervention, designed specifically for treating youth on probation, with 30 sessions, which targets deficits in social skills, moral reasoning, and emotional control. The participants in these small groups (typically four two eight youth) are run by two co-leaders, and all the youth have to start the program at the same time and attend all sessions to graduate. Group leaders present a fixed curriculum for each group session that involves role-playing, and presenting written material in a psychoeducational format. Group leaders have specific training in how to maintain prosocial attitudes and behaviors in group. Lower stages of moral development at the preconventional level are more likely to be associated with delinquent outcomes in this model. ART has research demonstrating its effectiveness in a number of studies in reducing recidivism for juveniles on probation (Goldstein, Nensén, Daleflod, & Kalt, 2005). Amendola and Oliver (2010) in summarizing the literature note that ART is a "Model Program" for the United States Office of Juvenile Justice and Delinquency Prevention and the United Kingdom Home Office. Related models have been developed by the authors of ART, including the Prepare Curriculum: Teaching Prosocial Competencies (Goldstein, 1999) and Thinking for a Change developed by Bush, Glick, and Taymans (1997).

ART was developed as a treatment for probation youth generally, and not specifically for youth with sexual offenses. However, three related studies of a residential program for juveniles who sexually offend indicate it promotes positive outcomes with these youth as well (Ralph, 2015a; Ralph, 2015b). In one retrospective study without controls (Ralph, 2015b), it was shown that youth who participated in the program had one quarter the rate of sexual acting out compared to those that didn't. In another of those studies Ralph, 2015a, the ART program was shown to be related to improvements in prosocial reasoning as measured by two instruments, the Prosocial Reasoning Outcomes (Ralph, 2016), and the Washington Sentence Completion Test (Hy & Loevinger, 1996). Both these measures are projective type tests which obtain a sample of the youth's thinking in an open-ended format, and can be reliably scored on a developmental continuum.

MRT also uses Kohlberg's theory of moral reasoning, as a basis for its model. It is a group based approach as well, that is used with adults and adolescents. In contrast to ART, MRT uses a workbook where youth work on at a self-paced rate, rather than a role-playing or lecture type didactic format. Group process is used in reinforcing treatment goals. An advantage of this approach compared to ART is that it has one group leader, not two as with ART, and new participants can join the group at any time. An optimal group would have 4 to 8 members, and have youth at various stages of progress in the workbook. Typically youth will take 26 sessions to complete the adolescent workbook. Ferguson and Wormith (2012) in a literature review, noted that MRT is effective in reducing recidivism across a variety of populations, including juveniles. MRT participants recidivate at about one third the rate of non-participants. Burnette, et al. (2004), in a residential study with 23 juveniles, found positive changes in moral reasoning levels associated with the MRT intervention, and specifically less antisocial characteristics, and lower levels of the lowest level of moral reasoning related to the MRT intervention.

ART and MRT have characteristics consistent with current research regarding best practices to promote successful outcomes. Both have a fixed curriculum, and each session is described explicitly. For ART, the group leaders have a set lesson plan to be followed. For MRT youth are using a workbook at their own pace, and the workbook format dictates they will be completing the relevant curriculum and presumably using the model with fidelity. For ART taught by some trainers, there are checklist after every session to make sure that the curriculum has been used reliably. MRT does not have such measures of fidelity. Both as used currently require that therapist complete a structured several day training. In addition, ART, in some versions requires ongoing supervision of group leaders. Neither ART are MRT have built in fidelity measures, and neither have built-in outcome measures to assess therapeutic outcomes from the intervention. Neither training had performance testing to certify providers.

Several studies by the author using qualitative methods examining youths' views of what they found beneficial from ART (Ralph, 2015a) also influenced the Being a Pro workbook. After quantitative analysis was done, two separate focus groups three years apart were conducted with youth who completed the ART program. In both focus groups youth identified that ART helped them to stop and slow down and not respond reactively to situations. They would have a reaction to a situation, then act impulsively, and then experience consequences they did not like. Their first reaction was frequently counterproductive, such as getting mad and swearing at a parent or probation officer. Only after they could stop were they able to think through the situation. Then they were able to describe a situation more thoroughly and more easily to identify "pluses and minuses" and the cost/benefits calculations of behaviors in a situation. Then they could make intelligent choices and options that had the best outcomes for them and for others, and did not violate rules or laws. The ART intervention helped them slow down emotional and behavioral reactivity, think things through better, and develop more adaptive prosocial choices. This helped the youth get along better with others, and feel better about themselves. The quantitative results from these studies in addition to the findings from the focus groups was a major influence in developing the Prosocial Model. It was a useful lesson that if you ask youth clearly what they get out of treatment, they can give you unexpected and useful answers. This approach using qualitative methods, in addition to quantitative methods, and also collaboration with research subjects, can also be part of good research.

The Being a Pro model is not only related to the theory and methods of ART and MRT, described above, and their developmental theory of prosocial development, but also research regarding prosocial reasoning in adolescents on probation. One influence was unpublished research by the author (Ralph, 2007) using the Roberts Apperception Test for Children 2 with probation youth. That study found that probation youth were more likely than non-probation youth, when matched on age and ethnic group, to use less sophisticated methods of problem identification, and problem resolution in a storytelling task, similar to the classic Thematic Apperception Test. Such youth were approximately four years delayed in their developmental levels compared to the matched sample. Youth in the probation sample, who were at lower levels on measures of prosocial reasoning on the Roberts test had higher rates of being ordered to home placement, though not a greater risk for rearrests. The relevant scales for the Roberts were Problem Identification, and Problem Resolution. These scales can be reliably scored from youth's stories for the Roberts pictures on a scale of "1" to "4." For example, regarding Problem Identification, Level 1 responses involved simple identification of feelings or situations without identifying preceding contributory factors. Level 4 responses in contrast would define the conflict situation and the reason for feelings and behaviors in an articulated fashion. Likewise for Problem Resolution 1, resolutions to problems were described in simplistic and elliptic terms, without elaboration of the process. Level 4 resolutions in contrast describe the steps and processes by which problems and feelings were constructively resolved. In the Roberts model, increased levels of prosocial thinking and reasoning are due to the use increased ability to articulate and identify what is going on in situations, and what are the steps and alternatives whereby prosocial resolutions can be developed. This implies that the youth has a greater set of ideas or templates around prosocial resolutions, and what are the necessary steps or processes might be used to attain those desired outcomes.

The Being a Pro workbook was also informed by the author's work with a newly developed assessment instrument, the Prosocial Reasoning Outcomes (PRO). This uses a story vignette format with probation youth to assess their level of prosocial reasoning. These vignettes describe everyday types of problem situations which give the youth an opportunity to demonstrate their ability to think through problematic situations and find prosocial alternatives. Similar to the Roberts test, it identifies different levels of complexity regarding problem formulations and resolutions. It describes three developmental levels of responses, Concrete, Normative, and Principled, which differentiate among the complexity of responses. The measure can be reliably scored, and higher levels reflected several factors including more detailed understanding of the social context of situations, elucidating cost-benefit and ends-means thinking, and articulation of prosocial principles. Exercises similar to those used in the PRO were incorporated as the main feature of the Being a Pro workbook, for treatment rather than assessment. This instrument was also used as an outcome measure assessing prosocial changes with youth who had gone through ART treatment.

The Roberts and PRO models provide a detailed model of what developmental changes occur regarding prosocial reasoning in adolescence, and how delays or deficits might be related to offending behaviors, and how to promote higher levels of reasoning. These models, in contrast to Kohlberg's developmental model of moral reasoning, describe the underlying characteristics that contribute to prosocial development. Prosocial development happens because the youth is able to understand social situations in terms of the context and complexity, such as what was going on before, the role of individuals thinking, feeling, and motivations. Prosocial development also includes a better understanding of the range of prosocial, positive, or "win-win" choices which address the practical problem but also the emotional needs of others. Developing moral or prosocial behavior is based on the youth's ability to use more sophisticated methods of problem analysis and resolution. Having an empirically validated theory of prosocial development is essential to understanding how prosocial development occurs, how to understand problem behaviors or deviations from prosocial behavior, and what type of factors might promote prosocial development in adolescents. For example, is suggests that youth with instruction and tools to better understand situations and develop prosocial resolutions, might be able to demonstrate higher levels of prosocial reasoning and behavior.

These developmentally related models of prosocial reasoning help us understand some of the hazards regarding harmful behaviors for adolescents. We don't let 12-year-olds drive a car in part because of their immaturity in social reasoning. But a 14-year-old male may have adult sexual abilities and drive with no practical experience, limited ideas of "ends/means" or "cost/benefit" thinking in relationships, rights of others, or consequences of sexual behavior. This mismatch between abilities and social reasoning creates the possibility of harmful sexual behaviors by adolescents. Having a developmental model of adolescent prosocial reasoning is important in understanding causes of sexually harmful behaviors for adolescents and what can reduce future problematic behavior. Treatment models for juveniles who sexually offended do not include a model of prosocial development in adolescence. There is a rapid developmental gradient regarding prosocial reasoning in adolescence that is important to understand regarding how harmful behaviors occur, and what might be done to reduce their occurrence for youth who sexually offend.

Another related area of research influencing the treatment model discussed here is regarding the adolescent brain. Dr. Laurence Steinberg in "The Age of Opportunity" (Steinberg, 2014) describes adolescence as a critical period for prosocial development in part based on the brain changes and plasticity during this period. Steinberg describes behavioral changes in adolescence as an increase in the drive or reward centers of the brain, behaviorally an increase in the risk taking in adolescents, and also a critical period of development of judgment and control centers of the brain to regulate behavior. The youth is simultaneously motivated to pursue rewarding activities, using more risky behaviors to accomplish it, under less direct supervision of adults often, while also waiting for controls over these behaviors to develop. The adolescent's ability for self-regulation/judgment is lagging ability and their drive to do things in the world. Consider the amount of supervision and ability to do many things independently for the average 12-year-old in contrast to a 16-year-old, particularly one who has a car available. Development of brain areas to regulate behavior are still developing physiologically until age 25. Relevant brain development occurs in two regions, the prefrontal cortex and the limbic system. The prefrontal cortex is the main brain area responsible for self-regulation, and the limbic system is responsible for emotion and what the person finds rewarding. Delinquent behaviors can be increased by lack of parental supervision, use of drugs and alcohol, and delinquent peers who can further pressure the youth into risky delinquent behaviors. These factors interact synergistically to promote delinquent and antisocial behavior. But adolescence is also a period of brain plasticity where there are opportunities for prosocial development which may not be available as much as in adulthood. Delinquent behavior/mistakes haven't become an enduring lifestyle/habit yet. The large "treatment effect size" observed in the delinquency literature regarding prosocial treatment methods in part may be related to this plasticity. For example, Kim, Benekos and Merlo (2015) in their meta-analysis study note the effect size sex of sexual offense treatment for adolescents is -.51, a medium effect size, and the effect size for adult treatment is -.14, less than a small effect size. All these factors make adolescence a particularly vulnerable stage of life, both for developing risky behavior but also prosocial ones. The classic "Age-Crime Curve" (see Figure 1) indicates that most crimes, including sexual and other serious ones are committed by adolescents, but drop off rapidly in early adulthood when brain maturation has progressed and can apply the "brakes" to problems behaviors. A related concept, is the "Accident Hump." Steinberg notes:

In all cultures and times, the mortality rate among boys spikes a few years after they become adolescents. It’s called the “accident hump,” and it occurs because the rise in testosterone that takes place at puberty makes males more aggressive and reckless. That makes them more likely to do things that get them killed, like picking fights or doing risky things on a dare (pg. 49).

A similar observation is in Shakespeare, The Winter's Tail (III.iii.59-63) written 400 years ago quoted in Coplely (1993).

I would there were no age between ten and three-and-twenty, or that youth would sleep out the rest; for there is nothing in the between but getting wenches with child, wronging the ancientry, stealing, fighting.

Figure 1: Age Crime Curve (Source: Farington, 1986)

The Workbook Model and Structure

The Being a Pro workbook has several components. The workbook structure is designed to be used in individual counseling or group counseling, and provide a structure for it to promote prosocial reasoning. For this study it was used only with individual counseling, however. Based on the research described above, a set of exercises and problems were developed to promote prosocial reasoning in adolescence which was described as the Prosocial Model, based on research described above. The model is described in Figure 2. which contrasted with the Reactive Model which described more impulsive models of decision-making. The counseling relationship, and the counselor modeling prosocial behavior is a key part of the intervention. The workbook is designed to be used over 10 sessions, has a set of structured exercises to be done with the therapist as part of the counseling. The use of this model does not preclude other interventions for problems like trauma, sexual aggression, family dynamics, substance abuse, or treatment for psychiatric conditions. These are factors that may require separate treatment approaches or probation management strategies. The treatment would not be suitable for youth who cannot read at less than a fifth grade level, or are less than 12. It has been used with 19-year-olds productively. Youth who are chronically aggressive and at risk for assault, who have severe intellectual delays, active suicidal ideation, psychotic states, or autistic spectrum disorders would likely not be appropriate for this intervention. It has been used with boys and girls, and appears suitable for youth with different socioeconomic, ethnic, and cultural backgrounds. The stories and characters in the workbook are multicultural, and designed for both boys and girls.

Figure 2: Being a Pro Model

To summarize, the present paper reports the results of an outcome study regarding a prosocial treatment approach in workbook form, for probation referred youth, including juveniles who sexually offend. It incorporated skill building approaches and fidelity characteristics described above by Tennyson (2009) and Lipsey, Wilson, and Cothern (2000). Fidelity with the model was promoted by the workbook structure, build in fidelity controls, and a requirement that the counselor pass an online training regarding the workbook. The theory and techniques of Being a Pro are described in greater detail in online manuals available through Safer Society Press, and the online training used in this study is available through PsychAcademy.net.

Materials and Methods

Samples

Four different programs participated in the study and all youth were under the supervision of probation. There was a total of thirty-nine youth, and all but one were in treatment for sexual offenses, and all but one were male. Thirty-one youth were in outpatient treatment and seven youth were in residential treatment. The average age was 15.7 and the age range was 13 to 17. In the sample 20.51% were Black, 53.85% White, 20.51% Hispanic, and 5.13% were other ethnicities.

Procedure

As part of outpatient or residential treatment, the Being a Pro workbook was used in one-to-one counseling. Before using the workbook counselors had to complete a 1 .5 hour online training in the workbook, which required individuals to pass a test demonstrating their competence in the method. The training explained use of the workbook in one-to-one counseling. It was to be used over 10 sessions, as part of the counseling relationship. Specific instruction was given regarding its use, although the workbook format largely delineated what was to be done. Guidelines were given regarding the counseling relationship, and that the counselor specifically was to model prosocial interactions with the youth. It was described that the counseling relationship and the role modeling it exemplified was to be viewed as equally important as the content of the workbook. All youth participating in the study completed a Pretest before starting the workbook, and a Posttest after completing it. The workbook structure of the intervention assured that a fixed and uniform treatment approach would be used, to promote treatment fidelity. Also the workbook included fidelity measures to further ensure treatment was carried out as intended.

Instruments

Consistent with best practices for psychological assessment with children and adolescents, a multi-method and multi-informant methodology was used in the outcome measures. Multi-method assessment as a concept describes that more than one type of data is used, such as self-rating or informant rating scales, and also actual samples of youths' thinking and reasoning. For example, with a youth with possible ADHD, behavior and attitude ratings for the youth from parents and teachers could be used, and self-ratings from the youth themselves. Also neuropsychological tests of attention or related domains could also be used. Multi-informant means different individuals provided the information, and with a ADHD child, this would include ratings by the parent, teacher, and the youth themselves. This combined approach (neuropsychological and adult ratings) with ADHD children in one publication was superior compared to use of one method alone (Jarret, Meter, Youngstrom, Hilston & Ollendick, 2016). With regard to the present study, ratings of prosocial behaviors and attitudes were obtained from two informants, the youth and the counselor. Also a sample of prosocial thinking was obtained from the youth themselves. It is assumed that a multi-method, multi-informant methodology provides the most reliable and ecologically valid method for assessment, and also that similar outcomes across different methods provides greater certainty that the treatment effects are genuine. Each of the measures were chosen to assess some aspect of prosocial reasoning and thinking with adolescents which the Being a Pro workbook was designed to increase. The intent was to take a concept which might be considered challenging to measure, or even "fuzzy", and find concrete tools to assess it, and changes using the Pre/Posttest methodology of the study.

Instruments were selected which fell into the following categories: Performance, Counselor, and Youth. These three categories defined the types of possible ways to assess the youth. The Performance scales were measures of prosocial reasoning completed by the youth. The Counselor scales were completed by the counselor rating the youth's prosocial behaviors and attitudes, and the Youth scales were completed by the youth doing self-rating.

Performance.
Prosocial Reasoning Outcomes- short form.
This measure of prosocial reasoning, using story vignettes, had been used as an outcome measure of prosocial reasoning (Ralph, 2015a; Ralph, 2016). The test obtains the subjects' responses and describes three levels of prosocial reasoning: Concrete, Normative, and Principled. The instrument has five vignettes, each with six questions to be answered. For this study only one vignette was used, rather than five. Therefore only one fifth of the test was used in this brief version. Ratings to assess inter-rater reliability were completed with this shortened version by two individuals for each of the six questions for seven youth. Lin's Concordance statistic was used to measure inter-rater reliability and was 0.8448 (two-sided, 95% CI 0.764 to 0.8991). An increase in the score on this instrument would indicate changes in a prosocial direction.

Washington Sentence Completion Test (WSCT) (Hy & Loevinger, 1996):
The test has adequate interrater reliability and validity research. It assesses levels of ego functioning or interpersonal maturity with a sentence completion instrument. It obtains a sample of the youth's thinking about social roles, relationships and perceptions. The stages are identified as Impulsive, Self-Protective, Conformist, Self-aware, Conscientious, Individualistic, Autonomous, and Integrated. The modal stage for younger teens is the Self-Protective level and for older teens, the Conformist (Westenberg & Gjerde, 1999). It was also used as an outcome measure in a prosocial intervention (Ralph, 2015a). An increase in the score would indicate changes in a prosocial direction.

Counselor.
IOWA Conners Scales:
This instrument was completed by the counselor and had two scales, each of which had five items. They were the Aggression, and the Inattention/Overactivity scales. Loney and Milich (1982) reported test-retest reliability coefficients of 0.86 for Aggression, and 0.89 for Inattention/Overactivity. The scales have been used as an outcome measure in numerous studies including Ralph, Oman, and Forney (2001). These scales were chosen because of their extensive research and normative basis, and that A decrease in scores regarding impulsivity, inattention, uncooperativeness, and defiance, might presumably show changes as a result of a prosocial intervention.

Prosocial Attitudes Questionnaire:
The Prosocial Attitudes Questionnaire/Counselor (PAQ-C) was filled out by the counselor, rating the youth's behavior regarding prosocial thinking and behaviors. This measure has 11 items, and was developed by the author for the purposes of the study. Its purpose was to assess changes in prosocial attitudes and behaviors which the Being a Pro workbook might change in a prosocial direction. The development and psychometric properties of this and a companion scale completed by the youth are discussed in Appendix 1. The individual items for the scale were developed by examining outcome research with ART discussed above including the focus groups and objective instruments, and the content of the PRO and WSCT noted above and their description of prosocial thinking. An increase in the score would indicate changes in a prosocial direction.

Youth.
The Prosocial Attitudes Questionnaire/Youth (PAQ-Y) is filled out by the youth, rating the youth's self-report of behavior regarding prosocial thinking and behaviors. This is the companion scale to the PAQ-C, completed by the youth to rate prosocial thinking and behaviors, and has 11 items. Scale items were developed in a similar fashion to the PAQ-C. An increase in the score would indicate changes in a prosocial direction.

Research Design

The design used for this study was a one sample Pre/Post Test Design, that is an assessment before the intervention, and another with the same measures after the intervention. This model has several limitations, especially compared to a randomized controlled trial. The major limitation of this model is its limited ability to rule out rival hypotheses, that is whether some factors other than the treatment caused the observed changes. These limitations, or threats to validity are usually conceptualized as history/maturation, testing, and statistical regression.

Statistical Analysis

Initially a multivariate procedure, the Hotelling Paired T-Test (Hintze, 2013) was used with all the outcome variables to compare Pre and Post measures, and the significance of changes. A randomization procedure was used which allows for the use of ordinal data and is more appropriate for small sample sizes. The null hypothesis used in this analysis was that all differences from Pre to Post were zero or no change on any variable. If the analysis showed that the null hypothesis could be rejected, then a comparison of individual variables from Pre to Post could be conducted. The Wilcoxon Test, a nonparametric paired T-test, was used for Pre/Post test comparisons (Hintze, 2013) with a one-tailed test in the direction of the hypothesized change in a prosocial direction, and a .05 level of significance. In addition, an effect size was calculated for statistically significant results only.

Additionally, for significant results only, an analysis of variance using a GLM technique was done ad hoc to assess if the change scores between Pre and Post were significantly correlated with either age or ethnic group (Hintze, 2013). Age was a continuous covariate, and ethnic group was used as a binary variable for White (W) versus all Other (O). This was done to see whether age or ethnic do group might be differentially associated with treatment outcomes.

Results

The analysis of all outcome measures comparing the Pre and Post scores, using the Hotelling Paired T-test is shown in Table 1. The results showed P<.0001, and the null hypothesis of all means being zero, or no significant change on all variables from Pre to Post test could be rejected. This result permitted individual comparisons of each outcome measure. Results are organized by the type of measure (Performance, Counselor, and Youth) reflecting the various components of the multi-method, multi-informant methodology. The results using the Wilcoxon Test are in Table 2.

The PRO-short form was the only measure that didn't show a significant Pre to Post change. This was the short version of this measure, using only one of five vignettes, and while it showed a small increase, this was not large enough to be statistically significant. All other measures showed a significant change. The largest treatment effects were for counselor ratings of changes with the IO, Ag, and PAQ-C.

The analysis of variance using a GLM technique was done for significant results only, using age, and ethnicity (White vs. Other) is show as the last column in Table 2. The results of significant results from the ANOVA analysis are in Table 3. Two variables showed significant correlation with the change scores of outcome variables. For the WSCT the change in Other was greater than White. For IO the change in White was greater than Other, and also younger teens had greater change. This suggests that while age and ethnicity had effect on some variables, it was not pervasive for all outcome variables, and doesn't suggest that the intervention was effective for only a particular ethnic or age group.

Discussion

The present research describes an evaluation using a simple Pre/Post design of a workbook formatted intervention, Being a Pro. The workbook was designed to be used in a one-to-one counseling relationship, over a 10 week period. Those participating in the study were required to take a 1.5 hour training and pass a competency test regarding using the workbook. The workbook was based on a large body of existing research regarding the development of prosocial reasoning in juveniles generally, probation youth specifically, and characteristics that contributed to harmful behaviors. Prior quantitative and qualitative research had identified the characteristics of prosocial development in these populations and interventions which promote this development. The research described results of the Pre/Post test changes regarding measures obtaining samples of prosocial reasoning by youth, rating scales by counselors, and self-rating scales completed by the youth. Five of six scales showed significant changes from the Pre to Post test, indicating higher levels of prosocial reasoning and behavior. This is consistent with the hypothesis that the Being a Pro workbook would promote such changes. All three measures completed by the counselor showed a large to very large treatment effect. The results from the WSCT, a measure of prosocial reasoning, and the PAQ-Y, a self-report measure of prosocial attitudes and behavior, showed a small to medium effect size. That changes were observed in all three assessment domains, Counselor, Youth, and Performance, supports the hypothesis that the changes observed were related to the intervention.

The model of prosocial development in adolescence discussed here is distinct from that used in ART and MRT, although similar. Other prominent models of treatment of probation youth such as Mindfulness, Dialectical Behavior Therapy, or Multi-Systemic Therapy don't include such models of prosocial development, even though the presumed end result of such treatment approaches would be an adolescent who can make prosocial choices in the future that don't have harmful outcomes and benefit the youth. It is reasonable, however, to believe that the beneficial outcomes some of these programs demonstrate result in promoting increases in prosocial reasoning which causally contribute to positive outcomes. For example, Dialectical Behavior Therapy with probation juveniles in part helps assist juveniles develop more sophisticated and flexible problem-solving strategies rather than use simplistic or impulsive approaches (Carr, Fitzgerald, & Skonovd, n.d.). This would be very similar to the approach used in Being a Pro, and might provide an alternative set of techniques to achieve the same end, increases in prosocial reasoning.

The development of the workbook was consistent with APA guidelines for evidence-based practice generally, and specifically with best practices for probation youth using individual skill building and fidelity methods (Tennyson, 2009; Lipsey, Wilson & Cothern, 2000). The rationale for the use of evidence-based methods is that since resources are always finite, it is best to use proven and effective methods, based on rigorous research methodology, the most reliable of which are randomized controlled trials (Cochrane, 1972). Additionally, current research supports that interventions such as the one studied here, with this population are likely to be effective. The workbook promotes concrete skill improvement on a criminogenic risk factor, prosocial reasoning. Also Pre/Post test measures which were used in the study are available and can assess whether there was a treatment effect for an individual, or for a program as a whole. Consistent with goals for public health interventions, the workbook intervention is inexpensive (less than $10), is completed in 10 sessions, training time for staff is modest and is done online in 1.5 hours, and had a substantial treatment effect.

The present research study has several limitations. The most prominent of these is the Pre/Post test design. The limitations of this model in contrast to other more powerful designs, especially a randomized controlled trial, are well described. Factors other than the treatment may account for the Pre and Post test differences. While only a more powerful design, such as a randomized trial, would rule out rival hypotheses, the threats to the validity of this study can be briefly addressed. The magnitude of the changes observed in this intervention, the majority of which were a large to very large treatment effect, appear to be greater than would be expected by other factors or rival hypotheses (e.g., maturation, history, testing, etc.). Significant changes were also observed across various types of measures (Performance, Counselor, and Youth) further suggesting that the changes were not attributable to some rival hypothesis such as enthusiasm of the youth or counselor for a new method and overrating positive results. Additional analyses were also conducted and none suggested specific rival hypotheses were supported. More research, using more powerful designs, with more subjects, would need to be carried out ideally with other researchers to assess the robustness of the findings reported in this study.  

Appendix 1

Since these are newly developed instruments additional statistical analyses were conducted to assess their psychometric properties. Out of the original twelve items, one item was deleted, which then left 11 items, that after initial psychometric evaluation, didn't contribute to outcome differences or interrater reliability. The Standardized Cronbachs Alpha was used to assess the amount each of the 11 items correlated with others and whether the scale items as a whole might measure a similar characteristic. The results for the PAQ-Y for the Pre-test was 0.833618, and for the Post 0.804103, and for the PAQ-C was Pre = 0.866147, and Post = 0.862520. One reference describes that a score greater than 0.70 is required for psychometric instrument use (“Cronbach’s alpha,” n.d.).

Also an analysis was done to see if the Pre and Post test scores of each measure correlated. The intervention of using the Being a Pro occurred between the Pre and Post test, so it was not a simple test-retest statistic, but it would be presumed that there were would be some correlation. The Pre-Post Spearman Correlation statistic was used, and for the total score for PAQ-Y was 0.220112, (P=0.178156, NS), and for the PAQ-C was 0.388124 (P=.015).

The Pre and the Post tests for the PAQ-Y and PAQ-C were analyzed to see if these measures were associated. Each measure was intended to measure prosocial attitudes and behaviors of the youth, based on the youth’s self-rating for the PAQ-Y and the counselor’s rating of the youth with the PAQ-C. While the underlying characteristic being assessed was the same, prosociality, there were different raters, the youth and the counselor. The Spearman Correlation statistic was used. The Pre correlation was 0.220924 (P= 0.176520, NS). The Post correlation was 0.325440 (P= 0.043212). The higher Post correlation might occur because the counselor and youth developed a better shared understanding of the youth's behavior by the end of the 10 workbook sessions.

The IOWA Conners Aggression Scale (Ag), a measure of aggressive and oppositional behaviors was correlated with the PAQ-Y and PAQ-C, since these measures were assumed to be similar, though not identical. If PAQ-Y and PAQ-C measure prosocial attitudes they might correlate with the Ag measure, and indicate convergent validity. Both the relevant Pre and Post scores for the Ag, PAQ-Y and PAQ-C were used. The Spearman Correlation statistic for the Pre-test of the Ag, and PAQ-Y and PAQ-C was 0.339144 (P= 0.034679), and 0.788257 (P<.000001), respectively. The same statistics for the PAQ-Y and PAQ-C Post test measures were 0.122311 (P= 0.458225), and 0.546553 (P= 0.000318) respectively. As can be seem from the analysis, the PAQ-C had Pre and Post test correlated significantly with the Ag score, but the PAQ-Y had only significant Pre-test correlation, less than the PAQ-Y. The Post test PAQ-Y had a positive, but non-significant correlation. It should be noted that the Ag score, like the PAQ-C is completed by the counselor whereas the PAQ-Y is completed by the youth so that correlation would be assumed to be higher.

Notes

¹ Effect size statistics give an indication of the magnitude of the change which isn't provided by the significance level alone, and relates to the magnitude of the difference in real world terms. An effect size of 0.5 is described as "medium" and is "large enough to be visible to the naked eye" (Cohen, 1988), and an effect size corresponds to the difference between the heights of 14-year-old and 18-year-old girls. Effect size was calculated as the difference between the Pre and Post means, divided by the standard deviation. Following usual conventions, effect sizes from .20 and .50 were classified as small to medium (S-M), from .50 to .8, medium as large (M-L), from .80 to 1.3, large to very large (L-VL), and over 1.3 was classified as very large (VL).

² Specific rival hypotheses were examined and some analyses were conducted. History/maturation refers to time related factors. A simple linear regression was used with age as the dependent variable. Significant outcome variables were used as the independent variables (WSCT, IO, Ag, PAQ-P, and PAQ-C). Analyses were done separately for Pre and Post samples. None but PAQ-Y were significantly associated with age. Testing refers to possible practice effects, such as taking the test more than once would increase skills and hand score on the target score. Statistical regression, refers to the effect such as choosing an extreme population, who would become improved anyway without any intervention. Testing or practice effects didn't appear to be likely with these measures, particularly counselor ratings of youth. Likewise, statistical regression didn't appear to be a factor since the youth were not an extreme subset of probation youth and prosocial thinking and behaviors would not likely spontaneously improve over such as short time frame. A randomized design or equivalent design would rule out these threats to validity more definitively, however (Ohlund & Chong-ho, n.d.).

References

1. Antonowicz, E. (2005). The reasoning and rehabilitation program: outcome evaluations with offenders. In M. McMurran & J. McGuire, Eds. Social problem-solving and offending: evidence, evaluation and evolution (pp. 163-182). Hoboken, NJ: John Wiley & Sons, Ltd.
2. Bruner, J. S., & Anglin, J. M. (1981). Beyond the information given: Studies in the psychology of knowing. London: Allen & Unwin.
3. Burnette, K. D., Swan, E. S., Robinson, K., Woods-Robinson, M., & Little, G. L. (2004). Treating youthful offenders with moral reconation therapy: A recidivism and pre-posttest analysis. Cognitive-Behavioral Treatment Review, 3(4), 14-15.
4. Bush, J., Glick, B., & Taymans, J. (1997). Thinking for a Change. Longmont, CO: National Institute of Corrections, United States Department of Justice.
5. Carr, L. J., Fitzgerald, T., & Skonovd, N. Dialectical Behavior Therapy: Evidence for Implementation in Juvenile Correctional Settings. Retrieved December 2, 2016, from www.cdcr.ca.gov/Reports_Research/docs/DBT_Evidence_Draft_04_06_2011.pdf
6. Cochrane, A. (1972). Effectiveness and efficiency: Random reflections on health services. London: Nuffield Provincial Hospitals Trust.
7. Copley, B. (1993). The word of Adolescence, Free Association.
8. Cognitive Centre of Canada. Retrieved November 20, 2016, from R&R2 Short Version For Youth: A Prosocial Competence Training Program, cognitivecentre.ca/rr2youth.
9. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.
10. Cronbach’s alpha - measurement of internal consistency. Retrieved November 20, 2016, from https://explorable.com/cronbachs-alpha
11. Curran, J. (2006). The effect that programmes can have on reducing antisocial behaviour. Applied Forensic Psychology Dissertation. University of Leicester.
12. Farrington D 1986. Age and crime, in Tonry M & Morris N (eds), Crime and justice: An annual review of research. Chicago: University of Chicago Press: 189-250.
13. Ferguson, L. M., & Wormith, J. S. (2013). A meta-analysis of moral reconation therapy. International Journal of Offender Therapy and Comparative Criminology, 57:1076-106.
14. Goldstein, A. (1999). The prepare curriculum: Teaching prosocial competencies. Champaign, IL: Research Press.
15. Goldstein, A. P., Nensén, R., Daleflod, B., & Kalt, M. (Eds.). (2005). New perspectives on aggression replacement training: Practice, research and application. John Wiley & Sons.
16. Goldstein, A., Glick, B., & Gibbs, J. (1998). Aggression replacement training. Champaign, IL: Research Press.
17. Farrington D. (1986). Age and crime, in Tonry M & Morris N (eds), Crime and justice: An annual review of research. Chicago: University of Chicago Press: 189-250.
18. Hintze, J. (2013). Number Cruncher Statistical Systems (NCSS) 2009. Kaysville, UT: NCSS.
19. Hy, L.X., & Loevinger, J. (1996). Measuring ego development (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
20. Jarrett, M., Van, A., Meter, Youngstrom, E., Hilton, D. & Ollendick, T. (2016). Evidence-Based Assessment of ADHD in Youth Using a Receiver Operating Characteristic Approach, Journal of Clinical Child & Adolescent Psychology, 24:1-13.
21. Kim B, Benekos, P. J. & Merlo A. V. (2016). Sex Offender Recidivism Revisited: Review of Recent Meta-analyses on the Effects of Sex Offender Treatment. Trauma Violence Abuse. 17: 105-17.
22. Leversee, T., & Powell, K. (2012). Beyond risk management to a more holistic model for treating sexually abusive youth. In B. Schwartz (Ed.), The Sex Offender, Vol.7 (pp. 19-1 - 19-24). New York, NY: Civic Research Institute.
23. Lipsey, M. W. (2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims and Offenders, 4, 124-147.
24. Lipsey, M.W., Wilson, D.B., & Cothern, L. (2000). Effective intervention for serious juvenile offenders. Juvenile Justice Bulletin, _, 1-7.
25. Little, G. L., & Robinson, K. D. (1988). Moral reconation therapy: A systematic step-by-step treatment system for treatment resistant clients. Psychological Reports, 62(1), 135-151.
26. Loney, J.& Milich, R (1982). Hyperactivity, inattention, and aggression in clinical practice. In D. Routh & M. Wohaich, Advances in developmental and behavioral pediatrics (pp. 113-147). Greenwich, CT: JAI Press.
27. Norcross, J. C., & Lambert, M. J. (2011). Psychotherapy relationships that work II. Psychotherapy, 48(1), 4-8. doi:10.1037/a0022180
28. Ohlund, B. & Chong-ho, Y. Threats to validity of research design. Retrieved November 20, 2016, from web.pdx.edu/~stipakb/download/PA555/ResearchDesign.html
29. Ralph, N. (2007). Psychological assessments and factors in juvenile probation. Presentation to Juvenile Probation Department, San Francisco, CA.
30. Ralph, N. (2012). Prosocial interventions for juveniles with sexual offending behaviors. In B. Schwartz (Ed.), The sex offender: Issues in assessment, treatment, and supervision (pp. 18-29). Kingston, NJ: Civic Research Institute.
31. Ralph, N. (2015a). A follow up study of a prosocial intervention for juveniles who sexually offend. Sex Offender Treatment. 10. Retrieved December 16, 2015, from www.sexual-offender-treatment.org/sot-1-2015.html.
32. Ralph, N. (2015b). A longitudinal study of factors predicting outcomes in a residential program for treating juveniles who sexually offend. Sex Offender Treatment.
33. Ralph, N. (2016). An instrument for assessing prosocial reasoning in probation youth Sex Offender Treatment.
34. Ralph, N. B., Oman, D., & Forney, W. (2001). Treatment Outcomes with Low Income Children and Adolescents with Attention Deficit. Children and Youth Services Review., 23(2), 145-167.
35. Ralph, N. & Wong, K. (2013). A prosocial collaborative model for juveniles who sexually offend. ATSA Forum, V(1).
36. Ross, R. R. & Hilborn, J. (2003). R&R 2: SHORT Version for Youth. Ottawa: Cognitive Centre of Canada, cogcen@canada.com; Cardiff: Cognitive Centre Foundation, Gaynor@cogcen.com).
37. Stams, G. J., Brugman, D., Deković, M., van Rosmalen, L., van der Laan, P., & Gibbs, J. C. (2006). The moral judgment of juvenile delinquents: A Meta-Analysis. Journal of Abnormal Child Psychology, 34(5), 692-708.
38. Steinberg, L. D. (2014). Age of opportunity: Lessons from the new science of adolescence. New York: Houghton Mifflin.
39. Tennyson, H. (2009). Reducing juvenile recidivism: A meta-analysis of treatment outcomes. (Doctoral dissertation, School of Professional Psychology Pacific University, 2009). Retrieved from commons.pacificu.edu/spp/109.
40. Washington State Institute for Public Policy (2004). Outcome evaluation of Washington State’s research-based programs for juvenile offenders. Retrieved from www.wsipp.wa.gov/ReportFile/852
41. Watson, M. (2002). Theories of Human Development. Chantilly, Virginia: The Teaching Company.
42. Westenberg, P. M., & Gjerde, P. F. (1999). Ego development during the transition from adolescence to young adulthood: A 9-year longitudinal study. Journal of Research in Personality, 33(2), 233-252.

Author address

Norbert Ralph, PhD, MPH
519 Estudillo Ave. #N
San Leandro, CA 94577
norbert.ralph@yahoo.com