Brief #10 – Comprehensive Multirater assessment of students’ SEL skills

Path:
← Back

Intervention Simplified

Comprehensive multirater assessment of students’ social–emotional learning skills

Stephen N. Elliott, PhD

Assessments are an essential aspect of all interventions. Screening assessments completed by teachers are helpful at the outset of a universal intervention designed to teach social–emotional learning (SEL) skills. However, when students do not respond as expected to an intervention, a more comprehensive assessment of their SEL skills is recommended to pinpoint improvement areas. The comprehensive nature of such assessments usually involves covering a wider range of desired skills, including perspectives of others (e.g., parents and students themselves), and more complex scoring and reporting of results to guide decisions about subsequent interventions.

A challenge for many educators involved with SEL interventions today is selecting a comprehensive, valid assessment that can help them (1) reliably identify children’s SEL strengths and weaknesses and (2) directly link results to specific interventions to improve SEL skills. This intervention brief describes key features of a multirater assessment, the SSIS™ SEL Edition Rating Forms (Gresham & Elliott, 2017) that accomplishes these assessment objectives and more.

The SSIS SEL edition rating forms in comparison to the SSIS rating scales

The SSIS SEL Edition Rating Forms were inspired to assess the SEL competency domains theorized by the Collaborative for Academic, Social, and Emotional Learning (CASEL, 2012) and are a transformation of the respected Social Skills Improvement System (SSIS) Rating Scales (Gresham & Elliott, 2008; Crowe, Beauchamp, Catroppa, & Anderson, 2011; Humphrey et al., 2011). Both the SSIS SEL Edition Rating Forms and the SSIS Rating Scales share the same items, were normed with the same national sample of students ages 3 to 18, and are used by multiple raters (teachers, parents, and students). Both the SSIS Rating Scales and the new SSIS SEL Edition Rating Forms are classic rating scale assessments designed to capture information of a behavior’s frequency

The accompanying figure highlights a few items and the frequency rating continuum from the SSIS SEL Edition Rating Teacher Form. Both assessments were designed with social–emotional skill intervention in mind. That is, they both facilitate identifying specific target behaviors for improvement and evaluating the effectiveness of interventions. The original SSIS Rating Scales feature the unique application of dual frequency and importance ratings to facilitate an analysis of students’ social behavior strengths, performance and acquisition deficits, and co-occurring problem behaviors. Such an assessment has been particularly useful for designing individualized Tier 2 or Tier 3 interventions. The new SSIS SEL Edition Rating Forms are available online and consist only of desired social behavior items rated for their frequency; no importance ratings are made for items, nor are there any problem behavior items because this assessment is used with SEL interventions and focus on improving desired behaviors. Thus, the SSIS SEL Edition Rating Forms require less time to complete than the SSIS Rating Scales.

A side-by-side comparison of these two assessments is provided in the figure below. An examination of this figure indicates that the SSIS SEL Edition version is the result of a technical transformation of the original SSIS Rating Scales based on the CASEL framework that theorizes there are five SEL domains: self-awareness, self-management, social awareness, relationship skills, and responsible decision making. Using this framework to guide a confirmatory factor analyses, the SSIS Rating Scales items originally designed to measure classic social skills (i.e., communication, cooperation, assertion, responsibility, empathy, engagement, and self-control) were reorganized as behaviors representative of the five CASEL competencies. As a result, empirical support for the CASEL model was provided and a more streamlined and flexible multirater assessment for use in multi-tiered SEL systems was created.

In an investigation of multiple-informant agreement, Gresham et al. (2018) found that users of the multirater SSIS SEL Edition Rating Forms can expect to find some disagreement among raters because students’ social behaviors are influenced by social situations and the environments they function in. That is, teachers and parents, teachers and students, or parents and students can be expected to rate the frequency of some social–emotional skills somewhat differently; however, on balance, the present research indicates much more agreement than disagreement, and when focusing on composite scores, the agreement levels are relatively high. Rating scales like the SSIS SEL Edition Rating Forms can be supplemented by direct observations and interviews to facilitate understanding of possible disagreements among raters and to provide more qualitative evidence about children’s social–emotional skills.

The SSIS SEL edition rating forms scale descriptions and interpretation

The SSIS SEL Edition Rating Forms (Teacher, Parent, and Student) offer five SEL subscales (i.e., Self-Awareness, Self-Management, Social Awareness, Relationship Skills, and Responsible Decision Making) that can be used to achieve a comprehensive view of a student’s social–emotional functioning and academic competence. As illustrated in the Score Profile figure, standard scores for each of these subscales and for a total composite score ranging from 85 to 115 (i.e., within ±1 SD of the mean) are considered average. Scores ranging from 70 to 84 are considered below average, and those ranging from 116 to 130 are considered above average. Scores of 69 or lower (i.e., more than 2 SDs below the mean) are considered well-below average, and scores of 131 or higher (i.e., more than 2 SDs above the mean) are considered well-above average. For all scales, higher scores are desirable because they reflect a higher level of competency in each area of social–emotional functioning or academic competence. Scores in the Below Average and Well-below Average ranges likely indicate a need for additional supports and interventions in a given area to further develop a student’s skills. In addition to the five SEL subscales, there is a Core Skills scale that consists of the 10 core skills taught in the SSIS SEL Edition Classwide Intervention Program (CIP; Elliott & Gresham, 2017a). This short scale can be used as a norm-referenced screening and outcome evaluation measure to complement the criterion-referenced SSIS SEL Edition Screening/Progress Monitoring Scales (Elliott & Gresham, 2017b).

For any SEL scale, but particularly for those that in the Below and Well-below Average range, an item level strengths and weaknesses analysis can be conducted with the SSIS SEL Edition Rating Forms to sort items into one of three categories: strength, performance deficit, and acquisition deficit. This figure highlights that the skills in each of these descriptive categories require different levels of support and intervention strategies to improve. The SSIS SEL Edition CIP provides the types of intervention strategies called for, especially with performance and acquisition deficits.

Other briefs featured in this Intervention Simplified series discuss more about the assessment of SEL skills and the evaluation of programs intended to improve them. For example, Brief #1 is about screening, Brief #8 is about integrating assessments to support a model of multi-tiered student support, and Brief #9 is about evaluating SEL program outcomes.

References

Collaborative for Academic, Social, and Emotional Learning. (2012). Effective social and emotional learning programs. Retrieved from https://casel.org/wp-content/uploads/2016/01/2013-casel-guide-1.pdf

Crowe, L. M., Beauchamp, M. H., Catroppa, C., & Anderson, V. (2011). Social function assessment tools for children and adolescents: A systematic review from 1988 to 2010. Clinical Psychology Review, 31(5), 767–785. doi:10.1016/j.cpr.2011.03.008

Elliott, S. N., & Gresham, F. M. (2017a). SSIS SEL Edition Classwide Intervention Program manual. Bloomington, MN: NCS Pearson.

Elliott, S. N., & Gresham, F. M. (2017b). SSIS SEL Edition Screening/Progress Monitoring Scales [Measurement instrument]. Bloomington, MN: NCS Pearson.

Gresham, F. M., & Elliott, S. N. (2008). Social Skills Improvement System Rating Scales [Measurement instrument]. Bloomington, MN: NCS Pearson.

Gresham, F. M., & Elliott, S. N. (2017). SSIS SEL Edition Rating Forms [Measurement instrument]. Bloomington, MN: NCS Pearson.

Gresham, F. M., Elliott, S. N., Metallo, S., Byrd, S., Wilson, E., & Cassidy, K. (2018). Cross-informant agreement of children’s social-emotional skills: An investigation of ratings by teachers, parents, and students from a nationally representative sample. Psychology in the Schools, 55(2), 208–223. doi:10.1002/pits.22101

Humphrey, N., Kalambouka, A., Wiglesworth, M., Lendrum, A., Deighton, J., & Wolpert, M. (2011). Measures of social and emotional skills for children and young people: A systematic review. Educational and Psychological Measurement, 71(4), 617–637. doi:10.1177/0013164410382896

Stephen N. Elliott, PhD, is the Mickelson Foundation Professor at Arizona State University and the co-author of the SSIS SEL Edition Assessments and Classwide Intervention Program.

© 2018 Pearson Education, Inc. or its affiliates. All rights reserved. Pearson and SSIS are trademarks, in the US and/or other countries, of Pearson plc. LRNAS15520 EL 9/18