Advancements in Assessment Tool Selection

REACH recently underwent the grueling process of recertification with BHCOE (Behavioral Health Center of Excellence) with a positive result, attaining a 2-year accreditation. The process encourages us to look at our clinical and administrative processes and procedures and reflect on what we can do better, how we can grow and develop as professionals and how we can strive for excellence in treatment and training.

Something that stood out for me through this process was the subject of assessment, how we assess?and what measures we use to assess and develop treatment plans. In my career to date, I have not used standardized assessment tools since my Master’s thesis days. Since entering the world of Applied Behavior Analysis with it’s single-case studies and focus on the behavior of the individual, it did not occur to me that there was a place for standardized assessments in ABA treatment in early intervention settings.

Through the BHCOE recertification process, I learned a lot about standardized assessment and how it can compare and contrast to treatment outcomes. In the land of insurance in the US, standardized assessments are commonplace in assessment and progress reporting practice. This was news to me until REACH became a Tricare overseas provider this summer. Even then I didn’t consider the value of these additional assessments, it was a requirement so we completed them and that was it.?

Using standardized assessments as part of my assessment process never occurred to me as treatments based on ABA are highly individualized to the specific needs of the client. Therefore, instead of using standardized measurement tools, ABA treatment providers have historically relied on their training and experience to? target and measure specific behaviors to increase or replace. In early intervention settings, where I have always worked, criterion referenced assessments of language and development are common place (e.g., VB-MAPP, PEAK, EFL), but not standardized measures such as the Vineland, PDDBI, etc. Or so I thought.??

What I have learned is this. With individualized data, it is difficult to compare outcomes of idiosyncratic approaches across patients and across providers. To do this, practitioners must agree on common data definitions and methods of data collection to measure the overall treatment gains across clients and to predict rate of improvement for clients with similar behavioral presentation.?

Standardized measurement tools provide common definitions and methods of data collection that allow for large-scale, cross-patient comparison of adaptive skills changes and socially significant gains. Standardized instruments offer the field a unified approach for a more global analysis of the effectiveness of the general approaches that ABA practitioners take.

With criterion-referenced assessment scores, the client’s performance is compared with an objective criterion. For example, the VB-MAPP uses criterion-referenced interpretation of scores, and a milestone score of 130 out of 170 for a 4-year-old client cannot inform if that raw score of 130 is average, below average, or above average for the 4-year-old client because there is no information on how a sample of typical 4-year-old children would perform on the VB-MAPP (Sattler, 2014).??

Tools that rely on norm-referenced interpretations allow comparison of the patient’s performance with the performance of same-age peers that represent a norm group or standardization sample (Sattler, 2014). Norm-referenced assessments allow for the comparison of a client’s performance to same-age peers, which provides valuable information about the level of deficit.? A norm-referenced interpretation answers the following question: “How does the examinee’s performance on a specific skill compare to others of the same age?”. The evaluator can determine if the child’s performance is within, below, or above the average range compared with the performance of all 4-year-olds in the standardization sample (Raynolds & Livingstone, 2012).??

In short, more robust measures of progress and achievement can be determined by selecting a combination of criterion-referenced and norm-referenced assessment tools. They can answer questions like; What are the clients’ rates of improvement? What are the types of clients (e.g., age, reason for referral) that they are better or worse at treating? Which supervisors are performing better or worse in treating different clients? Ultimately this will lead to improvements in quality of services and outcomes for all.

The BHCOE have an informative White Paper on Selecting Appropriate Assessment Instruments to provide a systematic approach for selecting instruments to assess and plan treatment for individuals with autism spectrum disorder (ASD); and second, inform data collection and reporting on treatment outcomes. This can be requested via their website: www.bhcoe.org.

By Sinéad Ní Fhloinn, Clinical Director at REACH Behavior and Development Center.

References

Sattler, J. M. (2014). Foundations of behavioral, social and clinical assessment of children. Jerome M. Sattler.

Reynolds, C. R., & Livingston, R. B. (2012). Mastering modern psychological testing: Theory and methods. Pearson Education.