How Do You Know Your Crisis Management Training is Successful?

There are a number of educators struggling with behavior issues ranging from off-task behavior to aggression and even self injury. They want students who are respectful, engaged, and ultimately meet their learning potential on a daily basis. As clinicians and educators, it’s been our observation that many times schools and districts inadvertently waste precious resources by throwing training at these behavior problems; unfortunately, many times the solution isn't training as the issue isn’t related to a skill deficit, but rather a performance deficit (i.e. it’s not a can’t do, but a won’t do).?

However, sometimes the issue really is a skill-deficit.?Under those conditions, training?is?needed.?One problem that continues to plague education in regards to either reducing behavior issues or increasing student achievement is getting a return on investment from professional development. Too often, training is provided as a one-shot endeavor with high (and unrealistic) expectations for achieving the desired result of people performing precisely what was taught in the training. And that's assuming what they are being taught in an evidenced based curriculum using evidence based training procedures!

Even if the curriculum is complete and being delivered rooted via evidence-based principles and procedures (which most are not), in order for professional development to be effective, a systematic and scientific approach for delivery and assessment should be in place to ensure learning occurs.?Kirkpatrick and Kirkpatrick (2007) suggest professional development be assessed and managed in four different levels: (a) reaction, that is how did the participants feel about it, (b) knowledge, what did they learn (c) behavior, what are they actually doing more, less, or differently in the classroom as a result of the training(d) and evaluation, did the changes in teacher behavior have the desired result on student achievement.?

Most behavior or crisis management training organizations only measure one of these: reaction. But that's like going to a surgeon and saying she was successful because she has a good bed side manner, but not looking at the outcomes of the surgery! Let's take a look at the measurements.

Level 1: Reaction

Reaction entails assessing social validity in terms of trainees’ perceptions of the actual delivery and significance of the professional development. For example, did they perceive the professional developer as engaging and knowledgeable, and was the content provided meaningful and pertinent to their job as it relates to organizational goals? These aspects of professional development are powerful and can make or break change initiatives. An engaging presenter can capture trainees’ attention and strengthen their motivation, particularly when participants find the information useful; however, a boring presenter might slow an initiative as those attending find it difficult to absorb the information. Moreover, content that is not directly meaningful to the audience and information that is delivered by a boring presenter might halt well thought out change initiatives.?

This measurement can be provided as a survey at the end of each training session. In addition, a comprehensive survey should be provided at the end of the end of the professional development program delivered. The purpose of this survey is not to measure knowledge or skills learned. However, the survey should help you determine the participants’ perception of the content as well as the trainer’s ability to effectively communicate concepts within the training environment. This will be useful for assessing and modifying the delivery of professional development.

Level 2: Measuring Knowledge

While a presenter may give an amazing presentation and captivate the audience,?learning?should be assessed to make sure the stage is set for a change initiative to occur. In order for learning to be assessed, simple pre-tests and post-tests might be delivered with the intention of measuring knowledge gained from the targeted professional development. Questions can be multiple-choice and the pre-test and post-test can be compared and used as a measure of knowledge learned.?But most importantly, the participants skills should be measured through performance and competency-based assessments which require them to engage in the targeted skills being taught.?It's not enough to only “know” what to do or even when to do it.?Participants need to be able to actually do it!

Level 3: Measuring Behavior

Even if the training was received well, and the participants demonstrated improvement in the training environment, it still comes down to behavior in the educational setting.?Are folks actually doing what they were trained to do which should be measured through direct observation.?The overwhelming answer here is typically…no.?They aren’t.

With the stage set through the initial professional development, follow up coaching becomes critical for success in order to achieve sustainability with change.?Even with the best training outcomes, this is where the rubber meets the road.?Unfortunately, this is also where ROI (return on investment) tends to fall short as most schools and organizations do not have an effective post-training system in place to ensure knowledge and skills learned in training effectively transfer into the work environment.

Level 4: Evaluating the Results

That is, did the cost of the training provide worth outcomes.?Was it worth it? While it is important to use the data gathered in levels one, two, and three to assess and plan, most organizations must measure their critical success factors as an evaluation of return on investment. These factors might include reductions in behavior issues, reductions in restraints, increases in morale, decreases in absenteeism, decreases in grievances, increases in student achievement, and more!

In the end, if schools do a thorough performance diagnostic and determine training?is?needed for key staff, they must also then determine if the training provided was?effective.?By measuring reaction, knowledge, behavior, and evaluating return on investment in terms of results, you will have good data that ultimately allows you to evaluate and shape your professional developments efforts.?

At?PCMA, our crisis management curriculum is?complete?and grounded in?behavior science; moreover, our training approach is grounded in behavior science, and the principles and procedures we teach are grounded in this same science. And to cap it all off, we have a?9 step approach?grounded in Organizational Behavior Management (i.e., the science of human behavior for making a positive difference in organizations) for supporting the transference of skills learned in the training environment to the effective application within the educational setting.

Check out our video on what a complete crisis management is made up of, and check out others that include using the 9 steps to get your crisis management to stick. If you have any questions, reach out to [email protected].

References

Kirkpatrick, D. L., & Kirkpatrick, J. D. (2007).?Implementing the four levels: A practical guide for effective evaluation of training programs. San Francisco: Berrett-Koehler Publishers.