How to Fail Successfully (a.k.a Agile Instructional Design)

Last year, I attended a fantastic Meetup event hosted by Ladies in Seattle Tech where the topic was “Learning from failure: I made a mistake and everything was okay.” It featured local women who talked about failures they experienced at work and the lessons they learned, and it got me thinking a lot about how I view failure. My internal dialog was something like, “Hmm, I don’t really fail.” followed quickly by, “Oh wait, I fail all the time. But I don’t view it as failure. Why is that?”

I fail all the time.

The Trainer’s Dilemma: Is Failure an Option?

My perception of failure has a lot to do with my industry: Learning & Development. There is no template for failure-proof instructional design.  Anyone can to identify the elements of bad design – raise your hand if you’ve experienced Death by PowerPoint – but good design is a creative endeavor that can take a multitude of shapes. The only way to truly know whether your design is successful is through testing with a live audience, preferably students who actually need your training for job success. But live testing brings up a terrifying thought:

“What if I deliver bad training and it hurts someone’s chances of professional success?”

Is failure an option in this high stakes environment? Yes! When the goal is to produce amazing educational experiences, failure will be an inevitable and integral part of that journey. After all, how can we expect to be innovative if we never try anything risky? But, you ask, what about the risk of hurting someone’s career? 

My answer to this struggle is a form of Agile Project Management, often called Agile Instructional Design (or Agile Learning Design). I have found it to be an excellent way to experience “little failures” that help my team polish content while ensuring the impact on students is minimal.

Project Management for Instructional Design

While instructional designers typically don't have formal education in project management, most are familiar with the ADDIE method (Analysis, Design, Development, Implementation, Evaluation) which is really just L&D speak for traditional (waterfall) project management. Whether you're talking ADDIE vs. SAM or waterfall vs. agile, these terms describe the same debate. Should you...

Fully design a course before rolling it out to the audience?

Design the course in small chunks and roll it out gradually?

I think this graphic does a wonderful job of illustrating the differences between the two options. (note: It's all over the web without any credit- if you know who created it let me know!)

In waterfall (a.k.a. ADDIE), all of the design is completed prior to any kind of release. With agile (a.k.a. SAM), we design small parts over the same time period, rolling out a little bit with each session.

Agile Instructional Design Success Story: Process

My team's flagship course is a two-week boot camp on the basics of evaluating U.S. passport applications, delivered to new hires as part of their formal onboarding. Evaluating passport applications requires a foundation in U.S. citizenship law and policy that cannot be gained through any prior work experience. Because the students come in as blank slates, our team mentality (mine included) had always been that an "info dump" was the only solution, therefore the class was 60-70% lecture. While the class wasn't "broken," the marathon of lectures was taxing on students and instructors alike- and worse, we feared we weren’t giving them the best education possible. While the entire team agreed that it was time for a revision, there were two major issues to contend with…

With a the team spread across the United States, and all of the instructional designers doubling as our trainers, our traditional course development method (waterfall/ADDIE) would take 1-2 years to complete. Out of frustration, I had a thought: what if we could change over the course in small increments, and if we ever felt like we'd gone too far, it would be easy to revise and recover? At the time, I had no idea that there was a term for that kind of design (agile). With my team's support, we decided to venture into unfamiliar design territory. Our agile design schedule broke down by days:

Day 1: This would be the foundation of the new version, focusing on "effective decision making," and had to go through three iterations before we were happy with the content.

Days 2-5: Since Days 2-5 had to build on our "effective decision making" content, the basic Day 1 framework and initial test results had to be executed before we could finalize the first draft for testing. Days 2-5 were completed after only two rounds of testing.

Days 6-7: These days represented the most challenging and risky content changes, and would remain static until all of the other content was revised. Once we found success with redesigning the easier Days, we were able to tackle the content over the course of 4 months. (not shown in image)

Days 8-10: Required substantial subject matter expert (SME), which lengthened the initial development period. Content finalized after only one round of testing.

Using what I later learned were agile techniques, we were able to successfully redesign 80% of the course over the span of about 6 months! Had we used waterfall instead, that timeline for just that portion would have stretched close to a year- simply because it would have taken at least 3-4 months to perform just the initial design.

Agile Instructional Design Success Story: Student Experience

In evaluating the success of our agile redesign, we relied heavily on stakeholder input from instructors and students. For each session where new content was tested, we collected/measured:

Student Feedback: We measured student reactions by having instructors query them after experiencing a new activity, as well as through periodic confidence and feedback surveys (see Level 1 Evaluation). The students overwhelmingly told us that they felt constantly engaged by the training and had gained confidence in their job skills. Win!!

"I am pleasantly surprised by how much I have been able to absorb in such a short period of time. I knew next to nothing before I began the class. How comfortable I feel with the material at this point is a testament to the professional excellence of my teachers. And I feel I am infinitely better than I was when I started."

"I expected the class to be dry because of the amount of information we had to cover, but every trainer made it fun and exciting while we were learning."

Student Success: In a lecture-heavy class, it is very challenging (impossible?) to assess during training whether students comprehend the content. Our new model uses a Read, Analyze, Apply methodology, which requires students to Read an excerpt of law/policy, Analyze through answering questions and group discussion, and Apply to sample passport applications. In addition to improved learning, it also provided multiple points where instructors could identify struggling students and intervene as necessary. By keeping track of success rates with individual activities, we were able to tailor the content during revision periods to support comprehension and retention. Win!!

Instructor Observations: Building on the students' feedback and success rates, we debriefed instructors after each session. While sometimes there were frustrating activities that just didn't work (i.e. failures), the instructors' observations supported what we were seeing from the students: comprehension and retention improved. And because the style was more engaging, instructors had more fun in the classroom and generated positive energy. Win!!

Benefits of Agile Instructional Design

The team has learned to love agile because it let's us try out new ideas with a lower risk of failure, and I think it's especially beneficial when embarking on the redesign of an existing course. Some ideas will definitely fail, but it's much easier for a trainer to recover from a small setback as opposed to coping with an entire course that isn't meeting expectations. Of the many benefits, I would highlight the following as being most likely to win allies in a transition to using agile:

Learner-Focused: The learners are engaged in the design process. Each time we test a segment, we tell the class that we are trying new content and ask for their input. Learner feedback drives the next iteration.

Produces Usable Products Quickly: Unlike traditional waterfall design, where it might take months or years to get a usable product, with agile you can potentially get a new usable product with each iteration. These "small wins" energize the instructors and allow change to happen gradually, and therefore in a way that is easier to absorb.

Reduces Need for Substantial Rework: Because the work is completed in small packages, the changes needed after each session are typically minimal. Some ideas will be scrapped, but it's rarely a total rewrite. Changes can be made quickly and be ready for implementation in the next session.

How to "Go Agile" & My Tips for Success

When you start to do research on agile project management, it can get very intimidating very quickly. Keep something in mind: Agile is a concept and there are many ways to execute it (e.g. Scrum, Lean, Kanban). Since I didn't even realize I was managing an agile project, I didn't use any specific methodology and everything turned out just fine. There are plenty of resources on applying agile to L&D projects, and my opinion is there is no one right way. While it would be awesome to have a Scrum Master to lead your agile projects, that isn't always realistic (especially for smaller L&D organizations). Never fear: you can start to apply the principles of agile to your next L&D project with little to no cost. Do a little bit of research, interview people who use agile, and pick a process that works best for your team and organization. 

If your team is accustomed to ADDIE/waterfall instructional design, you will need to employ change management techniques to generate buy-in and engagement. I suggest including the following in your strategy:

Train Your Stakeholders: Once you have picked a methodology, hold some training sessions to explain concepts and answer questions. Ideally, show the team a case study of successful agile instructional design. Above all, make sure they understand that failure in the classroom is expected and acceptable!

Pick One Project: While it's tempting to go all in, I suggest starting with one project so that your team can experience the new methodology and become believers. Maybe you don't need to be crazy like me and pick your biggest project- but you do you!

Be Transparent: Provide regular updates to the team on progress. Celebrate successes and failures!

Good luck and let me how your journey goes!