How a user test of your document can change everything

You’ve spent all your time and energy on that one fantastic document. It’s become the greatest thing that ever happened to technical documentation. You’re very proud of the result, and couldn't be more satisfied.

But what happens after you hit the “release” button? Who will read it? Will it make a difference?

You don’t know. Not exactly, anyway.

Here’s a story of something I did to find out, and how it made me start over from scratch.

Panic

I once was working for a company that made electronic devices. One day, these devices were sent out with the wrong configuration. So wrong, that there was a very small but not zero chance that it could have harmful consequences. Fortunately, the devices had only been shipped to warehouses and were not sold to end users yet. But regardless, immediate action was necessary.

An unorthodox solution

To combat the flaw, the devices had to be connected to a computer to update the firmware before they were sold on. In a normal situation this would have been done by service engineers, but given the urgency and scale of distribution, it was decided that this task would be performed by the on-site warehouse employees.

We planned to send laptops to every warehouse that stored the devices, with the updating software running automatically. All that the warehouse personnel had to do, was connect the device with a special cable, turn on the laptop, wait for some screens to pop up and then select the correct configuration.

The perfect document

Needless to say, because this relatively technical operation had to be performed by people completely unfamiliar with the devices, this needed infallible step-by-step instructions describing the steps at the lowest level imaginable.

I spent days testing every possible outcome, on every screen, documenting the results as I went along. I made flow diagrams and many graphical or iconized instructions, so that the chance of misinterpretation was minimal. Zero even. Less than zero, even! Management was very happy with the document. A job well done. I was proud of myself and felt great.

The document was placed on top of the laptop, in a way that this would be the first thing the user saw when they opened the box. The front page already showed the first necessary steps: connecting and turning on the laptop. We had thought of everything.

User testing

The time had come to test the package. We found a willing volunteer in one of the ladies from our warehouse, and placed the package in front of her. Eagerly we awaited the outcome.

She opened the package, put the instructions to one side, took out the laptop and started fumbling with it for about 10 minutes, trying to connect it to the mains and to the device, and to turn it on.

She did WHAT???

She. Put. The. Document. Aside.

All my hard work. All the praise of my coworkers and manager. The perfect document. Perfectly useless.

I cannot express how hard it was not to intervene, and just show her that everything she needed to know was right there on the front page of the enclosed document. But I let her struggle. After some time, which felt like ages, she figured it out and started up the laptop.

Not completely useless after all

After the laptop had started up, the updating part of the firmware started. Becoming somewhat confused, our volunteer finally picked up the instructions (yes, she did! She did!!!) and after some back and forth between the paper and the screen, completed the task successfully.

At last, the document turned out to serve its purpose. When the user got stuck, it actually helped.

Lessons learned

The mission was accomplished. A non-technical person had managed to update the firmware of the device. However, 10 minutes were wasted fumbling with the cables. Knowing how great my guide was, I never would have guessed that not reading it was even an option. But it was. Of course it was. People just don't have time to RTFM.

We eventually decided to print the cable and laptop information on stickers, and attach these to the laptop and the cables. That way, the instructions were physically available where and when the user needed them.

In addition we added some extra on-screen help, so that the user didn’t need to look away from the screen quite so often.

Swallowing my pride

What was left of the document after all these changes was only a bare framework of the supposedly perfect original, of which I had been so proud. So much work down the drain. But in its new sticker-and-screen form, the document served its purpose much better.

And let's be honest, in the end, that’s the objective of any instructional text: help the user to achieve their goals easily and feel good about themselves. To make a difference in their lives. Which, in my opinion, makes technical writing one of the best jobs in the world! Although, I have to admit, I might be slightly biased.

________________________________________

Update

I recently had a user experience myself, in which I ruined something by not reading the documentation first. My fault, or the documentation's fault? Read all about it here: