How we user-tested pelicanbooks.com before launch

The pelicanbooks.com website lets users read a small but slowly growing stable of new-release Pelican books in the browser.

During the build, our mission was to create a beautiful in-browser reading experience for the first Pelican books to be published for thirty-odd years.

User testing was going to be even more vital than usual here, because the product we’re selling is an online experience.

We knew we had to reduce friction, and we had to get our readers into the page as quickly as possible (as quickly as opening a book!), absorbed in reading the text, and converted to the pleasures of owning online books. User feedback was going to be vital in achieving this.

How did we go about user testing?

We kept the process lightweight:

1. Recruited 5 volunteers from across the company who hadn’t been involved with the project (following the guideline that 5 testers are enough to identify 85% of usability issues).
2. Prepared a simple script asking users to go through the main actions on the site.
3. We asked our testers to voice their thought processes as they went. Why were they doing what they were doing, clicking and going here instead of there? Where are they getting stuck? What do they like, and not so much?
4. As people navigated, we observed, sitting to the side as unobtrusively as possible, and I took notes.
5. Afterwards I categorised the feedback by feature and page and gave each point a rough priority. I distilled the feedback into the top 2 or 3 main takeaways (as feedback often overlapped and where it did we knew we needed to focus), and then we had an action plan to work on with our development agency, Fiasco Design.

The first round of user testing gave us a lot to chew on, and reminded us how truly indispensable testing is.

What did we discover, what did we learn?

Three major themes came out of the first round, which were repeated by several respondents.

First, the website proposition wasn’t clear to our users?—?a cardinal sin.

The website told users they could choose a book to read, and gave them words to dive in and read quickly?—?we had successfully minimised the barriers to getting into the text?—?but we had hidden the fact that you can buy and own the book, to read it in its entirety online.

This was because readers were only prompted to purchase (and only actually told that they could purchase) when they reached the end of the free first chapter.

We’d assumed a little too much that when people clicked on the title and began reading, they’d get to the end of the chapter, and reach the prompt to purchase.

Although this linear progression makes perfect sense if we think about people reading in an ideal environment, it doesn’t fit how people read online?—?dipping in and out.

Both of our key personas were time-poor and information-overloaded?—?this is the common denominator of anyone consuming “brain food” subject matter on the internet. But this hadn’t factored enough into our design decisions.

We had made the classic mistake of assuming things we knew were self-evident to the user too. It’s so hard to avoid that. Luckily discovering those things is exactly what user testing is for!

How did we fix it?

We alerted users that you can buy and own the books to read online by:

• Adding a “Buy Book?—?[Price]” button to the nav bar at the top of the free first chapter pages (see image left). The Buy Book isn’t (only) a salesy plea for people’s money, it’s a functional statement about what you can do on that page?—?you’re not just reading some isolated fragment of text, but can own the book.

• Adding a “Buy The Book” button next to each book on the Library page.
• Adding to the homepage, a rich visual guide to what you can do on the site, and how?—?emphasising that you can buy and keep the books to read online.

The second major point was that people found the navigation confusing in places.

Here we had multiple issues:

• There was inconsistent labelling?—?the link to the homepage was called “Library” in some places, Home in others.
• People couldn’t find their way back from some pages and sections.
• And, relatedly, because we had relied on full-screen overlays for certain interactions?—?which looked like pages rather than overlays?—?when people resorted to the browser back button, they felt like they jumped back 2 pages rather than 1.

While some aspects of the navigation worked well, the overall impression was of unpredictability, unreliability. A multitude of small mis-steps combined to create a sum (of confusion) bigger than its parts.

It was really important to simplify and make it consistent.

So our takeaways here were:

• Use the label “Library” consistently, everywhere, in preference to “Home” (even though when you’re logged in, the Library page, is the homepage).
• Make it clear when a page is actually a modal, to be closed using the “X” link, rather than a page to be navigated forward or back from?—?by adding transparency, so the underlying page is visible beneath.
• Ensure that from every page or menu there is a clear step back or up.

The third major issue was that we’d built features users couldn’t find?—?for instance highlighting. The feature was hidden?—?it was only when you began to highlight text, that you were told you could save your highlight.

To address this, we added a visual illustration of the feature to the homepage intro guide.

There was also a raft of smaller feedback. And lots of encouragement. So we felt we were on the right track! People said nice things like it’s “gorgeous”, “easy to use”* and that they’d buy and read the books.

Three months later we had implemented the changes which had been identified, and were ready to take the product to the panel again. We sat down with a mix of fresh eyes and some veterans from the first round.

The second round of feedback was much more marginal. There were further feedback and tweaks to be made, but the relatively much smaller scope of the issues uncovered and the fact that the most important aspects were working, made us feel confident enough to launch the product.

Sure, it wasn’t perfect; but we knew that if we shipped and it was perfect we’d have launched too late. It was time to launch to the wide world, and get real feedback from real paying customers, paying real money and trying to read our books online in the midst of their busy day.

We certainly relearned the lesson that testing with users is the single most important thing you can do to maximise your product’s chances of success. You just cannot tell how a product will be received, perceived and used, until it’s in real users’ hands. User test as early and often as humanly possible!

*the lion's share of the acclaimed design and frontend of pelicanbooks.com was produced by Matthew Young at Penguin. Kalle Mattila came up with lots of the concepts, and growth hacked like a fiend. I was really honoured to be able to join up with Matthew and Kalle and help make their strong vision for the website a reality.