User testing

In my third week in the CXL Minidegree I keep talking about conversion, but from the viewpoint of user testing, possibilities, benefits. User testing is about insight into the user's mind, and usability.

Usability basically means how easy user interfaces are to use. If a website (or a process on the website – like checkout) is complicated to use, people won’t buy from it. If they can’t find a product they want, they can’t buy it. If they can’t figure out what the site is about, they will leave. So its kind of important.

How user testing differs from A/B testing

Testing is the method to test two variants to know which outperforms the other, thus implementing the right the one.

We need to apply A/B testing in a company to ensure we are making the right decision to change element of a page based on data, not on opinions.

The key to increase your sites revenue is constant testing, and making the changes based on results.

A/B testing is done on live website with real visitors who have no idea that they are part of a test. User testing on the other hand usually involves people observing recruited testers complete a given set of tasks on the website.

When to conduct user tests

• Whenever you start optimizing a new website
• When you’ve done a design makeover for a website, but before you make it live (note that when creating a brand new design, keep usability principles in mind from the get-go)
• Whenever you change a critical part of your website (check the checkout, change category filters etc).

Sample size

The minimum number of people to conduct user testing with is 5 and maximum is 15.

Ten people will find 95% of your problems. 15 will find basically all of them.

Due to the law of diminishing returns, it’s typically not cost-effective to use more than 10 people:

Multivariate Testing

Example we can have two headlines, 3 buttons and 2 images to test. Our combinations can look like:

• headline 1, button 1, image 1
• headline 2, button 2, image 2
• headline 1, button 3, image 2

The multivariate testing is only feasible when we have a large number of audience members coming to our site. If it is less then we should better stick to A/B Testing.

Questions that I typically ask, and why:

What to observe:

• How well people understand what the site is about??
• Which path will they go down once on the site?
• How easy is it for them to find a product (a specific product and browsing, both)
• How easy is it to compare and choose a product (if the website sells many)
• Will they encounter any issues during the checkout / signup process?

?What do I ask them?

Start with a mindset:

Example: If you run an online store selling allergy relief products, you could say something like, “Imagine you were searching for products on Google and landed on this site”.

Tasks that I usually have testers do:

• “What is this website about as far as you can tell?”
• This is about the clarity of the value proposition. If they don’t understand (or takes a long time), you have a problem.
• “Shop around for a dress you like”
• This type of question gives them free space to go about the task.
• Evaluate how they do it, if there are any tedious repetitive tasks that could be optimized, any difficulty they experience. Is it easy to find a product they like? What’s missing?
• “Find a pair of green pants in size 36, under $50”
• Give some highly specific tasks to see whether people are able to use filters, search, and other features.
• “Find 3 suitable air purifiers. Evaluate them and pick the best one for you”
• How do users compare items? How hard is it to compare items? Do users find important product information easily?
• “Find the product Canon EOS 5D Mark II 21.1MP”
• How do they go about finding it? Via search, product categories, etc? Any challenges?
• “Start and complete the checkout process”
• Is there any friction in the checkout process?
• Users should go through the entire checkout process – no stopping at the payment screen. It’s a critical page and you need to observe if there’s any friction.
• Provide testers with dummy credit card numbers, 100% off coupons ,or use real credit cards if there’s no other option and refund later.

Once the test is complete, you want to have a brief reflection / summary session. Ask:

• “What was your overall experience like?” Their perception is a reflection of their reality.?
• “What was the best part of the experience?” Understand what they liked
• “What was the worst part, what would you change?” Insights into how they’d like to buy

Prioritizing hypothesis

There are few frameworks that we can use to prioritize hypothesis on, i describe the PXL Model (recommended by CXL institute)

This model instead of asking us to give rating, provides us with objective questions with yes/no answer. And then calculates the importance by scoring the answers based on researched data on its significance.

In the next week i going to talk about A/B testing mastery.

?

?