Testing isn’t a luxury.

Saturday, August 24, 2013


This is your baby design, swimming in big wide ocean. (I had to make this photo relevant somehow.)

Three main things:

  • People will surprise you.
  • Testing tends to be worth doing.
  • It doesn’t have to be expensive.

Testing is seen as a luxury. In all design and software projects, some degree of validation is necessary to check you’re not going down a route that’s a bad idea and would be expensive to change later on.

Rapid prototyping is brilliant for this reason: it allows multiple ideas to be explored, iterated, and tested like it’s a real thing. In a worst case scenario, your favourite design might baffle or alienate a person you’re trying to serve. But it’s actually exciting when people do struggle with something and you immediately understand why and how it could be made better!

Scrappy but useful

Testing in the real world, and not in academia, tends to be scrappy and not particularly scientific. This is fine as it yields feedback that is still useful. Plus, we designers usually test consumer-facing UI rather than the effects of medicine or the safety of a vehicle.

Nevertheless, it’s handy to be aware of bias:

  1. We test a low number of people: eight is not a large sample size.
  2. We tend to influence results in lots of ways, for example, unconsciously confirming problems that we simply suspect exist.

Improvement epiphanies

Analysing themes in the data don’t usually indicate a clear “yes” or “no”, and it’s easy to bias the results by giving greater attention to one problem over another.

However, in the process of testing, you’ll find people experience the same problems, or struggle over the same hurdles. The results will aid a judgment call.

Quick test guide

  1. Have a simple prototype ready.
    I make a clickable prototype in Axure, but you can use anything (simple images, paper, HTML). The prototype usually demonstrates a simple flow of the design.
  2. Find people to test.
    You need to get hold of some appropriate people to use the prototype.Decide who you need to test – ideally, these would be people who might typically use the product.
    Decide how many people to test – 8 is about right.
    Decide how long you want to test them for – 1 hour is normal.
    Decide a reward – £50 is about right.
    Plan a schedule – I’ve tested 8 people over two days, but it might be better to test over four days to allow more time between sessions, better note-keeping, or even iterative improvements between testing sessions (especially if you see something easy that you can fix).
  3. Write a script.
    I create two documents: a warm-up / introduction with a few questions (10 mins) related to the topic (for example, retail or shopping apps, and real-world shopping habits), followed by a longer session (30-45 mins) which involve the users performing tasks on the prototype.
  4. **Collect data. **
    Ideally, you want
    someone to take notes
    audio recording of the session
    video recording can be useful, but not necessary, to collect people’s responses and reactions as they use the prototype
  5. Analyse data.
    You might have noticed themes arising during the testing. Make a note of all problems and repeat problems. One way to do this is by jotting recurring themes down on post-it notes.
  6. Iterate.
    Make some design improvements and test again, if you can. Unfortunately, it’s notoriously tricky to sell user testing to clients, so you might not be able to do a second round of testing.