Overall Outline:

1. The 3 Types of Usability Testing and When To Use Each

2. Story Time: Closed Test

3. Usability Theater Design

4. Practice: No Explaining!

NOTE:

We can deliver a workshop about one of the three types of testing a half-day format, or teach all three types of testing in the full-day format.

Core Learnings

Our brain biology is wired to ignore everything we see repeatedly. The things we do every day, or especially multiple times per day, we do not remember. Users will not tell us the most important requirements, the things they do constantly daily, because we biologically wired to not remember.

The biological mechanism for finding what users “forgot” to tell us is in the visual cortex. When users see that something is missing, then it jogs their memory. We, as designers, are more likely to see these daily requirements than the users, because it is all new to us. You must observe users using your product in order to uncover all the requirements.

Introduction: 3 Types of Usability Testing

Presents a quick overview of the 3 main types of testing: closed, open, and directed. The intent of this first section is to help attendees see there are many facets to Usability testing and that each has their own rules. Picking the wrong type of test will produce the wrong outcomes.

We will start with a description of “Directed” testing, since most people will be familiar with this type of testing. It is the type used for User Acceptance Testing. You already have a product built and user experience designed. The goal of the test is to verify if the product meets expectations (yours and customers).

Second type is “Closed” testing, where the goal is to optimize the usability & likability of your UI. This type has very specific rules where you cannot use Steps to Reproduce. Users are only given a goal to accomplish during the test. We’ll talk about the importance of prohibiting facilitators from helping test subjects use the product, so that you learn to understand how users think, instead of forcing them to understand how you think.

The third type is “Open” testing, where you follow your users around and experience their lives and needs. There is no product involved, and it is the most “pure” form of user testing. This format is the most-powerful when you don’t know what your customers want.

We’ll end this first section with a quick quiz, presenting 3 problem scenarios with a product, and they have to choose the best format of test to use.

Story Time

As we deep-dive into each type of test, we will start with a story to outline the core problem of when we can or cannot use each type of test.

The Rules of Usability Testing

We will present the rules of Usability Testing:

1. You cannot “lead the witness” or “bias the test”. We’ll talk about what to say instead like, “What would you expect that thing to do?” We’ll cover “thinking out loud” and what to so to get users to open up about their opinions.

2. Always have a 2nd tester, a silent participant to take notes.

3. Record all sessions, including the user’s face and emotions, and the product itself.

4. You only need 3-5 users per demographic group (as documented by the Nielsen Norman Group), after which the novel new discoveries drop off substantially.

Practice

The hardest part of usability testing is to not say anything to direct the user towards the outcome that you want, so we will practice this. Participants will pair up. The “tester” will present a “product”. It could be a real product, a website, a phone app, etc. The tester picks one goal for the “user” to do. They practice following the rules of Usability Testing. At the end of 15 minutes, they switch roles and the other person gets a chance to practice facilitating.

Usability Theater Design

Then we’ll talk about “Usability Theater” where you get your whole team together to watch the recordings. Make a party out of it! Get popcorn and sodas for people! Let them shout at the screen, like Rocky Horror Picture Show, or Horror Movie House Rules, e.g. “Don’t go in there!” “Don’t split up!” “ Why?! Why would you do that?!!” Many won’t believe test results until they can see for themselves. When the team can see the results first-hand, they will stop debating your conclusions and immediately start helping you design better solutions! After watching the videos, everyone makes a list of problems that they saw in the videos, and brainstorm on ways to solve each problem.