Last week I spent a lot of frustrating hours wrestling with how to redo a product section of a client’s site to improve their search engine visibility. Brian, our SEO guru, had a vision for the section that I had big concerns about. We went back and forth a lot, and eventually it occurred to me that — rather than continuing to debate it — I could prototype it.
What a difference that made. Within 1 hour I went from feeling overwhelmed with questions to having clarity and being excited. Once I could see it, ideas quickly jumped to mind for ways to address the concerns I had, and I could sketch those immediately on the screen. When I was done, rather than having to spend another couple hours writing out our approach and explaining it by phone to our client, I simply shared a link to the prototype. The prototype sold the vision, narrowed the scope of interpretation, and answered questions much more effectively than I could in words.
The democratization of wireframing and prototyping in the last few years is one of the most exciting things I’ve seen since I started working on the Web. From startup founders to big-company business analysts to online marketing agencies, lots of people who’d never considered prototypes in the past are using them today. This is in large part due to new tools and techniques that allow anyone to quickly create interactive prototypes of web pages, mobile apps, and even tangible products. Without any knowledge of HTML, you can create web prototypes in hours that in the past would have taken a developer days or weeks. The recent popularity of Lean Startup and UX principles have accelerated this growth.
As in my example above, a great thing about prototypes is that they provide a much more effective platform for exploring, communicating and debating ideas than email, specs, slides, or whiteboards. The prototype is becoming, as Jared Spool says, “our silver bullet to shared understanding fast.”
But there’s an even better part about prototypes, and it’s one that’s vastly underutilized in my view: putting prototypes in front of your product’s users.
If they do it at all, most companies only do usability testing after they’ve spent hundreds of hours writing, designing, debating and coding. Fixing problems at this stage is often 100 times more expensive than if you’d caught them in a prototype. Rather than taking a couple hours to fix, it often takes 6 months.
By starting your user testing with an early-stage, code-free prototype, you can create a super-fast feedback loop: test Monday morning, iterate Monday afternoon, test Tuesday morning, iterate Tuesday afternoon, and so on. And with the rise of remote usability testing tools, it can even look like this: test at 9am, iterate at 10am, test at 11am, and so on.
Now, as some usability legends are quick to point out, none of this is new. Paper prototyping has been part of the UX toolkit for decades. And a sketch on a napkin in a coffee shop is often still the best place to start your user testing.
True and true. Still, I love the new breed of prototyping and user testing tools. They make prototyping a lot more fun. And they make recruiting and watching users a lot faster, easier, and less intimidating. No they don’t magically make people good designers, writers, and researchers. But they get many more people in the game, and that counts for a lot.
There are plenty of articles out there on different prototyping tools — and on different remote user recruiting and testing tools. But there aren’t many on how to combine these two sets of tools — and that’s where the real magic is. With that in mind, here are 3 combos worth trying.
Combo #1: Balsamiq + UserTesting.com
Balsamiq and UserTesting are both easy-to-use tools and are a good place to start. Here’s a rough list of steps to follow:
- Sketch a wireframe of your page/site/app using Balsamiq
- Add interactivity by linking pages together
- Click the “prototype” link and grab the URL from your live web-based prototype
- Think of some key tasks or questions to ask users about your prototype (if this is your first time writing a test, get Steve Krug’s Rocket Surgery Made Easy)
- Go to UserTesting and sign up for a test with 3 users
- Enter your Balsamiq prototype URL, your tasks, and identify the types of users you want
- Within 1-2 hours you should have 15-minute videos of people using your prototype
- Watch the videos and write down the problems you observe
- Identify the smallest possible solution for your biggest problem(s)
- Go back into Balsamiq and iterate your prototype
- Repeat the process again and again
Costs: Balsamiq has a free trial; plans start at $12/month after that. UserTesting.com is $39 for each user/session/video. Well worth it.
Combo #2: Treejack + Ethnio + Join.me
Even people who like to prototype and test often jump too quickly to testing page layout, design, and copy. If you’re creating a new app or redesigning a site or doing anything that involves restructuring and relabeling your navigation, you’ll want to test the information architecture (IA) before you test wireframes, content, etc. There are things you’ll want to do to generate potential IAs (card sorts, user interviews). Once you have one that’s promising (a hypothesis), the best way to validate/invalidate it is through tree testing; and the best tool for tree testing is Treejack from Optimal Workshop.
While panels like UserTesting.com are great, the best way to recruit for most user tests is to intercept people as they’re using your site/app. This makes it much more likely you’ll get a representitive user who behaves in a natural way during the test.
Ethnio is the best tool I know of for doing live intercept recruiting. We use it mostly for customer development/user research interviews and traditional usability tests of a live site. But it’s also a great way to recruit for prototype tests. Here’s a high-level process you can use with tree tests:
- Head to Treejack and enter your navigation structure
- Treejack will automatically generate a prototype of your IA
- Enter the most important tasks that users need to complete on your site
- Use Ethnio to recruit actual users of your site/app by adding a popup
- Call users one at a time (ideally right away) and use Join.me to give control of your desktop
- Open your Treejack test and tell users to think aloud as they go through it
- Write down users’ top points of confusion and failed tasks — and check your findings against the data visualizations in Treejack
- Identify ways to improve the IA through changing the structure or words/labels
- Go back to Treejack and edit the navigation
- Repeat the process again and again
Costs: Treejack has a free version for small studies ($109/month for larger studies). Ethnio is free for small sites, $50/month and up for larger sites. Often you’ll want to give users an incentive to do the test (e.g. $25 Amazon gift card). Join.me is $19/month for their business plan.
Combo #3: Axure + Loop11 + Mechanical Turk
Ready to do more advanced prototyping and testing? Here are a few options:
Axure allows you to prototype a more interactive interface than Balsamiq and similar tools. Anytime I need to have users enter free-from data on a site or app (doing a search, filling out a form), I go with Axure.
Loop11 is great for running quantitative usability tests with large numbers of people. Both of the types of tests described earlier are qualitative; they focus on insights derived from observing and listening to individuals as they use your site, one at a time. Qualitative tests are easier, cheaper, and faster, and generally produce better insights. But sometimes you’ll want to have bigger numbers — because management wants it or because you need to choose between two closely competing designs.
A challenge with Loop11 is that you need to recruit participants yourself; there’s no panel as with UserTesting.com. Ideally you’ll recruit people from your site, database, or social media channels — using Ethnio, email, and/or other techniques. But I find there are times when that’s not feasible, and I need 100+ representative users quickly. If it’s a broad-base B2C product, I often go to Amazon Mechanical Turk, a crowdsourcing platform where people perform quick “human intelligence tasks” (HITs) for micropayments. Typically I’ll run tests that take between 1 and 5 minutes, and pay between $0.50 and $2.00 per user/session. It’s a clunky interface and takes some practice to get quality results, but once you learn it it’s an amazingly fast and cost-effective recruiting tool.
Here’s a rough process to use with these tools:
- Create your prototype in Axure
- Sign up for AxShare to create a sharable version of your prototype
- Go to Loop11 and create your test, adding tasks and questions
- Go to Mechanical Turk and “Create” a new project, setting the context for the test and pointing people to Loop11
- Start a “Batch”, specifying how many users you want for your test, and how much you’ll pay per user
- Once your batch is complete, go to Loop11 and review the results, focusing on things like task success rate, completion times, or patterns in comments
- Go back to Axure and iterate your prototype based on what you learned
- Repeat the process again and again (ideally combining it with qualitative testing)
Costs: Axure is around $500, and has a 30-day free trial. Loop11 is $350 per project after a free trial project; an annual license starts at $1,900. Mechanical Turk pricing is variable based on the number of users you need and your price per user. We typically end up spending $50 to $200 for a test.
Bonus Technique: Testing Competitors’ Sites
Even companies that do regular usability testing rarely think to test their competitors’ sites or apps. That’s a huge missed opportunity. Here’s how Steve Krug puts it: “someone has gone to the trouble of building a full-scale working prototype of a design approach to the same problems you’re trying to solve, and then they’ve left it around for you to use.”
Once I started thinking of competitor sites as prototypes, I made testing them a first step on most projects. They’ve produced some of our best ideas — and helped us avoid countless mistakes (just because competitors are doing it doesn’t mean it’s a good idea.)
You can include competitor sites in any of the above combinations.