Usability Testing for BankFinancial: The Step-by-Step Project
BankFinancial is a publicly-traded regional bank with 19 locations in the Chicago area. In the spring of 2019, the bank’s marketing team was preparing to launch a new version of its public-facing website for consumers and businesses. They partnered with Marketade to lead pre-launch usability testing on their staging site.
Project Goals & Specs
- Test with a variety of users across personal and business audiences
- Observe users on desktops, phones and tablets
- Identify any show-stoppers, low-hanging problems, and long-term challenges
- Generate short-term fixes and long-term solutions
- Conduct a lightweight study as quickly as possible
Based on BankFinancial’s goals, we proposed qualitative usability testing as the primary research method: 1-on-1 moderated sessions with 10 participants. We supplemented this with quantitative usability testing: unmoderated sessions with 125 participants.
We broke down the project into 5 phases, each lasting about a week:
Week 1: Planning
We led a 1-hour kickoff call to discuss goals, roles, KPIs and timeline.
We designed the research and wrote a 7-page test plan, including usability test interview questions and tasks.
We created a recruitment plan including email messaging, online screener questions and phone screener questions.
Week 2: Recruiting
We sourced candidates from a variety of places. Candidates answered online screening questions to see how well they matched our recruitment criteria.
We analyzed responses, narrowed the pool, and then conducted phone screen interviews to identify the best candidates.
Participants scheduled their sessions through our web-based calendar tool and received automated confirmation and reminder emails.
“Got an objective perspective from actual users”
Phase 3: Research
We moderated 4 usability sessions with users on desktops …
… and 6 sessions with users on phones or tablets.
We conducted quantitative usability testing on Loop11 (with recruiting through Amazon’s mTurk platform) — with 100 consumers and 25 business users, on both desktop and mobile.
Week 4: Workshop
Stakeholders spent the morning observing nearly-full recordings of the most useful 5 sessions from qualitative testing — and taking structured notes throughout.
After lunch, stakeholders went through a series of steps to analyze, synthesize and align on top UX problems.
We ended the day by breaking into 3 small groups. Each group brainstormed and sketched solutions for 1 of the top problems — and presented their ideas to the larger group. We closed by aligning on next steps.
“We found out what customers are looking for in a bank website … and gained insight on what needs to be improved.”
Week 5: Report
After the workshop, we sent a 13-page summary report. It included details on research participants and links to all session recordings.
Findings & Solutions
It captured the team’s high-priority and low-priority findings from the workshop, along with their solutions ideas.
And it summarized the action items and owners identified at the end of the workshop.
“Roadmap for big and small wins moving forward …”