Challenge: Complex Medical App for Cancer Survivors

The team at Mount Sinai’s Icahn School of Medicine in New York City was worried. Supported by a grant from NIH, they’d been working for months on a new web app to 1) improve oral health and rehabilitation in oral cancer survivors, and 2) improve the coordination of care between survivors and their family caregivers.

They’d written pages and pages of tutorial content, and built out numerous modules that users could walk through step by step. As the team tried out the app in a test environment, everything worked and made sense to them. But 2 months away from their target launch date, they were left wondering: How would the experience be for their target audience? Would users engage with the tool, return to it, and share it with others? Or would they abandon quickly out of frustration or confusion or boredom?

The stakes were high. About 50,000 Americans are diagnosed with oral cancer each year, including many smokers and tobacco users. Of those, only a little more than half will be alive 5 years later.

The period after oral cancer treatment is critical and challenging. Eating, drinking, tasting and breathing can all pose difficulties for survivors and their caregivers.

The Mount Sinai team had observed these problems first-hand with their own patients, and wanted to develop a web-based tool to help oral cancer patients across the U.S. during this post-treatment period. Because they had a deep understanding of the audience and their problems, they were confident they could develop a useful, effective portal. Yet they knew how difficult it is to educate the public about complex medical topics, and to motivate people to change behaviors in a stressful time.

They also knew that, while they were experts on the subject matter, they lacked expertise on web user experience design. Even if the content resonated with users, would the flows and interactions make sense to them? And would users be able to find the content they needed most?

The Mount Sinai team reached out to Marketade to conduct a usability study and provide UX recommendations that would increase the app’s chances for success. With their looming launch date, they needed the study recommendations within 3 weeks.

Solution: Rapid Usability Testing

Within 10 business days, Marketade completed a lean UX study with 12 participants — including 8 cancer survivors and 4 caregivers — and provided human-centered design recommendations to the Mount Sinai team.

Here were the key steps in our process:

  • Day 1: Interviewed the Mount Sinai team about goals, scope, etc.
  • Day 2: Wrote draft test plan and shared with Mount Sinai for feedback
  • Day 3: Launched recruiting of oral cancer survivors and caregivers
  • Day 5: Piloted test script with 2 participants; made rapid revisions to tasks and questions
  • Day 7: Ran study with remaining 10 participants
  • Day 8: Analyzed sessions and identified problems
  • Day 9: Documented and prioritized findings; explored and sketched solutions
  • Day 10: Shared and discussed findings, recommendations and sketches with Mount Sinai

Result: “Such Valuable Feedback”

Through our observation of a small number of representative users along with heuristic analysis, Marketade identified 33 usability problems with the app, including 9 that we classified as high impact. In addition assigning a priority level to each finding, we categorized them by app section or topic to make them easier to tackle in batches. For instance, we spotted 4 problems with the login/registration process, 6 problems with navigation, and 5 problems with links.

One of our biggest findings was “Site doesn’t say who is behind it, hurting credibility”. In reality, some of the top oral cancer experts in the world, from one of the most reputable hospitals in the world, were behind the site. But as far as users could tell, the site could have been the product of an amateur.

Other examples of problems:

  • Medical jargon, unclear language, and too much fluff
  • Buttons and some links are same color as body content
  • Progression of lessons is not clear
  • Visited links don’t change color

For about half of the problems, we shared video clips from our usability testing to help designers, writers and developers understand the issue and empathize with users. This is never as good as true team-based research, but it’s better than just sharing the written findings, and was our best option for this project. There was plenty of positive feedback about the site from users, and we mixed that into our clips.

Of the 33 problems we identified, we recommended a rough solution or guideline for 31 of them. These were not detailed requirements or designs, since the Mount Sinai team was in a better position to come up with those. Our goal was to guide them (e.g. “use plain language”) or provide an example to stimulate solution thinking. Where possible, we provided screenshots to show both the problems and potential solutions.

Despite the large number of problems identified, the Mount Sinai team was energized by the study.  After we shared the results, the research professor leading the project wrote us to say: “We really love the videos and report. It is such valuable feedback and we look forward to going through everything and making the suggested changes to make the site better! The testers certainly gave us a lot to think about!”