Usability Testing with IT Security Managers for Malwarebytes

Last updated: June 30, 2026

Goals

Cybercriminals often enter corporate networks through employee desktops, laptops, and mobile devices — or what IT security professionals call “endpoints.” The rise of remote work during the Covid-19 pandemic increased the number of endpoints that could be hacked, heightening the risk of data breaches and other cyber threats.

Malwarebytes Nebula is a leading cloud-based platform for endpoint security. Small, mid-size, and large businesses use the product to manage the security of employee devices and to prevent cyber attacks.

A Malwarebytes team was recently considering a UX redesign of part of the Nebula product. To help guide decisions around this redesign, the Santa Clara-based company partnered with Marketade to conduct contextual interviews and usability testing of the current product, with a focus on the deployment flow.

Their specific research questions included:

• How do IT admins/managers deploy their endpoint security products?
• What are their reactions to Malwarebytes’ deployment process? What pain points do they experience?
• How successfully can they accomplish their goals using Malwarebytes’ process?

Approach

We used our standard 5-step approach for this project:

• Discovery: Aligned with the project team; planned the project; designed the research.
• Recruiting: Sourced, screened, selected, and scheduled research participants.
• Research: Moderated 1:1 remote interviews.
• Workshop: Facilitated a remote workshop with the Malwarebytes team.
• Report: Delivered a summary report with prioritized opportunities and solution ideas.

1. Discovery

To start the project, we facilitated a 1-hour kickoff meeting with the Malwarebytes project team. We followed this up with a couple of product walk-throughs.

We then wrote a recruitment plan that defined the target audience and provided 2 rounds of screening questions: one for our online survey and one for follow-up screening via phone calls.

Next, we wrote a discussion guide for the research sessions that included:

• 25+ potential questions for the 1st part of the session: a contextual interview
• 2 scenarios — each with 10+ potential tasks, sub-tasks, and questions — and 6 potential follow-up questions for the 2nd part of the session: usability testing

2. Recruitment

We recruited 10 participants who were:

• Responsible for deploying and managing endpoint security software at their company
• At a small or mid-size organization (100 to 500 employees)
• In a Windows or hybrid environment
• Currently using a cloud-based security product
• Familiar with a specific Microsoft network security service

Beyond this, we recruited participants that represented a mix of deployment approaches (e.g. manual vs. tool-based).

Finding these participants was difficult given the specificity of the target profile and the nature of the subject matter: people who manage cybersecurity for their company are reluctant to participate in interviews where they need to talk about their security practices and processes. Fortunately, we have conducted plenty of recruiting in the digital security space and were able to address concerns and build the necessary trust with these participants.

3. Research

We conducted one-on-one 60-minute interviews with the 10 participants.

In each session, we interviewed the participant about their current role and how they handle endpoint security deployment. Then, we asked the participant to share their screen, log into a virtual machine, and use Nebula to complete a deployment task and provide feedback.

4. Workshop

We often use a team-based workshop approach for synthesizing user research because it brings teams to a consensus faster than other methods. And because it builds buy-in across the organization, this approach results in more solution ideas going live.

7 Malwarebytes team members participated in this half-day workshop. Their roles included:

• Head of UX
• UX researcher
• VP of corporate products
• Lead product designer
• Manager, software development

To capture and prioritize user feedback, workshop participants:

• Watched 3 research session recordings and took structured notes on what they observed prior to the workshop.
• Prioritized the notes most relevant to the research questions and put 15 notes on a virtual whiteboard.
• Reviewed the shared notes and sorted them into groups based on common themes.
• Voted on which themes/groups were most critical to address in the workshop based on the project goals.

The workshop team identified 20 finding themes. Of those, 3 were prioritized as “quick win opportunities” and 3 were prioritized as “big opportunities”, i.e. those having the biggest potential to improve this product flow’s UX.

For each of the 3 quick-win opportunities, the full team explored and prioritized solution ideas, and assigned owners to each viable solution. One of the solutions was simply to remove a certain section of content for first-time users of the product.

For the 3 big opportunities, we broke into 3 small groups and assigned each group an opportunity.

Each group used a document template to capture the problem and possible solutions. We followed the steps below:

• State the users’ problem(s) and note the underlying issues with this problem in 2-3 bullets.
• Note specific references to the issue from the research using the virtual whiteboard.
• Brainstorm solutions or paths to solutions and note risks to success.

Then, each group presented their solutions to the larger workshop group and discussed any questions or comments on their solution(s). Based on the discussion, the team agreed on the solutions to pursue further and assigned the next steps for each.

5. Report

After the workshop, we wrote and delivered a 26-page report that included:

• 1-page summaries of the 10 research sessions, including their current deployment process and their pain points with the Nebula process.
• Documentation of the findings, solution ideas, and next steps generated in the workshop.
• Interview guide, participant information, and other research process artifacts.

Outputs

By the end of this research project, Malwarebytes had gained a deeper understanding of its target audience, users’ current security deployment processes and challenges, and user pain points with Nebula’s deployment flow.

Based on these user research insights and our collaborative process, the Malwarebytes:

• Identified 1-2 specific solutions for each of the 3 “quick win opportunities”.
• Generated, discussed, and prioritized among 14 specific solutions in their 3 “big opportunity” areas.
• Identified and discussed 6 potential risks to success for these solutions.
• Agreed on 13 specific next steps and owners to move the most viable solutions forward.

After the project, we received the following feedback from Malwarebytes team members:

Collaboration was successful and focused us in on key priorities to fix.
— VP of Product

Excellent research and workshop activities. The participants were true potential customers. And the affinity clustering was a great tool.
— Manager, Software Development

We appreciate your excellent service and look forward to future partnerships with you.
— Director, User Experience

‍

‍