A real estate company adopts a culture of research and testing, and within months achieves a 180% increase in leads.

Challenge: Renter Leads Down 60%

When I first saw the results — a 180% jump in leads — I kicked myself for not coming up with the idea.

Later, I realized that the opportunity was obvious only in retrospect, and that only someone like the marketing director could have spotted it. This was, after all, what we had promised the real estate company:

  • Get to know your customers, and watch them use your site and web app
  • You’ll gain insights and ideas that you’ve never considered
  • Some of these ideas might radically improve your business metrics
  • You, not we the consultants, are in the best position to come up with the best ideas
  • But only if you participate in the research process

***

5 months earlier, the multi-state apartment finder company had called us in a panic. Their #1 KPI, renter leads, was down 60% over prior year. They’d overhauled their site a few months back, with a new design, new content, and a new property search flow.

Some metrics were up, but the most important ones were down. In particular they’d taken a hit on organic search traffic.

Since we’d worked with them on earlier versions of the site, they called us to see if we had ideas for getting their lead numbers back up. “Yes,” we said, “there are some SEO changes you should make. But the bigger opportunity we see is with improving the user experience and your conversion rate.”

And that’s when we made our promise above, with the requirement of team-based research.  “We’re in,” they responded.

Solution, Part 1: Customer Interviews, Personas, and Usability Testing

We went to work implementing an iterative program of 1) collaborative user research validated by 2) web analytics, followed by 3) design and content changes validated by 4) A/B testing.

One of the first things we did was interview a bunch of their leasing agents and customer support staff. We quickly gained an understanding of their renter customers: their behaviors, pain points, questions and concerns.

We prototyped 2 primary renter personas — along with 2 sub-personas — that we would refine over time as we talked to actual customers. Early on, we saw that new and improved content was a big opportunity with the site, so we had Nora our content specialist lead the customer research and persona creation.

Next we started usability testing the site with people who were in the market for a new apartment. The research team observing the sessions included the real estate firm’s marketing director and CEO, along with their design and development team.

Over the course of multiple rounds of testing, the research team identified 51 problems with the site and web app. For many of these, we turned to Google Analytics to validate and size the problem.

We collaborated to identify initial solutions for most of these problems. To help prioritize the work, we gave each solution an “Opportunity Score” of 1 to 10, which combined separate sub-scores for Impact and Effort. We ended up with five “9”s and fifteen “8”s.

Of these, we identified a number of solutions worth A/B testing.

Solution, Part 2: 7 A/B Tests Launched in 8 Weeks

The A/B testing ideas were the ones the team was most excited about. They included the most ambitious changes, as well as some that we hoped would be quick wins.

Over the years we’ve learned a surefire way to kill energy and momentum around a research-and-testing program: saddle already-busy developers with all the solution implementation work.

So we used Optimizely to implement all of the tests, allowing us to bypass the developers. We launched the tests in waves of 1 to 2 at a time every couple weeks. By the end of 8 weeks, we’d launched 7 tests. 3 of those were purely content changes; the other 4 included content and layout/design changes, including some that affected multiple pages.

We tracked all experiments in a shared Google Sheet that included short names for each design variation, links to screenshots, and a hypothesis for each experiment.

a/b test dashboard

A 2nd thing we’ve learned over the years: even though it’s exciting to launch A/B tests and get quantitative results, you can’t drop the qualitative user research. As you’re launching the experiments, you must circle back to usability testing to observe users interacting with the new designs. Qualitative research might show you why a new design is failing. It might suggest ways that a winning new design could be tweaked to perform that much better. Or it might open your eyes to a completely new solution.

Result: 180% Lift in Renter Leads

Of those first 7 experiments, 4 saw no significant change one way or the other. The other 3 each achieved renter lead conversion rate increases of over 25%.

The team was encouraged by these results. As the experiments passed surpassed 90% or 95% confidence levels, we passed the variations onto developers to implement as the new control. But as we looked more closely at the numbers and annualized the increases, we realized we’d still be well below last year’s lead numbers even with all the improvements in place.

That’s when the return to qualitative research saved us.

It was in one of the follow-up usability sessions that something struck the marketing director. “Renters seem to understand what we’re offering better now,” she said to the team. “And they’re more engaged.  More of them are jumping into the apartment search functionality and having a better experience there. But it’s still not a great experience. And we know from analytics that most people do a few searches and then bail.”

“Our online inventory can’t compete with Apartments.com. But once we get someone connected with a leasing agent, they have a great experience and usually find an apartment. What if we tried removing the apartment search option completely, and just relied on all that great new content we’ve created to get them excited about working with one of our agents?”

It seemed like a crazy idea. And the first responses were skeptical.

The group’s assumption to that point was that the only path to renters becoming leads is after they do some property searches on the site. These days, how could you be a web-powered apartment finder company and not offer the ability to look for apartments?

If the team hadn’t just watched a bunch of users have a sub-par experience searching for an apartment on the site, the idea would have gone nowhere. But as the marketing director reminded us of what we’d just watched, the idea started to seem less crazy. And discussion shifted to “How much work would this take to test?” and “Could we do this in Optimizely?”

Property search functionality and calls-to-action were integrated throughout the site, including in the navigation, so it seemed at first like a daunting task.  As it turned out, it took less than a day to launch a new, search-less version of the site within Optimizely.

And it took less than a week to gain 95% confidence that the new design was a big winner. The percentage of site visitors to convert into a renter lead shot up from 1.5% to 4.3% — a 180% improvement, and more than enough to close the gap from lost organic traffic.

We emailed the results to the team. The CEO responded enthusiastically and publicly thanked his marketing director: “It’s a brilliant idea.”