Zestimate Prize

by Jim the Realtor | Jan 31, 2019 | Jim's Take on the Market, Listing Agent Practices, Zillow | 1 comment

Hat tip to SM for sending in this detailed article about Zillow’s contest to improve the zestimate. No surprise that the winner was a group of analyzers who blended the algorithms to move the needle a couple of ticks.  If they would have asked me, I’d say score each agent, and factor in +/- 5% based on who is selling the house – your agent makes that much of a difference!

In Seattle, the typical Zestimate is off by 4.7 percent, which amounts to $35,000 on the median home. Data scientists from around the world competed to improve the algorithm and expect to get the median error rate down to about 4 percent.

On Wednesday, the Seattle-based company awarded a $1 million prize to the winners of a public contest to improve its algorithm. The winning team, three guys from Raleigh, Toronto and Morocco who teamed up despite never having met in person, came up with a way to beat Zillow’s own data scientists to a better estimate.

The contest started a year and a half ago with 3,800 teams from 91 countries and was narrowed down to 100 finalists last year. The teams were given seven years’ worth of data on a sample of millions of homes across the country, and were tested to see how closely their estimated values for each home matched up with the actual sale prices of homes that sold in the ensuing months.

Jordan Meyer, the American on the winning team, reduced his workload at his day job as CTO of an analytics company and poured about six hours a day into the contest, communicating with his teammates, Moroccan computer science professor Chahhou Mohamed and Canadian artificial intelligence startup founder Nima Shahbazi, on the messaging application Slack.

Meyer started by finding every data source he could — the exact longitude and latitude of houses could be used to determine the proximity to streets and therefore determine noise near the house. Slight differences in distance from a body of water could influence a home price by thousands of dollars. In the end each home had hundreds of different data points.

But the strategy that set them apart was trying wildly different algorithms and merging the ones that worked together to get the best blended average.

“It was extremely hard,” Meyer said in an interview. He called the process “relentless experimentation” and echoed Shahbazi, who said in a statement: “For every idea that worked, there were a hundred that didn’t work. But we kept going.”

Zillow has slowly improved its Zestimate from a median error rate of 14 percent when it started in 2006 to 5.7 percent when the contest began in mid-2017. It’s now down to 4.5 percent nationally (it’s higher in some cities and lower in others), and once the winners’ tweaks to the algorithm are incorporated, the company expects the error rate to dip to about 4 percent.