A former engineer at Amazon has come up with a way to speed up the process of finding better math to produce suggestions of things you actually might want to buy.
There are many holy grails in online commerce, but one that has frustrated C-level executives and engineers alike is how to produce better recommendation algorithms. Produce better recommendations, and you?ll sell more stuff.
Historically, however, there has been one major structural impediment to making significant breakthroughs on this front. But the Chief Scientist at RichRelevance, which provides personalization solutions for the likes of Walmart, Sears, and Overstock.com, may just have fixed that.
First, however, the impediment: The people who are likely to produce breakthroughs--the really smart smarty-pants in the math departments of the world?s universities--don?t have access to large bodies of real-world data. And without real-world data, they can come up with as many hypotheses and new types of math as they like, but they?ll never really know if it actually works in the real world. It?s like trying to learn how to serve without tennis balls. You can swing as much as you like, but until you actually hit a real-live ball, you can never be sure if your swing would actually place a ball in the serve box.
For their part, the people who have real-world data--the Amazons and eBays of the world--can?t share it with the researchers for reasons of customer privacy. ?Even if we anonymize it, we?re handcuffed because we can?t give out data that can be reasonably be used to reconstruct who someone really is,? the Chief Scientist, Darren Vengroff, tells Fast Company.
Vengroff, however, has come up with a novel solution: He?s created a ?black box? of sorts with real-world data that researchers can use to run experiments on. Researchers won?t be able to look at the data, but they will be able to dump their algorithms in and have the box spit out results, which the researchers can then use to refine their hypotheses.
It?s a simple idea, but it wasn?t really possible to execute until the advent of the cloud. Now researchers from any part of the globe will be able to use the system to run experiments. (In principle, of course--in practice, a committee will vet proposals and choose which ones will actually run.)
Vengroff, who once worked as a Principal Engineer at Amazon, says he got the idea for the project while attending a computing conference last fall. ?In one of the sessions, there were three consecutive papers in a row where about two-thirds of the way through, I was really excited about what was being presented, and then they went down a different path than I thought they were going to go,? he says. ?I realized if they only knew what the real-live data, that I look at every day, says, they wouldn?t have gone that way. They would have gone the right way and gotten to a much better solution.?
?Seeing these brilliant ideas get misapplied because of a very reasonable assumption about how shoppers might shop, but happens not to be true in the empirical data--I realized we?ve got to find a way to have this not happen anymore.?
The system, which is in beta right now, will launch next month at conference in Palo Alto. Says Vengroff: ?We?ve got a significant new path that we think is really going to change things.?
E.B. Boyd is FastCompany.com?s Silicon Valley reporter. Follow me on Twitter, or email me.
Blu Cantrell Jaime King Lokelani McMichael Freida Pinto Eva Longoria
No comments:
Post a Comment