The recommendation lists at Amazon are legendary, and how they work is almost as big a secret as the Coca-Cola formula. Just kidding – it’s not, of course. We’ll explain the “magic formula” because it’s crippling (not to mention, annoying) when everything is declared a mystery. If you’re waiting for a magician to pull a rabbit out of his hat, you are the one who is becoming a rabbit that is facing down a snake.
In our last post, we demanded that managers make data mining an issue for themselves and suggested that they can do that as well because data analysis is impressive but not magic. At Amazon, it is primarily the product of tedious work. Three different teams (the Amazon Personalization Group, Automated Merchandising Team, and Automated Content Selection and Delivery Team) use data analytics so that their customers at home fill up their shopping carts.
Mustard or beer? What should the recommendation be when the customer already has sausage in his cart and previous customers bought 6 x sausage and beer; 5 x sausage, mustard, and beer; and 40 x beer and 3 x mustard? Amazon would recommend mustard, and we would, too.
The Amazon algorithm is easy to understand. Similar to the association method for shopping cart analyses, it deals with frequencies. The recommendation for Product A is Product B, which is the one that customers have bought most frequently when they already have Product A. In addition, an automatic correction ensures that the sheer frequency alone is not decisive for the recommendation. The frequency is divided by a factor that grows larger the more frequently A is bought without B, and B without A. But those are just details. Still in all, I could explain that better to myself at the dining-room table than at my desk and by using my own habits as an example. If I buy Bavarian-style veal sausage, I always buy sweet mustard and wheat beer along with it. I never buy sweet mustard without that kind of sausage, but I do purchase wheat beer all by itself. If I bought veal sausage at Amazon (yes, it’s possible), sweet mustard should appear first, followed by wheat beer. The measure that Amazon uses can do that. The wheat beer that I buy without sausage also lands in the dominator and makes the measure smaller than the one for mustard, which I never buy alone. Voilà!
Since you can also use Amazon to set the table, you can check if I am suitable for random sample #1. I got a recommendation for mustard (but not wheat beer) with the veal sausage. For wheat beer, however, I got a recommendation for mustard. Oh well. Nobody is perfect.
With those three paragraphs, we kept our promise. So… what does a manager need to consider when it comes to recommendation systems? Here are a few our own suggestions:
Recommendation systems may be great but they are still no word-of-mouth recommendation
Most people agree that they would most likely listen to the recommendations of their friends than anything else. Word-of-mouth recommendation, therefore, is the most effective form of advertising. Unfortunately, you cannot buy real word-of-mouth recommendation: Although word-of-mouth recommendation can be the result of marketing, it cannot be one of its means. Therefore, recommendation systems and the effects of recommendations are twofold. The computer that processes the suggested articles at Amazon is not a friend, so to speak. It can become one, however, if its recommendations are genuine.
The smallest store window in the world: From the perspective of a brick-and-mortar store, the available space to display additional goods in e-commerce shops is pretty skimpy. Source: Amazon.de.
Click to enlarge.
It’s no wonder that Amazon makes 30 % of its revenues through recommendations.
In every store we see more than just the article that we originally intended to buy. We see all of them – and that usually leads to further purchases. The more time you spend in the store, the further you walk, the longer you wait in the checkout line, the more items will land in your shopping bag. Supermarkets and home improvement centers profit from the fact that you don’t leave their stores with your eyes closed and, therefore, are subject to hundreds or even thousands of temptations. An Internet shop doesn’t have these opportunities. If you decide to purchase a book, Amazon only dangles four further types of bait – in this case, recommendations – in front of your eyes. Therefore, it is just rational that Amazon spends the money that it saves for leasing stores to find the right four ones. In our opinion, knowing which percentage of revenues comes from unplanned purchases isn’t as interesting as which recommendation algorithms work the best.
Sometimes you also hear that Amazon has increased its revenues by 30% through its recommendation system. “Through” is the decisive word in this case. Proving cause-and-effect relationships in human behavior is impossible. You need a workaround: In drug testing, for example, it is even mandatory. A test group gets the medication while a control group receives the same pills but without the active ingredient. If both groups feel equally good or equally bad, you can reasonably doubt if the medication makes a difference, in other words, if it works.
Many companies, however, conduct forecasting projects without a control group. Eric Siegel, an author, long-time professor at Columbia University, and big fan of predictive analytics, traces that back to the missing expertise among data analysis experts. In his opinion, not having a control group has been the norm for decades. Like I said before, more Kirk, please!
In most companies, it isn’t unusual to send a mailing to customers and to be satisfied if the response rate has increased. What you don’t know, however, is what would happen if you didn’t send a mailing because no one is measuring and comparing what the order rate is for customers that don’t receive one. They would be the control group. Good, clean methods are crucial because you have to calculate with stubborn people. Nicholas Radcliffe calls them sleeping dogs, boomerangs, or do-not-disturbs. Mobile phone providers are all too familiar with them. When a mobile phone contact is running out, they offer their customers an upgrade to the newest model. This only reminds the stubborn types that they can cancel their contracts, which they then do.
Customers that you should just leave alone: Sleeping dogs do the exact opposite of what you would like them to do – when you address them. Source: Nicholas Radcliffe, Generating Incremental Sales, 2007 (PDF).
Barack Obama’s 2012 reelection campaign was known for its intensive usage of control groups. Thousands of voters received visits or phone calls, got e-mails or letters in the mail, or were left alone. Afterwards, all of them were surveyed and divided into four groups:
- Sure things: Obama fans that would vote for him anyway.
- Sleeping dogs: Stubborn people who would only vote for him if you left them alone. Otherwise, they would vote for Mitt Romney.
- Lost causes: Obama opponents who would never vote for him.
- Persuadables: Undecided voters who could still be won over for Obama through one of the communication channels.
The costly campaigning – and that refers to any form of advertising in which you can directly address a Person – was targeted to the persuadables. The sure things make campaigning unnecessarily expensive while the sleeping dogs could lead it down the path of absurdity.
If what Siegel says is true, you also cannot trust the experts. Being the boss is a lonely job. That’s why we think it is so important for managers to understand the basics of data analysis and demand to see the plausibility of analytical assumptions. Henry Ford once wondered how half of his advertising budget could be thrown out of the window. Sleeping dogs show, however, that it can get even worse – especially when the boss has been bewitched into a rabbit.
