Strategic Insights and Clickworthy Content Development

Month: March 2017

Why Recommendation Engines Still Aren’t Accurate

Recommendation engines are deeply embedded in American culture. Anyone who shops online, subscribes to a streaming media service, conducts on online search, or uses social media is encouraged to do something — buy this, click on that, listen to this song, watch that movie. Sometimes the recommendations are accurate. Sometimes they’re not.

For example, Google’s search engine thinks I’m male. Netflix thinks I might enjoy comedies aimed at college-age men. Amazon’s recommendations can be strange if not laughable.

“On some level, these algorithms are amazing, but the types of errors that can be made are foolish,” said Patrick Wolfe, professor of statistics and honorary professor of computer cience at University College London (UCL) and Chair of IEEE Signal Processing Society’s Big Data Special Interest Group. “It probably wouldn’t take a lot of data to teach Google you’re not male.”

Credit: Pixabay

Credit: Pixabay

Context is Everything

Google and Facebook continue to serve up ads for the fringed boots I bought at Macy’s two weeks ago. They’re also recommending the infant and toddler car seats I bought as a gift in the same time frame. Clearly, the recommendations lack appropriate context.

“One of the reasons I think recommendation engines aren’t as accurate as they could be is that much of machine learning is about making predictions — predicting the weather or whether a document is about politics or not,” said Thorsten Joachims, a professor in the Department of Computer Science and in the Department of Information Science at Cornell University. “A recommendation engine has to be smart about the actions it takes.”

Pandora serves up more music that I like than dislike, although my genre preferences, Native American flute music and smooth jazz, represent dramatically smaller universes of choices than classical music or rock and roll. Pandora doesn’t tell me anything about local musicians or the local music scene. Granted, that would be an extremely difficult undertaking given the number of cities that consider their local music scene an integral part of their culture — Chicago, Las Vegas, and Ithaca, New York, for example.

Ithaca College Associate Professor Doug Turnbull and Cornell’s Joachims are tackling that very problem. With the help of their students, they developed MegsRadio, a music app that recommends local musicians and local music events. The app is specific to Ithaca’s local music scene, although others could benefit from the work if they chose to do something similar in another city, Turnbull said.

I tried the app and created a smooth jazz station. The playlist included some of favorite smooth jazz artists and some very talented artists (presumably from Ithaca) with whom I was unfamiliar.

Solving the Problem Pragmatically

Turnbull and Joachims are using counterfactual machine learning to compensate for the fact that they lack a user base large enough to train the algorithms. Counterfactual learning considers the recommendations an algorithm made, users’ historical choices, the accuracy of the recommendations made by the algorithm, and what would happen if a different algorithm were applied in the same circumstance.

“Counterfactual learning is saying that if we don’t have new users but we do have what the old algorithm predicted and the probability of the thing it predicted and given what the user historically picked in the past, would this new algorithm be better?” said Turnbull. “It’s like an alternate parallel universe.”

By comparison, popular search engines, shopping sites, and streaming media companies use A/B testing on a small percentage of users to determine whether one algorithm performs better than another. If the new one performs better than the existing one, the new new algorithm is eventually rolled out to all users. Counterfactual learning enables offline research. Users can be brought in later for validation purposes.

“It’s really making these systems much more like an agent that thinks about what the effects of its actions will be so that it can learn from past mistakes,” said Cornell’s Joachims.

What’s Your Take on Recommendation Engines?

As more analytics become prescriptive, recommendation engines will find their way into more use cases. Are recommendation engines helping or hindering you? What improvements would you like to see?

Why Embedded Analytics Will Change Everything

Analytics are being embedded in all kinds of software. As a result, the ecosystem is changing, and with it so is our relationship to analytics. Historically, analytics and BI have been treated as something separate — we “do”analytics, we’re “doing” ad hoc reporting — but increasingly, analytics are becoming an integral part of software experiences, from online shopping to smart watches and to enterprise applications.

“We’re creating whole industries that are centered around data and analytics that are going to challenge the status quo of every industry,” said Goutham Belliappa, Big Data and Analytics practice leader, for Capgemini North America. “Analytics will become so ubiquitous, we won’t even notice it.  From a business perspective, it’s going to transform entire industries.”

Three drivers are collectively changing how we experience and think about analytics. The first, as previously mentioned, is embedding analytics into all kinds of software. The second is automation, and the third is a shift in the way software is built.

Automation is Fuel

Modern software generates and analyzes more data than ever, and the trend is going to accelerate. The resulting glut of data is outpacing humans’ ability to manage and analyze it, so some analytics necessarily have to be automated, as do some decisions. As a result, analytics has become invisible in some contexts, and it’s going to become invisible in still more contexts soon.

“Frictionless” is a good way to describe what people are striving for in effective user experiences.  Certainly, with more automation and more behind-the-scenes analytics, how we think of analytics will change,” said Gene Leganza, VP & research director at Forrester Research. “We’ll be thinking about the results — do we like the recommendations of this site’s or this app’s recommendation engine or is that one better?  We’ll gravitate towards the services that just work better for us without knowing how they do it.”

That’s not to say that automated analytics should be implemented as black boxes. While humans will apply less conscious thought to analytics because they are embedded, they will still want to understand how decisions were made, especially as those decisions increase in importance, Leganza said. Successful software will not just automate data management and analytics and chose the right combination of microservices to achieve a particular result, it will also be able to explain its path on demand.

Microservices Will Have an Impact

Software development practices are evolving and so is the software that’s being built. In the last decade, monolithic enterprise applications have been broken down into smaller pieces that are now offered as SaaS solutions. Functionality is continuing to become more modular with microservices, which are specific pieces of functionality designed to achieve a particular goal. Because microservices are essentially building blocks, they can be combined in different ways which impacts analytics and vice versa.

Tableau has embraced microservices so its customers can combine B2B tools in a seamless way.  For example, Tableau is now embedded in, so a sales rep can get insights about a customer as well as the customer details that were already stored in

“The more embedded you get, APIs and developer extensions become more relevant because you need more programmability to make [analytics] more invisible, to be seamless, to be part of a core application even though it comes from somewhere else,” said Francois Ajenstat, chief product officer at Tableau.

Software continues to become more modular because modularity provides flexibility. As the pace of business accelerates, software has to be able adapt to changing circumstances quickly and without unnecessary overhead.

“In order to automate more and more actions and to enable adapting to a myriad of conditions, we’ll be having software dynamically cobble together microservices as needed.  The granularity of the services will have to be synced to the patterns in the data.  For the near future, the task will be to make the software flexible enough to adapt to the major patterns we’re seeing,” said Forrester’s Leganza.