a thumbs up from facebook like

Human Curation Improves Big Data Algorithms

For the past decade, algorithms have been taking over the digital world. Just a few examples:

  • Social networks are constantly tweaking their algorithms to decide what to show us in our news feed (e.g. Facebook).
  • Search engines are constantly tweaking their matching algorithms to decide what to show us in our search results (e.g. Google).
  • Advertisers use targeting algorithms to customize the ads that are served to web pages we are browsing based on our profile and the task we seem to be pursuing (e.g. that ad for Caribbean vacations that seems to be following me around the Internet).
  • Music streaming services are using algorithms to decide what songs each of us might like (e.g. Pandora).

But there is a new trend that is putting the human back into the picture. Smart organizations have come to realize that as powerful as algorithms are, people are just too complicated and diverse to be modeled with math alone. They acknowledge the incredible value that human insight (the third type of thinking I teased here) can add to even the most sophisticated algorithm when it comes to understanding and predicting what a user might be interested in. If I ask an algorithm to find songs that have Jack Kerouac lyrics and Sex Pistols music, I will have to wait about 20 years for it to figure something out. But a 1970s style DJ could do it in a heartbeat.

Users are obviously the best judge of what they want to see, but we’re also notoriously reluctant to spend time and effort curating—take a look at the Google+ Circles experiment, for example. Facebook is likely hoping these changes help strike the right balance between control and convenience.

My Take

    So what is the result of this brilliant epiphany? We are starting to see hybrid curation/algorithm strategies for deciding what to present to users.

  • Facebook recently announced that it would allow users to select particular friends whose posts will appear above the algorithm-based news feed.
  • Amazon is creating curated shopping lists that take their usual “customers who liked that book also liked this book” and enhancing them with expert-based recommendations.
  • Music streaming, fashion, food, and other experiential product providers are recruiting experts to assemble unique personal collections.
  • Fantasy sports platforms are enhancing their algorithm-based player search with more nuanced recommendations from sports authorities.

I think this trend is sorely needed. Perhaps I am an outlier, but algorithms fail miserably to predict what I am most interested in. I have never bought a book from an Amazon collaborative filtering recommendation. I am always disappointed at what Facebook decides to prioritize in my news feed. I gave up completely on recommendation-based music streaming and just create my own playlists.

I always thought that a little human insight could go a long way. If I just purchased a particular brand of cereal, perhaps ads for that same cereal would be relevant again. But if I purchased a flight to and hotel in Los Angeles, is that a reliable indicator that I will be going there again soon? Apparently, even the best marketing aggregators think so. But a human who better understands the context might figure out why this is silly (and even annoying when it follows me around).

Score one for humanity!!

Your Turn

I am sure you are all familiar with algorithm-based displays and recommendations. Are they good at matching your interests and preferences? Or do they fail with you as miserably as they fail with me? Do you think a human curator could do better?

Let us know.

Image credit: “Facebook like thumb” by Enoc vt used under Public domain

One thought on “Human Curation Improves Big Data Algorithms”

Leave a Reply

Your email address will not be published. Required fields are marked *