Why You Should Not Build a Recommendation Engine

One does not simply build an MVP with a recommendation engine

Recommendation engines are arguably one of the trendiest uses of data science in startups today. How many new apps have you heard of that claim to “learn your tastes”? However, recommendations engines are widely misunderstood both in terms of what is involved in building a one as well as what problems they actually solve. A true recommender system involves some fairly hefty data science — it’s not something you can build by simply installing a plugin without writing code. With the exception of very rare cases, it is not the killer feature of your minimum viable product (MVP) that will make users flock to you — especially since there are so many fake and poorly performing recommender systems out there.

A recommendation engine is a feature (not a product) that filters items by predicting how a user might rate them. It solves the problem of connecting your existing users with the right items in your massive inventory (i.e. tens of thousands to millions) of products or content. Which means that if you don’t have existing users and a massive inventory, a recommendation engine does not truly solve a problem for you. If I can view the entire inventory of your e-commerce store in just a few pages, I really don’t need a recommendation system to help me discover products! And if your e-commerce store has no customers, who are you building a recommendation system for? It works for Netflix and Amazon because they have untold millions of titles and products and a large existing user base who are already there to stream movies or buy products. Presenting users with  recommended movies and products increases usage and sales, but doesn’t create either to begin with.

There are two basic approaches to building a recommendation system: the collaborative filtering method and the content-based approach. Collaborative filtering algorithms take user ratings or other user behavior and make recommendations based on what users with similar behavior liked or purchased. For example, a widely used technique in the Netflix prize was to use machine learning to build a model that predicts how a user would rate a film based solely on the giant sparse matrix of how 480,000 users rated 18,000 films (100 million data points in all). This approach has the advantage of not requiring an understanding of the content itself, but does require a significant amount of data, ideally millions of data points or more, on user behavior. The more data the better. With little or no data, you won’t be able to make recommendations at all — a pitfall of this approach known as the cold-start problem. This is why you cannot use this approach in a brand new MVP. 

The content-based approach requires deep knowledge of your massive inventory of products. Each item must be profiled based on its characteristics. For a very large inventory (the only type of inventory you need a recommender system for), this process must be automatic, which can prove difficult depending on the nature of the items. A user’s tastes are then deduced based on either their ratings, behavior, or directly entering information about their preferences. The pitfalls of this approach are that an automated classification system could require significant algorithmic development and is likely not available as a commodity technical solution. Second, as with the collaborative filtering approach, the user needs to input information on their personal tastes, though not on the same scale. One advantage of the content-based approach is that it doesn’t suffer from the cold-start problem — even the first user can gain useful recommendations if the content is classified well. But the benefit that recommendations offer to the user must justify the effort required to offer input on personal tastes. That is, the recommendations must be excellent and the effort required to enter personal preferences must be minimal and ideally baked into the general usage. (Note that if your offering is an e-commerce store, this data entry amounts to adding a step to your funnel and could hurt sales more than it helps.) One product that has been successful with this approach is Pandora. Based on naming a single song or artist, Pandora can recommend songs that you will likely enjoy. This is because a single song title offers hundreds of points of data via the Music Genome Project. The effort required to classify every song in the Music Genome Project cannot be understated — it took 5 years to develop the algorithm and classify the inventory of music offered in the first launch of Pandora. Once again, this is not something you can do with a brand new MVP.

Pandora may be the only example of a successful business where the recommendation engine itself is the core product, not a feature layered onto a different core product. Unless you have the domain expertise, algorithm development skill, massive inventory, and frictionless user data entry design to build your vision of the Pandora for socks / cat toys / nail polish / etc, your recommendation system will not be the milkshake that brings all the boys to the yard. Instead, you should focus on building your core product, optimizing your e-commerce funnel, growing your user base, developing user loyalty, and growing your inventory. Then, maybe one day, when you are the next Netflix or Amazon, it will be worth it to add on a recommendation system to increase your existing usage and sales. In the mean time, you can drive serendipitous discovery simply by offering users a selection of most popular content or editor’s picks.

The following two tabs change content below.

Valerie Coffman

Founder and CEO at Feastie
I'm a physicist turned data scientist and entrepreneur. Founder of Feastie -- search and analytics for the foodie blogosphere. I also blog at valeriecoffman.com.

Latest posts by Valerie Coffman (see all)

This entry was posted in Commentary, Data Science DC, Methods, Rant and tagged , , , . Bookmark the permalink.

3 Pingbacks/Trackbacks

  • http://raymondduke.com/ Raymond Duke

    Just 15 minutes ago I tossed out the idea similar to this. While designing my MVP, I got a little carried away with filters, sorting, and other things that can be potentially added later.

  • http://www.facebook.com/nikki.ralston.52 Nikki Ralston

    Very well written Valerie. I agree that Pandora has done a wonderful job, bu tit is NOT automatic. They have an army of music analysts listening to tracks and manually assigning tags. Netflix and Amazon deserve respect for realizing the importance of recommendations, but their approaches need serious re-engineering. Netflix held its famous $1 Million competition, but it was mostly a publicity stunt in the end as they didn’t even use the winning algorithm. Why? Because it was CF based and only very marginally improved a pretty bad solution. Most users consider Amazon suggestions ridiculous and annoying.
    Another great point that you make is that there will never be one big, mighty algorithm for everything in the universe. Engines must be built within specific domains, kike Feastie (food), Pandora (music) and Jinni (TV and movies).
    Jinni chose a semantic approach that involves deep knowledge of recommended products, automatic tagging and rich set of intuitive discovery features to give the user his choice of how to find content that suites his taste and mood.
    Good luck to Feastie – there CEO is certainly on the right track!

    • http://www.feastie.com Valerie R. Coffman

      Thank you! And I stand corrected on Pandora being automatic, but that’s even more to the point — you either need automation or an army of experts. Amazon and Netflix may be limited, but Amazon claims that their recommendations drive 35% of sales (a few billion dollars) and Netflix claims that recommendations drive 75% of use. I’m not sure it’s possible to build a perfect recommendation engine, but if it moves the needle, then even a recommendation system that misses the mark much of the time still serves its purpose.

  • Pingback: Episode 004: Bowling for Anti-Dollars | The Super Science Happy Hour with Matt & Matt

  • Chris Daltas

    I think your definition of a ‘recommendation engine’ is too narrow. We built a basic recommendation engine for Hundo as part of our MVP and it has increased our retention by over 10x.

    Here is how it works (maybe you would consider this a recommendation engine, except providing recommendations is exactly what it does):

    A user signs up for Hundo to get the best products every where they shop.

    On Hundo “best product” = highest rated. Top influences are rating, relationship to the reviewer, date created, and distance. You can sort content by any of the influencers.

    So, if I was on Hundo looking for the best Coffee in Seattle…

    Search for coffee. Hundo displays coffee results. If one is selected we also display recommendation, users who rated this coffee highly, rated this coffee over here even higher AND of your friends, this is the highest rated coffee.

    Hundo does recommendations in the passive state as well…

    When one of my Facebook friends rates a place/item highly I get a Notification that says “Ryan Johnson, just rated the Dark Roast Coffee @ Seattle’s Best 85 out of 100. But Ryan’s top coffee is Organic Drip @ The Little Cafe, currently 1.3mi away.”

    We also send out a weekly email of what your friends rated highly this week if it something close to you, or if you have searched for it in the past.

    Point is, this isn’t very much work, and works really well…

    • Peter Acworth

      I agree completely with the above. We are in the midst of replatforming kink.com, a (NSFW) adult entertainment website. We think a recommendation engine based on collaborative filtering will work well for us. However, the engine itself is not the core product and the whole point of an MVP is to get something working asap in order to start getting user feedback, so we will not start iterating on recommendation engine until well after the MVP launch.

  • Chris Daltas

    Video games are another industry ripe with recommendation engines that are not difficult, yet work very well without massive data.

    All good multiplayer games match like-skilled gamers together, otherwise it wouldn’t be fun and people would stop playing.

    So, if you are making a multiplayer video game you need to be able to automatically recommend that a gamer play with these other gamers.

    Taking a step back: A good recommendation engine relies on the ability of the product creator to select the top signals for their platform. If one attempts to use all the possible signals there will be too much noise (poor assumptions).

    For Hundo we use Distance/Friends as our only signals for recommendations.

    Meanwhile, video games use “skill rank” (strength of opponents/games played) * (success rate). This lets them recommend games with players of a similar skill level.

  • Pingback: Why Crowdmull? – The Crowdmull Blog

  • Pingback: 71 useful articles on online behavior change in 2013 you can choose not to read | Mats Stafseng Einarsen

  • http://www.aboutourwork.com/hunegnaw David Hunegnaw

    I just stumbled upon your article, hope I’m not too terribly late to the conversation…

    We’ve been working on a platform/network for small business owners for the past two years, one that can identify potential B2B opportunities and then automate introductions between members. Think “eHarmony for small businesses.”

    Our current platform is more “matching” vs “recommendations” and those matches are base on key data points collected (location, industry, tags/keywords, etc.) from the small business member when they create a profile on our network. Once the member creates a profile for their business, we then ask them to create “MatchUps” (think Google Alerts) for each type of business they’d like to connect with… for example:

    - I’m a “wedding planner” and I would like to connect with “event photographers”
    - I’m a “wedding planner” and I would like to connect with “cake decorators”
    - I’m a “wedding planner” and I would like to connect with “travel planners”

    or

    - I’m a “web developer” and I would like to connect with “copyrwriters”
    - I’m a “web developer” and I would like to connect with “SEO experts”
    - I’m a “web developer” and I would like to connect with “front-end designers”

    Once they create each MatchUp, we auto-suggest appropriate tags/keywords and allow the member to choose an unlimited number of them for their MatchUp.

    We currently have 125+ users and 3 million datapoints on our users. We hope that over time we can evolve to become a true recommendation engine that can predict and identify b2b opportunities for our members.

    There isn’t a single small business on this planet that can work in a bubble, without b2b relationships. As you mentioned in your post – “the user must justify the effort required to offer input on personal tastes.” If we can successfully predict & identify the right opportunities at the right time for our small business members, we believe that we can bring real value to each small business member and ultimately transform the way small businesses network and connect with one another. No status updates required.

    Great article.

  • Sébastien Vigé

    Hello,

    Quite interesting discussion here – has any one heard about Oracle Real-Time Decisions ? It does have answers on applying Self-Learning decision management to various type of business processes including personalisation, fraud detection and profile matching use cases. More to read here: http://www.oracle.com/us/solutions/business-analytics/business-intelligence/real-time-decisions/overview/index.html

  • electioncraft

    Great article. I would argue that even Pandora is not optimal. Perhaps recommendation engine success should be viewed through a cultural or generational lens – thoughts? John Reneski jreneski@mac.com