Multi-modal Recommendation Algorithms

Classic collaborative filtering uses a single kind of behavior recorded as a relation between users and a single class of objects. The classic examples include people buying books, people watching movies or people listening to music.

In the real world, however, real people interact in many ways with many kinds of things. People even interact (abstractly) with intangible, abstract entities such as musical styles or different food cuisines. They say things. They go places. These many kinds of behavior give strong clues about what other things these people might like to do and recommendation engines should use these multi-modal cues to make better recommendations.

I will describe how the basic mathematical structure of recommendation engines can be extended to take account of these many kinds of recommendations using a "pivot-set" representation. This leads to an extremely straightforward outline for how to design a multi-modal recommendation algorithm.

Mathematics and algorithms are not enough, however. Real-world implementation is required.

Happily, multi-model recommendation algorithms can be implemented in a remarkably simple fashion. I will provide a complete outline of how this can be done using standard tools like Pig, Mahout and SolR. I will also provide concrete examples of how this works in the real world.