Real-time Learning for Fun and Profit

Track: 
scale
Speaker(s): 

I will describe how real-time Bayesian learning can optimize real processes. These processes can be web-sites, user interfaces, ad-targeting servers, back-end server farms, search engines or physical systems. Google, Microsoft and Yahoo have all adopted the algorithms described in this talk for ad targeting because they produce superior results, but you don't have to be on that scale to benefit as well.

The positive impact of these new learning techniques can be massive. Conventional techniques are harder to implement and make it harder for the consumers of the results of such tests to understand the results and to take correct actions. Worst of all, conventional techniques waste enormous amounts of precious user data making it harder to react quickly.

Counter-intuitively, while Bayesian techniques for real-time learning are based from complex mathematical theories that have only recently been fully understood, these techniques are actually very simple conceptually, are much easier to implement correctly than conventional statistical approaches and produce results that are much easier to understand, especially for non-statisticians.

In this talk, the audience will learn the basic ideas behind effective real-time learning, but also see detailed implementation techniques and learn how to architect effective testing systems. I will also cover methods for starting gently without massive system upheavals and how to build consensus around how real-time learning and optimization. The focus throughout the talk will be on practical methods that can be applied in real life.

The code described in this talk will be made freely available.

About the speaker: 
Ted Dunning is the Chief Application Architect at MapR Technologies. Ted has held Chief Scientist positions at Veoh Networks, ID Analytics and at MusicMatch, (now Yahoo Music). Ted is responsible for building the most advanced identity theft detection system on the planet, as well as one of the largest peer-assisted video distribution systems and ground-breaking music and video recommendations systems. Ted has 15 issued and 15 pending patents and contributes to several Apache open source projects including Hadoop, Zookeeper and Hbaseâ„¢. He is also a committer and/or project management committee member for Apache Mahout, Apache Zookeeper and Apache Drill. Ted earned a BS degree in electrical engineering from the University of Colorado; a MS degree in computer science from New Mexico State University; and a Ph.D. in computing science from Sheffield University in the United Kingdom. Ted also bought the drinks at the very first Hadoop User Group meeting.

Schedule info

Time slot: 
4 June 16:50 - 17:35
Room: 
Kesselhaus