The State of Open Source Logging


Create your own real time, interactive, searchable and sharable dashboards and analytics interfaces on top of Elasticsearch (, and do it from the comfort of a browser. Kibana 3 ( is an open source, Apache licensed, project for creating analytics interfaces to machine generated (or people generated!) data.

As the speed of machine generated data accumulation increases as does the importance of having up to the moment access to real time (or near real time) analytics of that data. Dashboards convey huge amounts of information in a concise, aggregated, meaningful and actionable way. And we're going to build a bunch of them!

The tool chain

We'll discuss the pluses and minuses of existing tools, what they're getting right and what we've learned from using them. We'll spend time talking about machine generated data: Logs and events from infrastructure and the tool chain (e.g. Logstash for collecting them. Shay Banon, founder of Elasticsearch, will join me to discuss why Elasticsearch works so well for storing and searching machine generated data.

Live demos

After that we'll move to real world use cases and how we can easily build interfaces to machine data that suit operations, development and management in a matter of minutes. We'll also play with some fun data: social feeds and our very own fictional company: MonkeyCo, who tracks the progress of an infinite number of monkeys distributed around the world, producing the collected works of Shakespeare

This talk will skew heavily towards live demonstration. We'll build interfaces from scratch on top of multiple real time data sets and show how they can be saved, searched and shared within an organization.

About the speaker: 
Rashid Khan is a data geek and a developer at Elasticsearch. He has a passion for everything open source and is the author of the open source log analysis tool Kibana. He has spent over a decade building infrastructure and writing tools to figure out the meaning of all those blinking lights. Shay Banon is founder of ElasticSearch, an open-source, distributed, RESTful, Search Engine. Shay also founded the Compass project in 2004, an open-source Java Search Engine library built on top of Lucene.

Schedule info

Time slot: 
3 June 14:45 - 15:30