Monitorama 2016 wrapped up yesterday and we thought a little recap was in order. The speaker lineup was well-picked as usual, with lots of great talks ranging from the humorous to the informative and everything in between. It was a packed house again this year, clocking in at nearly 600 people. Monitorama has built a reputation as one of the friendliest and most welcoming conferences in the industry; in the course of packing Portland’s Gerding Theatre to the gills, great conversations occurred and new friends were made.
There were two major themes at Monitorama this year, which I sum up as monitoring-in-theory and monitoring-in-practice.
Many talks and hallway conversations were had over the idea that monitoring is a platform, not a set of tools; it works best for people when provided as a consumable service. Many companies spoke about the importance of providing training, workshops, and other educational efforts, both on how use the monitoring service and how to approach monitoring in general.
There was a lot of focus on the human aspect of monitoring - much more than in previous years - and this resonated with the audience. Nicole Forsgren and Sarah Lake Hagan spoke about measuring people and culture, while Mark Imbriaco implored us to embrace the interaction between people and tools and not forget that we build tools *for* people, not to *replace* people. Relatedly, the topic of the human impact of alert fatigue and possible solutions was a common discussion point, both on stage and off.
On the monitoring-in-practice side of things, several of the talks focused on implementing better monitoring, ranging from cultural implications and struggles, to tips and lessons learned, to “we built a thing” stories from companies such as Pinterest, Twitter, Etsy, and Basecamp.
Talks That Blew Us Away
Monitorama has a very high bar for speaker and content quality, making it practically impossible to proclaim the best talks. It was, however, fairly easy to agree on talks that blew us (and the crowd) away.
Joe Damato, packagecloud - All of Your Network Monitoring is (Probably) Wrong
Joe impressed us with the flexing of his low-level tech knowledge. This time, he turned his attention to the topic of the Linux network stack. Following the creation of a ninety page overview of the Linux network stack for a customer (posted on the packagecloud blog), he presented us with a humorous, informative, and somewhat sobering critique of the state of the Linux network stack. Basically, it’s rife with poor documentation, bugs, and inconsistent network driver implementation—all of which conspires to make getting driver-level metrics somewhere between hard and why-god-why.
Thankfully, there’s an upside to things, Joe reminded us: you probably don’t care, and that’s okay. Here are Joe’s slides for your further enjoyment.
Justin Reynolds, Netflix - Intuition Engineering at Netflix
Monitoring UX and design are paramount to Librato, so we were quite amped up for Justin’s Vizceral demo. Vizceral is Netflix’s intuition-based traffic visualization tool, and it’s clear they have put a lot of thought into its design. Following the Great Christmas Eve AWS Outage of 2012, Netflix started building their service to run multi-region, which also necessitated being able to shift customers around regions seamlessly when problems with one region manifested. Rather than focusing on raw numbers or historical trends, Netflix wanted a way to visualize traffic at a glance and use intuition to understand whether things were “normal”. Thus, Vizceral was born. Netflix has open-sourced Vizceral, which you can find on GitHub.
Dave Josephsen, Librato - Five Lines I Couldn’t Draw
Are we biased? Sure. But we were also watching the crowds enjoy Dave’s witticisms, rawness and honesty about his experiences. The premise of Dave’s talk: good monitoring changes people. Dave realized he had been carrying a misapprehension about what monitoring was and who it was for. His prior experiences with monitoring were much like those of folks he nowadays meets at conferences: monitoring is terrible, alerts are flooding from everything, and the world is probably burning right now.
Through observing all teams - ops, data engineering, design - interact at Librato, he realized that the purpose of monitoring isn’t creating alerts but asking questions. There is no “owner” of monitoring, as everyone has the ability to measure things and ask their own questions.
The ping-pong tournament on Monday evening at Pips & Bounce returned for another round this year, and is on its way to becoming a much-loved Monitorama tradition. There were more vendor after-parties on Tuesday night than in previous years, and in Monitorama fashion, we all plowed through Blue Star donuts on Wednesday morning to distract us from the dawning realization that the conference was coming to a close. We hope to see you next year!