Christoph has gigs of log data and he's looking to Clojure for some help.

Introducing a new topic.
The last few weeks we focused on Twitter and automatically posting to it.
Surprised by how much there is to talk about in a focused problem.
"There will always be more problems for the world to solve."
(01:53) Imagine if you will, the world of DonutGram!
A fictitious social network where people post their donut experiences.
This series is not about making DonutGram, but living DonutGram.
"Oh, the dark underbelly of application development: support."
"Let's take the shiny rock and flip it over and look at all the worms and bugs."
We talked about forensic data before, and much of DevOps is about looking at that data.
You want to paint the most complete story. It's a development problem too.
Most of the time, forensic data is written to a log file. It's the first line of investigation.
Imagine all of the components of your application speaking into one pipe, and you now get to reconstruct what happened.
(05:59) Everything is rosy at DonutGram, but then users start having issues.
Users get the "Fail Donut" and start posting that.
We will be assuming the role of the heroic DevOps team.
There is quite a bit of data written to the log file, more and more with each bugfix.
We're going to use Clojure to investigate this problem.
(08:50) What problems might we encounter?
Problem 1: Size
- No one configured log rotation!
- The log file is too large to load for analysis.
Problem 2: Unstructured data
- Everything is a line or multiple lines.
- We can build up layers of abstractions to gradually build understanding.
Problem 3: Non-linear data
- We want to tell a story about what went wrong.
- Many times, the pieces of that story are in different areas of the log file, and must be collated.
- There can be multiple competing stories.
- The parts of the story are like dominoes. How do you know if the third domino is important without knowing that the first two have fallen?
Problem 4: Alerts
- "If you have a really good story to tell, how do you tell anyone about it?"
- Alerts should be timely, depending on the audience.
Call out to the audience: do you have battle stories about processing log files? Let us know!

Clojure in this episode:

nil

Ep 028: Fail Donut

About this episode

Comments

Rate this episode

Episode Details