Structured Logging

On weekends I was reading latest Technology Radar from Thought Works.

It’s really interesting! Some of things they mentioned we are already using here at Compellotech (such as HAL, Angular JS, Nancy, TypeScript, etc.) Others are good not know about, for example Structured Logging.

Structured Logging

Lately I was thinking on the best approach to enabled monitoring in our distributed system. The goal is to have some universal solution which will allow us to release something in 1 or 2 sprints (we use one week sprint).

I did some research and I suppose it’s a way to go for us.

  • Allows to enable monitoring in all the components without significant changes
  • Allow to setup Alerts for different things (if they appear in the log)
  • Allow to search for issues for example by Customer ID. Just imagine you type CustomerID=”777” and you see all the events in the system related to that customer!

Technical details

There is a great Structured Logging framework http://serilog.net/. It allows you to publish logs to different systems. For example you would use Seq web application. Alternative approach is to publish logs to Elastic Search and use Kibana as a Front End. Another interesting option is to push logs into MongoDB. So you ‘ll be able to do something like  

db.log.find( { customerId: “777”, application: “PaymentGateway” } )