Icinga2 has the ability to send monitoring events onto Graylog. We could then use this to correlate monitoring alerts more easily with logs, therefore improving debugging and increasing the time to resolution for certain incidents.

When we hit random 503s as well in a minute time period at 2am when no one is around, we could be able to investigate these in the morning by then being able to retrieve all the logs that came in immediately before the PROBLEM log event until the RESOLVED log event.