Page MenuHomeMiraheze

Setup centralised logging for services
Open, NormalPublic

Description

Currently, access/audit and error logs for all of our services are stored locally, which makes exploring logs harder. Miraheze should aim for a centralised (probably ELK, though there are other options) logging stack providing role-based access control.

Event Timeline

Paladox triaged this task as Normal priority.Dec 31 2019, 16:02
Southparkfan renamed this task from Setup logstash and send the logs to it to Setup centralised logging for services.Jan 29 2020, 23:34
Southparkfan updated the task description. (Show Details)

Conversation from the staff channel is:

@Zppix brought up that the WMF use logstash, @Southparkfan brought up graylog. Based on reading up, graylog is less to install then the logstash setup.

The logstash setup is made for big data, whereas graylog is made for logs.

Heres one source https://medium.com/@logicify/advantages-of-graylog-grafana-compared-to-elk-stack-a7c86d58bc2c

Some questions we can use for logstash vs graylog:

<+SPF|Cloud> what are the differences between solutions X and Y? what are the pros and cons of both? does either solution lack something we consider to be critical? what are the recommendations of people on internet in discussions comparing both solutions?

<+SPF|Cloud> is one solution more secure than the other (support for transit and at-rest encryption? security track record)? are there performance differences (requires less resources to do the same thing)? how easy is it to setup new log sources? is it to be expected that one of the solutions requires less maintenance than the other?

<+SPF|Cloud> this list is not exhaustive, you may find some questions to be irrelevant or missing (use your own judgement) - but they allow you to make a good comparison between these solutions

Note from staff channel: ELK stack vs graylog (though both ones use Elasticsearch).