Currently, access/audit and error logs for all of our services are stored locally, which makes exploring logs harder. Miraheze should aim for a centralised (probably ELK, though there are other options) logging stack providing role-based access control.
Rollout (last update: 2021-01-01 19:59 UTC) status:
- [x] bacula2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @John to verify all bacula logs are sent to syslog instead of local files
- [x] cloud3.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @Paladox to verify all proxmox logs are sent to syslog instead of local files
- [x] cloud4.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @Paladox to verify all proxmox logs are sent to syslog instead of local files
- [x] cloud5.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @Paladox to verify all proxmox logs are sent to syslog instead of local files
- [] cp3.miraheze.org: NOT done yet @Southparkfan
- [] cp10.miraheze.org: NOT done yet @Southparkfan
- [] cp11.miraheze.org: NOT done yet @Southparkfan
- [] cp12.miraheze.org: NOT done yet @Southparkfan
- [] db11.miraheze.org: syslog daemon present and logging, MariaDB logging NOT DONE yet @Southparkfan
- [] db12.miraheze.org: syslog daemon present and logging, MariaDB logging NOT DONE yet @Southparkfan
- [] db13.miraheze.org: syslog daemon present and logging, MariaDB logging NOT DONE yet @Southparkfan
- [x] gluster3.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @Paladox to verify all gluster logs are sent to syslog instead of local files
- [x] gluster4.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @Paladox to verify all gluster logs are sent to syslog instead of local files
- [] graylog2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, graylog internal logging not checked yet (@Paladox)
- [] jobrunner3.miraheze.org: jobrunner/jobchron logging DONE, php-fpm logging DONE, cron logs NOT DONE yet @Southparkfan
- [] jobrunner4.miraheze.org: jobrunner/jobchron logging DONE, php-fpm logging DONE, cron logs NOT DONE yet @Southparkfan
- [x] ldap2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, slapd logs seem to be fine
- [x] mail2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, postfix/dovecot/roundcubemail logs are sent to syslog
- [x] mon2.miraheze.org: NOT done yet, dependency check for icinga logs (ie are local logs needed for icinga-miraheze IRC bot?)
- [x] mw8.miraheze.org: syslog daemon present and remote logging (graylog) enabled, nginx logging DONE, php-fpm logging DONE
- [x] mw9.miraheze.org: syslog daemon present and remote logging (graylog) enabled, nginx logging DONE, php-fpm logging DONE
- [x] mw10.miraheze.org: syslog daemon present and remote logging (graylog) enabled, nginx logging DONE, php-fpm logging DONE
- [x] mw11.miraheze.org: syslog daemon present and remote logging (graylog) enabled, nginx logging DONE, php-fpm logging DONE
- [x] ns1.miraheze.org: syslog daemon present and remote logging (graylog) enabled, GDNSD logging done, confirmed working
- [x] ns2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, GDNSD logging done, confirmed working
- [x] phab2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, phd logs go to syslog
- [x] puppet3.miraheze.org: syslog daemon present and remote logging (graylog) enabled, probably still a lot of puppet daemons logging to local files (@Paladox )
- [x] rdb3.miraheze.org: syslog daemon present and remote logging (graylog) enabled, redis logging done, confirmed working
- [x] rdb4.miraheze.org: syslog daemon present and remote logging (graylog) enabled, redis logging done, confirmed working
- [x] services3.miraheze.org: syslog daemon present and remote logging (graylog) enabled, citoid/proton/restbase/electron logging DONE
- [x] services4.miraheze.org: syslog daemon present and remote logging (graylog) enabled, nginx logging DONE, citoid/proton/restbase/electron logging DONE
- [x] test3.miraheze.org: syslog daemon present and remote logging (graylog) enable, nginx logging DONE, php-fpm logging DONE