Currently, access/audit and error logs for all of our services are stored locally, which makes exploring logs harder. Miraheze should aim for a centralised (probably ELK, though there are other options) logging stack providing role-based access control.
Rollout (last update: 2020-12-29 22:10 UTC) status:
- [] bacula2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @John to verify all bacula logs are sent to syslog instead of local files
- [] cloud1.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @Paladox to verify all bacula logs are sent to syslog instead of local files
- [] cloud2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @Paladox to verify all bacula logs are sent to syslog instead of local files
- [] cloud3.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @Paladox to verify all bacula logs are sent to syslog instead of local files
- [] cp3.miraheze.org: NOT done yet
- [] cp6.miraheze.org: NOT done yet
- [] cp7.miraheze.org: NOT done yet
- [] cp9.miraheze.org: NOT done yet
- [] db11.miraheze.org: syslog daemon present and logging, MariaDB logging NOT DONE yet
- [] db12.miraheze.org: syslog daemon present and logging, MariaDB logging NOT DONE yet
- [] db13.miraheze.org: syslog daemon present and logging, MariaDB logging NOT DONE yet
- [] gluster1.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @Paladox to verify all gluster logs are sent to syslog instead of local files
- [] gluster2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @Paladox to verify all gluster logs are sent to syslog instead of local files
- [] graylog1.miraheze.org: syslog daemon present and remote logging (graylog) enabled, graylog internal logging not checked yet
- [] jobrunner1.miraheze.org: jobrunner/jobchron logging DONE, php-fpm logging NOT DONE yet and need to check if errors are sent correctly
- [] jobrunner2.miraheze.org: jobrunner/jobchron logging DONE, php-fpm logging NOT DONE yet and need to check if errors are sent correctly
- [] ldap1.miraheze.org: syslog daemon present and remote logging (graylog) enabled, slapd logs seem to be fine
- [] mail1.miraheze.org: syslog daemon present and remote logging (graylog) enabled, needs @John to verify postfix/dovecot/roundcubemail logs are sent to syslog instead of local files
- [] mon1.miraheze.org: NOT done yet, dependency check for icinga logs (ie are local logs needed for icinga-miraheze IRC bot?)
- [] mw4.miraheze.org: syslog daemon present and remote logging (graylog) enabled, nginx logging DONE, php-fpm logging NOT DONE yet
- [] mw5.miraheze.org: syslog daemon present and remote logging (graylog) enabled, nginx logging DONE, php-fpm logging NOT DONE yet
- [] mw6.miraheze.org: syslog daemon present and remote logging (graylog) enabled, nginx logging DONE, php-fpm logging NOT DONE yet
- [] mw7.miraheze.org: syslog daemon present and remote logging (graylog) enabled, nginx logging DONE, php-fpm logging NOT DONE yet
- [x] ns1.miraheze.org: syslog daemon present and remote logging (graylog) enabled, GDNSD logging done, confirmed working
- [x] ns2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, GDNSD logging done, confirmed working
- [] phab1.miraheze.org: syslog daemon present and remote logging (graylog) enabled, phd logs go to syslog but NOT confirmed working YET
- [] puppet2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, probably still a lot of puppet daemons logging to local files (needs to be checked)
- [x] rdb1.miraheze.org: syslog daemon present and remote logging (graylog) enabled, redis logging done, confirmed working
- [x] rdb2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, redis logging done, confirmed working
- [x] services1.miraheze.org: syslog daemon present and remote logging (graylog) enabled, citoid/proton/restbase/electron logging DONE
- [x] services2.miraheze.org: syslog daemon present and remote logging (graylog) enabled, nginx logging DONE, citoid/proton/restbase/electron logging DONE
- [] test2.miraheze.org: syslog daemon present and remote logging (graylog) enable, nginx logging DONE, php-fpm logging NOT DONE yet