Page MenuHomeMiraheze

Restart services running on older openssl binaries
Closed, ResolvedPublic

Description

A security announcement was made regarding two vulnerabilities in OpenSSL: https://www.openssl.org/news/secadv/20210325.txt. Both vulnerabilities have received the 'high' severity rating from OpenSSL.

Patches were installed from debian-security: see https://security-tracker.debian.org/tracker/CVE-2021-3449 and https://security-tracker.debian.org/tracker/CVE-2021-3450. However, in order to ensure the changes are propagated to processes using libssl.so, services depending on the binaries must be restarted.

Affected processes (list may be incomplete):

  • bacula-fd
  • glusterfs (as in the client, as found on MediaWiki servers)
  • icinga2
  • ircecho
  • kvm (on proxmox hosts)
  • master
  • mongod
  • mysqld
  • node (citoid, restbase, proton)
  • nginx
  • nrpe
  • php-fpm7.3
  • pvedaemon
  • pveproxy
  • pvestatd
  • python3 (presumably adminlogbot, ircrcbot and icinga2-irclog)
  • spiceproxy
  • syslog-ng
  • systemd (restarting the init process is equal to rebooting the server, obviously)
  • systemd-logind
  • systemd-journald
  • systemd-udevd
  • tuned
  • qmgr
  • stunnel4
  • zed

General analysis:

  • CVE-2021-3449: potential DoS attack against servers, not on clients. Services that are not public facing - such as Bacula and MariaDB - are unlikely to be impacted. Advice: restart cache proxies and other servers with public facing nginx first,
  • CVE-2021-3450: absolutely no idea if/where X509_V_FLAG_X509_STRICT is used, usage does not seem to be very common. Not sure if Miraheze uses it anywhere.

Advice:

  • Rebooting is the safest option, but in certain cases (e.g. proxmox hosts), manually restarting services is doable. systemd services are not using TLS to my knowledge anyways. Therefore: reboot VMs that can be depooled and repooled, keep critical servers (proxmox hosts, database servers, DNS servers, monitoring hosts) online for now.
  • Start with public facing servers / services first, anything that is critical but firewalled has less exposure. Unless other instructions have been received, skip critical services on the following hosts: db1[1-3], cloud[3-5], mon2 and ns[12].

Event Timeline

19:58:18 <+SPF|Cloud> my advice: reboot all VMs with services that can be depooled and repooled easily, in order to preserve uptime, do it the normal way (adhere to the 5 minutes DNS TTL, depool from varnish, wait until requests have finished, etc)
20:00:49 <+SPF|Cloud> on the critical servers, db1[1-3], cloud[3-5], mon2 and ns[12], restarting syslog-ng / IRC bots is fine, anything else shouldn't be touched (yet)

I've done all servers apart from the critical ones you mentioned (instead I just restarted syslog-ng and where approbate the irc bots).

Servers that haven't been rebooted, except for db1[1-3] / cloud[3-5] / mon2 / ns[12]:

  • dbbackup1
  • dbbackup2
  • mem1
  • mem2

Do we have an update on this? Also, who is taking responsibility for this?

I've restarted all the servers that I could without causing downtime. I'm waiting on SPF for the next moves.

I recommend rebooting the dbbackup servers. They may or may not be affected by CVE-2021-3450, but as long as these servers are rebooted gracefully, we can survive without them for a few minutes.

The vulnerable services on cloud and db hosts sit behind firewalls. The likelihood of those services being attacked is low. Even if such an attack would occur because other servers in the network have been compromised, the maximum impact is a DoS / reboot. Since a reboot fixes the issue anyways, I am not very concerned. A successful exploit does not impact the integrity or confidentiality of Miraheze data. Given the low likelihood and the presence of sufficient remediation (rebooting a server will fix the vulnerability), I'll accept the risk. The only public facing service (non-SSH) on ns[12] is GDNSD, which doesn't use TLS anyways, so there's no impact there.

I've restarted both dbbackup1 and dbbackup2. I think we can close this as resolved per your acceptance of a low risk.

@Southparkfan should we make this task public viewable?

Yes.

@Southparkfan should we make this task public viewable?

Southparkfan changed the visibility from "Custom Policy" to "Public (No Login Required)".Sun, Mar 28, 22:40
Southparkfan changed the edit policy from "Custom Policy" to "All Users".