Page MenuHomeMiraheze
Create Task
Maniphest T7582

Create automated system for managing SSL requests
Open, LowPublic
Actions

Description

Following the positive experience with ImportDump and given very limited experience with webservices, we've decided that for the time being it would be better to follow that model for SSL. These are the steps that are being/were taken

  • Step 1: Automatically copy and commit private keys
  • Step 2: Automatically push certificates to GitHub
  • Step 3: Create RequestSSL and automatically add ManageWiki $wgServer entry
  • Step 4: Create an API endpoint on puppet141 and configure it to listen to requests from the RequestSSL extension and execute '/root/ssl-certificate' when requests are marked as completed
  • Step 5: Automatically add DNS zone for wikis that have pointed their NS to Miraheze (check whether CNAME, if not add zone)
  • Step 6: Decommission the SSL admin group

<Original plan; kept as an archive; might not follow exactly>
Stage 1:

  • Update check_reverse_dns to check records present too.
  • Move SSL generation from mwtask141 to puppet141
  • Automate copying private keys
  • Automate pushing certificates from puppet141 to GitHub

Stage 2:

  • Update certbot cli to check rDNS is correct and either CNAME or NS record is present. Add argument to skip this.

Stage 3:

  • Create a web form to automate creating SSL tasks + checking validity - refuse to create if invalid.

Stage 4:

  • create a new wrapper for generating new ssl certs, include updating ManageWiki (puppet-user will be pointless at this point).

Stage 5:

  • Move all SSL requests to the new ssl self serve site and allow one click to do everything.

Revisions and Commits

rSSL SSL configuration
rSSLaa6d0d18918c Certs: move beta to top (#533)
rPUPC Puppet Configuration
rPUPC5fc09dd83150 add varnish/nginx to puppet111 (adding SSL) (#2592)
rPUPCb810b11ee3f7 switch ssl-renew IP to puppet111's IP
rPUPCb61884fef540 automate copying private keys for SSL certs (T7582) (#2600)
rPUPC2228d4514b26 letsencrypt: ensure /var/www exists (#2593)
rPUPCf49d62d0a539 Introduce `role::letsencrypt` (#2594)
rPUPCc700d9371e86 letsencrypt: use /var/www/html/ for nginx root (#2595)
rPUPC150b894bbe75 Remove LetsEncrypt from mwtask111 (#2599)
rPUPC0cbce4b6aed6 Fix varnish syntax
rPUPCa94f37091ea6 Move change-password to LetsEncrypt (#2602)
rPUPCcccc39d61c17 mediawiki: remove acme-challenge (#2603)
rPUPCe56ee5ee6185 Remove acme-challenge from roundcubemail and icingaweb2 (#2604)

Related Objects
Search...

Event Timeline

RhinosF1 created this task.Jul 3 2021, 08:48
Herald added subscribers: Bukkit, Unknown Object (User), Reception123, NDKilla. · View Herald TranscriptJul 3 2021, 08:48
RhinosF1 triaged this task as Low priority.Jul 3 2021, 08:49
RhinosF1 claimed this task.Jul 3 2021, 09:27
RhinosF1 updated the task description. (Show Details)
RhinosF1 added a comment.Jul 3 2021, 09:55

I've created a script, will add it to puppet, deploy it everywhere later.

RhinosF1 updated the task description. (Show Details)Jul 3 2021, 12:12
RhinosF1 added a subtask: T6759: Automate the adding of SSL private keys to puppet3.Jul 3 2021, 12:52
RhinosF1 added a project: SRE Automation.Jul 3 2021, 12:58
RhinosF1 updated the task description. (Show Details)EditedJul 3 2021, 21:38

Icinga will now go off if the domain points in a manner we don't expect.

I'll send a PR tommorow for updating ssl-certificate to run the check too.

After that, I will start work on the beta site for creating new requests. There is a varnish config PR to force all requests via jobrunner3 for it.

Redmin subscribed.Jul 4 2021, 12:02
Unknown Object (User) moved this task from Backlog to Goals on the MediaWiki (SRE) board.Jul 4 2021, 16:54
Unknown Object (User) moved this task from Backlog to MediaWiki on the Goal-2021-Jul-Dec board.Jul 31 2021, 00:28
Reception123 removed a subscriber: Unknown Object (User).Aug 24 2021, 19:05
John closed subtask T6759: Automate the adding of SSL private keys to puppet3 as Declined.Oct 15 2021, 21:27
John subscribed.Dec 5 2021, 21:08

Can we have an update on this goal please? Last update was on July 3rd.

Can we also have a plan for realistic completion or significant progress before this goal period is update December 31st please?

RhinosF1 added a comment.EditedDec 5 2021, 22:16

So far none. This is looking unlikely it'll be done this period.

RhinosF1 removed RhinosF1 as the assignee of this task.Dec 5 2021, 22:17
Unknown Object (User) added a comment.Jan 1 2022, 17:06

I purpose that this task does not become a goal for the new goal period, as there was no progress on it last time, and does not seem that there is anyone who intends to work on it. Goals should ideally have someone fully committed to getting them done.

Unknown Object (User) moved this task from Goals to Long Term on the MediaWiki (SRE) board.Jan 1 2022, 17:06
Unknown Object (User) removed a subscriber: Bukkit.Jan 30 2022, 22:36
Reception123 added a subscriber: Unknown Object (User).Jan 31 2022, 18:07
In T7582#172916, @Universal_Omega wrote:

I purpose that this task does not become a goal for the new goal period, as there was no progress on it last time, and does not seem that there is anyone who intends to work on it. Goals should ideally have someone fully committed to getting them done.

While this is super late, I agree that tasks should not be goals unless there is a clear and expressed desire for someone (or multiple people) to work on it. However I think it's a shame that no one wants to work on this particular task which while not urgent (since we've been doing this for 6 years now) is important to allowing for quicker resolution of SSL-related tasks.

I propose that since maybe the extent of this task is what puts people off from doing it, we start with a simpler task which would just be to automate adding private keys to puppet111 (i.e. when using the script the key is added to puppet111 automatically) therefore allowing MWEs to generate SSLs, then further automation/user-integration can be done separately.

Unknown Object (User) unsubscribed.Feb 12 2022, 07:25
Ajhalili2006 subscribed.Apr 22 2022, 15:14
Reception123 added a comment.May 9 2022, 05:18

Since there is no progress, I propose we simply move ssl-certificate to puppet111 (since no mw-admins have done any SSL tasks in recent years anyway) and have the private keys automatically added.

Restricted Repository Identity mentioned this in rPUPC96acb6a62e3f: automate copying private keys for SSL certs (T7582).May 28 2022, 05:23
Restricted Repository Identity mentioned this in rPUPCb61884fef540: automate copying private keys for SSL certs (T7582) (#2600).May 28 2022, 07:52
Reception123 added a comment.May 28 2022, 08:21

SSL certificates are now generated on puppet111 directly and private keys are automatically copied.

Unknown Object (User) subscribed.May 28 2022, 08:38
Unknown Object (User) added commits: rPUPCe56ee5ee6185: Remove acme-challenge from roundcubemail and icingaweb2 (#2604), rPUPCcccc39d61c17: mediawiki: remove acme-challenge (#2603), rPUPCa94f37091ea6: Move change-password to LetsEncrypt (#2602), rSSLaa6d0d18918c: Certs: move beta to top (#533), rPUPC0cbce4b6aed6: Fix varnish syntax, rPUPC150b894bbe75: Remove LetsEncrypt from mwtask111 (#2599), rPUPCc700d9371e86: letsencrypt: use /var/www/html/ for nginx root (#2595), rPUPCf49d62d0a539: Introduce `role::letsencrypt` (#2594), rPUPC2228d4514b26: letsencrypt: ensure /var/www exists (#2593), rPUPCb61884fef540: automate copying private keys for SSL certs (T7582) (#2600), rPUPCb810b11ee3f7: switch ssl-renew IP to puppet111's IP, rPUPC5fc09dd83150: add varnish/nginx to puppet111 (adding SSL) (#2592).May 28 2022, 08:41
Reception123 updated the task description. (Show Details)May 28 2022, 11:00
LisafBia subscribed.Jun 6 2022, 17:43
Unknown Object (User) updated the task description. (Show Details)Nov 10 2022, 16:45
Restricted Repository Identity mentioned this in rPUPC3b25afd23868: allow direct pushing of certs to GitHub T7582.Nov 11 2022, 13:09
Reception123 added a comment.Nov 11 2022, 13:59

Opened https://github.com/miraheze/puppet/pull/2996.

Due to the fact that the initial options seems more complicated, our currently shorter term plan would be to implement it in a similar way to ImportDump and in the end only have SRE copy/paste the command for everything.

For automatic approve = creation, that will be a longer term goal that will likely be separated out in separate tasks. For now I think it's enough of an improvement if we can get to the point where SRE members just copy a command and that's it.

Reception123 updated the task description. (Show Details)Nov 26 2022, 18:48
Restricted Repository Identity mentioned this in rPUPCa890fcdd62cf: allow direct pushing of certs to GitHub T7582 (#2996).Nov 26 2022, 19:01
Reception123 added a comment.EditedDec 15 2022, 11:57

While this hasn't been fully tested yet here is my not-so-perfect version of step 3 which I unfortunately didn't integrate into the ssl-certificate script due to lack of knowledge on how to do so properly and not wanting to make things too messy. Here is the current script that can be used in the meantime (not 100% sure if DNS works yet): https://phabricator.miraheze.org/P474 and hopefully be integrated into the main one soon so that step 4 can follow.

Though unofficially, adding DNS zones manually is no longer necessary!

Restricted Repository Identity mentioned this in rPUPC4f12ceeb0224: ensure /srv/dns exists on puppet141 (for dns zone automation) T7582.Dec 15 2022, 12:03
Reception123 added a project: Goal-2023-Jan-Jun.Jan 2 2023, 11:21
Reception123 renamed this task from Create better system for managing SSL requests to Create automated system for managing SSL requests.Jan 20 2023, 13:20
Reception123 added a comment.Feb 1 2023, 09:30

Just so we don't forget, the current idea would be to try using https://github.com/wikimedia/acme-chief and have an API backend for ManageWiki with the web app being MediaWiki.

Herald added a subscriber: BrandonWM. · View Herald TranscriptFeb 1 2023, 09:30
Reception123 updated the task description. (Show Details)Feb 1 2023, 09:31
Reception123 added a subscriber: OrangeStar.EditedMar 14 2023, 11:58

https://github.com/Reception123/RequestSSL has been created based on ImportDump. Things that are required to make it operational

  • Add a method so that $oldstatus and $newstatus is known in order for the updateManageWiki function to be executed only when the status is changed from something else to completed
  • Implement logging (can be copied from ManageWiki) so that when RemoteWiki is executed it logs it as if someone had changed managewiki on wiki
  • Fix issues with timestamp (might just be a problem with the SQL implemented on beta)

Due to limited knowledge on my part, it would be preferable if someone else had a go at this.

  • Check if all i18n messages make sense (can be done by anyone) [DONE!]
Reception123 updated the task description. (Show Details)Mar 14 2023, 12:01
Unknown Object (User) unsubscribed.Mar 18 2023, 03:25
Reception123 added a subscriber: Joritochip.Mar 27 2023, 09:13