Page MenuHomeMiraheze

RhinosF1 (Samuel)
Sysadmin

Projects

User Details

User Since
May 24 2019, 17:22 (62 w, 5 d)
Availability
Busy Busy at E120: SRE Duty until Aug 9.
IRC Nickname
RhinosF1
GitHub User
RhinosF1
Miraheze User
RhinosF1 [ Global Accounts ]

FOSS developer and Miraheze Sysadmin.

See https://login.miraheze.org/wiki/User:RhinosF1

Recent Activity

Yesterday

RhinosF1 triaged T6021: Fix test2 permissions for /srv/mediawiki/config as Normal priority.
Wed, Aug 5, 22:04 · Site Reliability Engineering, Configuration

Tue, Aug 4

RhinosF1 renamed T6016: /host/v1 to /api/rest_v1 rewrite rule doesn't seem to work from Help with Citoid Setup to /host/v1 to /api/rest_v1 rewrite rule doesn't seem to work.
Tue, Aug 4, 22:02 · Site Reliability Engineering, Extensions, Configuration
RhinosF1 triaged T6016: /host/v1 to /api/rest_v1 rewrite rule doesn't seem to work as Normal priority.
Tue, Aug 4, 22:00 · Site Reliability Engineering, Extensions, Configuration
RhinosF1 added a comment to T6016: /host/v1 to /api/rest_v1 rewrite rule doesn't seem to work.

There's a rewrite rule at https://github.com/miraheze/puppet/blob/b81b54995d672b3549903635c0d4376bf1628d02/modules/mediawiki/templates/mediawiki-includes.conf.erb#L67 which should make this not happen.

Tue, Aug 4, 22:00 · Site Reliability Engineering, Extensions, Configuration
RhinosF1 added a comment to T6016: /host/v1 to /api/rest_v1 rewrite rule doesn't seem to work.

Hmm, that looks wrong. I'll get someone to look at this tommorow.

Tue, Aug 4, 21:57 · Site Reliability Engineering, Extensions, Configuration

Mon, Aug 3

RhinosF1 closed T5995: Import for wizpedia101wiki as Resolved.
Mon, Aug 3, 22:20 · Production Error, Import, MediaWiki
RhinosF1 added a comment to T5995: Import for wizpedia101wiki.

Thanks for your assistance! I really appreciate it. Hope the error gets fixed easily

Mon, Aug 3, 22:02 · Production Error, Import, MediaWiki
RhinosF1 is attending E120: SRE Duty.
Mon, Aug 3, 07:55

Sun, Aug 2

RhinosF1 closed T6009: 503 Backend fetch failed as Invalid.

Long imports will timeout and show a 503. Please create a seperate task with the file to import and wiki and we will do it server side.

Sun, Aug 2, 16:27 · Site Reliability Engineering
RhinosF1 added a comment to T5994: Upgrade jobrunner capabilities .
In T5994#117426, @John wrote:

I was under the impression @Reception123 or @Paladox were dealing with this the other day.

Sun, Aug 2, 09:29 · Site Reliability Engineering
RhinosF1 added a comment to T5994: Upgrade jobrunner capabilities .
In T5994#117422, @John wrote:

Why does SPF need to approve something anyone in SRE can do of their own volition? If this is such an important issue, you’re creating a slowdown intentionally

Can it be approved then? Multiple members of SRE have seen this discussion and not actively stated an approval / decline.

Sun, Aug 2, 08:45 · Site Reliability Engineering
RhinosF1 added a comment to T6006: Migrate to a Kafka Job Queue.
In T6006#117420, @John wrote:

“ would require more resources to implement - potentially more than we are able to suitably give.” is a vital thing you did not reply to, the more critical part.

That's something I'm going to look into more

Sun, Aug 2, 08:44 · MediaWiki, Site Reliability Engineering
RhinosF1 added a comment to T6006: Migrate to a Kafka Job Queue.

Would not solve problems

There's 100% duplication in the way jobs are processed. If the duplication detections works as I understand, it would reduce the load as less jobs to process. Concurrency limiting would also reduce the amount of resources a single type of job can take up and therefore reduce the impact of a spike in edits on loginwiki.

Sun, Aug 2, 08:15 · MediaWiki, Site Reliability Engineering
RhinosF1 added a comment to T5994: Upgrade jobrunner capabilities .

See T6006 as well, I think they'll go hand in hand well

Sun, Aug 2, 07:55 · Site Reliability Engineering
RhinosF1 moved T6006: Migrate to a Kafka Job Queue from Radar to New Features/Services on the Site Reliability Engineering board.
Sun, Aug 2, 07:55 · MediaWiki, Site Reliability Engineering
RhinosF1 triaged T6006: Migrate to a Kafka Job Queue as High priority.
Sun, Aug 2, 07:55 · MediaWiki, Site Reliability Engineering
RhinosF1 assigned T5994: Upgrade jobrunner capabilities to Southparkfan.

@Zppix had to restart the jobrunner service again last night due to lag.

Sun, Aug 2, 07:25 · Site Reliability Engineering

Sat, Aug 1

RhinosF1 added a comment to T5999: Extension:Tabs - MobileTabsPlugin.

Opened https://github.com/fuerthwiki/MobileTabsPlugin/pull/1 for

There's also a debugging statement left in extensions/MobileTabsPlugin/resources/tabs2accordion.js, which should be taken out but which doesn't really hurt anything.

Sat, Aug 1, 12:20 · Extensions
RhinosF1 moved T6001: Extension:AgeParse from Backlog to Review Needed on the Extensions board.
Sat, Aug 1, 12:17 · Amanda Catherine, Extensions
RhinosF1 edited projects for T6001: Extension:AgeParse, added: Extensions; removed MediaWiki.
Sat, Aug 1, 12:17 · Amanda Catherine, Extensions
RhinosF1 moved T5975: Extension:MediaWiki-CalendarConverter from Review Needed to Actions Needed (Review) on the Extensions board.
Sat, Aug 1, 12:13 · Upstream, Extensions
RhinosF1 closed T6003: Extension:Labeled_Section_Transclusion as Invalid.

Already available. Please check Special:ManageWiki/extensions before filing tasks.

Sat, Aug 1, 11:46 · Extensions
RhinosF1 claimed T5863: Extension:Score disabled due to multiple security issues.
Sat, Aug 1, 11:04 · Extensions, Security
RhinosF1 renamed T5863: Extension:Score disabled due to multiple security issues from Extension:Score disabled due to security issue to Extension:Score disabled due to multiple security issues.
Sat, Aug 1, 10:42 · Extensions, Security
RhinosF1 changed the status of T5863: Extension:Score disabled due to multiple security issues from Open to Stalled.

Per above

Sat, Aug 1, 10:41 · Extensions, Security
RhinosF1 added a comment to T5863: Extension:Score disabled due to multiple security issues.

Score restricted in ManageWiki until it's resolved.

Sat, Aug 1, 10:40 · Extensions, Security
RhinosF1 updated subscribers of T5998: OOUI library.

Cc @John as I think ManageWiki uses it so he can confirm but is it not already available?

Sat, Aug 1, 10:12 · MediaWiki

Fri, Jul 31

RhinosF1 closed T5981: Add custom meta tag to entire Wiki as Resolved.
Fri, Jul 31, 09:33 · Configuration
RhinosF1 claimed T5981: Add custom meta tag to entire Wiki.
Fri, Jul 31, 09:20 · Configuration
RhinosF1 updated subscribers of T5993: jobrunner1 temporality was unable to run jobs leading to a critical backlog..

@RobLa: I'll talk to @Southparkfan about how we don't get here again but it looks to be just a case of bringing the backlog under control now.

Fri, Jul 31, 08:27 · Amanda Catherine, Site Reliability Engineering
RhinosF1 renamed T5993: jobrunner1 temporality was unable to run jobs leading to a critical backlog. from jobrunner1 temporality was unable to run Jobs leading to a critical backlog. to jobrunner1 temporality was unable to run jobs leading to a critical backlog..
Fri, Jul 31, 07:41 · Amanda Catherine, Site Reliability Engineering
RhinosF1 renamed T5993: jobrunner1 temporality was unable to run jobs leading to a critical backlog. from jobrunner has stopped running jobs to jobrunner1 temporality was unable to run Jobs leading to a critical backlog..
Fri, Jul 31, 07:41 · Amanda Catherine, Site Reliability Engineering
RhinosF1 added a comment to T5993: jobrunner1 temporality was unable to run jobs leading to a critical backlog..

my runJobs script was accidently on mw7 but load looks fine so I'm leaving it there to avoid restarting it. I'll move if it causes an issue.

Fri, Jul 31, 07:40 · Amanda Catherine, Site Reliability Engineering
RhinosF1 lowered the priority of T5993: jobrunner1 temporality was unable to run jobs leading to a critical backlog. from Unbreak Now! to High.
Fri, Jul 31, 06:54 · Amanda Catherine, Site Reliability Engineering
RhinosF1 updated the task description for T5988: Review all custom domains with unusual rDNS entries.
Fri, Jul 31, 06:53 · SSL
RhinosF1 added a comment to T5993: jobrunner1 temporality was unable to run jobs leading to a critical backlog..

A restart of jobrunner1 seems to have stopped it failing. Let's hope runJobs.php clears the backlog and it stays this way.

Fri, Jul 31, 06:20 · Amanda Catherine, Site Reliability Engineering
RhinosF1 added a comment to T5993: jobrunner1 temporality was unable to run jobs leading to a critical backlog..

It's my understanding from chatting with @RhinosF1 that some of the jobs in the job queue are getting backed up. I'm pretty rusty on my MediaWiki skills, so I'm not yet familiar with with which set of jobs, and whether the problem is with something in MediaWiki core or with one of the many Miraheze custom jobs.

It's anything ran by the jobrunner

Fri, Jul 31, 05:19 · Amanda Catherine, Site Reliability Engineering
RhinosF1 updated the task description for T5993: jobrunner1 temporality was unable to run jobs leading to a critical backlog..
Fri, Jul 31, 04:58 · Amanda Catherine, Site Reliability Engineering
RhinosF1 triaged T5993: jobrunner1 temporality was unable to run jobs leading to a critical backlog. as Unbreak Now! priority.
Fri, Jul 31, 04:54 · Amanda Catherine, Site Reliability Engineering

Thu, Jul 30

RhinosF1 added a comment to T5943: Problem with Previously Manually Patrolled Revisions Now Unpatrolled across WIkis.

I can just do a mass mark for meta when at my pc. I'm happy to do that as based on my local authority.

Thu, Jul 30, 20:54 · MediaWiki
RhinosF1 triaged T5990: remove custom domain wiki.bydorn.de as High priority.
Thu, Jul 30, 19:53 · SSL
RhinosF1 added a subtask for T5988: Review all custom domains with unusual rDNS entries: T5990: remove custom domain wiki.bydorn.de.
Thu, Jul 30, 19:53 · SSL
RhinosF1 added a parent task for T5990: remove custom domain wiki.bydorn.de: T5988: Review all custom domains with unusual rDNS entries.
Thu, Jul 30, 19:53 · SSL
RhinosF1 changed the status of T5988: Review all custom domains with unusual rDNS entries from Open to Stalled.

Stalling for 7 days, then will send 2nd notification.

Thu, Jul 30, 10:01 · SSL
RhinosF1 updated the task description for T5988: Review all custom domains with unusual rDNS entries.
Thu, Jul 30, 09:56 · SSL
RhinosF1 updated the task description for T5988: Review all custom domains with unusual rDNS entries.
Thu, Jul 30, 09:38 · SSL
RhinosF1 updated the task description for T5988: Review all custom domains with unusual rDNS entries.
Thu, Jul 30, 08:57 · SSL
RhinosF1 added a comment to T5988: Review all custom domains with unusual rDNS entries.

All not serving Miraheze have been pulled and unset, I will alert crats for them wikis shortly.

Thu, Jul 30, 08:54 · SSL
RhinosF1 updated the task description for T5988: Review all custom domains with unusual rDNS entries.
Thu, Jul 30, 08:23 · SSL

Wed, Jul 29

RhinosF1 added a comment to T5988: Review all custom domains with unusual rDNS entries.

Wikis where 3 applies will be removed in the morning. For 1/2, as above crats will have 14 days to make the change.

Wed, Jul 29, 21:34 · SSL
RhinosF1 added a comment to T5988: Review all custom domains with unusual rDNS entries.

As discussed on IRC, we'll send 2 notifications within the next 14 days. No response = CD revoked.

Wed, Jul 29, 21:22 · SSL
RhinosF1 added a comment to T5986: Mass deletion of pages with the "Thread:" prefix on Real Life Villains Wiki.

He can revert his own change, we can fix it without touching ManageWiki.

Wed, Jul 29, 19:58 · MediaWiki
RhinosF1 added a comment to T5986: Mass deletion of pages with the "Thread:" prefix on Real Life Villains Wiki.

If they want it deleting, then just get them to give the go ahead and we'll take it from there. Don't worry, there's a script to fix this kind of thing when creating namespaces.

Wed, Jul 29, 19:51 · MediaWiki

Tue, Jul 28

RhinosF1 added a comment to T5982: mw7/4 went down (and caused a mini outage).
In T5982#116884, @John wrote:

So, resolved then?

Tue, Jul 28, 19:45 · Site Reliability Engineering
RhinosF1 updated the task description for T5982: mw7/4 went down (and caused a mini outage).
Tue, Jul 28, 15:58 · Site Reliability Engineering
RhinosF1 updated the task description for T5982: mw7/4 went down (and caused a mini outage).
Tue, Jul 28, 15:58 · Site Reliability Engineering
RhinosF1 added a comment to T5982: mw7/4 went down (and caused a mini outage).

Also noting that the dns checks took up that much of icinga it was impossible to see at the time what on earth happened.

Tue, Jul 28, 15:56 · Site Reliability Engineering
RhinosF1 triaged T5982: mw7/4 went down (and caused a mini outage) as High priority.
Tue, Jul 28, 15:55 · Site Reliability Engineering
RhinosF1 created T5982: mw7/4 went down (and caused a mini outage).
Tue, Jul 28, 15:55 · Site Reliability Engineering
RhinosF1 reopened T5978: Set $wgNamespaceRobotPolicies to noindex for NS_SPECIAL as "Open".

that's not merged

Tue, Jul 28, 14:50 · Configuration
RhinosF1 claimed T5978: Set $wgNamespaceRobotPolicies to noindex for NS_SPECIAL.
Tue, Jul 28, 14:50 · Configuration
RhinosF1 closed T5462: Fix RC Relay sleep after message as Resolved.

We fixed this the other week. It now works.

Tue, Jul 28, 13:40 · Site Reliability Engineering
RhinosF1 added a comment to T5942: Miraheze Phabricator not remembering login session.

Should we go and file a report with them then?

Tue, Jul 28, 08:46 · Phabricator

Mon, Jul 27

RhinosF1 closed T5936: Extension:Expiry as Declined.

It requires the BlueSpiceFoundation extension, so it sounds like this is not possible.

Mon, Jul 27, 22:24 · Extensions
RhinosF1 triaged T5978: Set $wgNamespaceRobotPolicies to noindex for NS_SPECIAL as Normal priority.
Mon, Jul 27, 18:48 · Configuration
RhinosF1 closed T5977: Enable .stl file uploads on nmfwiki as Resolved.
Mon, Jul 27, 13:53 · Configuration
RhinosF1 claimed T5977: Enable .stl file uploads on nmfwiki.
Mon, Jul 27, 13:52 · Configuration

Sun, Jul 26

RhinosF1 closed T5971: Support Delete Batch as Declined.

Please request this from sysadmins and we'll do the deletion. We don't allow it as this mimics as shell script.

Sun, Jul 26, 09:21 · MediaWiki
RhinosF1 moved T5970: The Visual Editor is acting up from Backlog to Deployed Extension Bugs on the Extensions board.
Sun, Jul 26, 09:03 · Upstream, Extensions
RhinosF1 edited projects for T5970: The Visual Editor is acting up, added: Extensions; removed MediaWiki.

VE isn't even bundled (yet), switching to Extensions

Sun, Jul 26, 09:03 · Upstream, Extensions

Sat, Jul 25

RhinosF1 added a comment to T5965: Create a restricted local group allowing "abusefilter-privatedetails" and "abusefilter-privatedetails-log".

Not sure whether this changes anything. But while these two permissions may be equivalent to CheckUser permissions, I wasn't requesting either of the permissions with "checkuser" in their titles. And my understanding is that the "abusefilter-privatedetails" permissions are more restrained in terms of access.

They still give access to PII. It's not allowing you to check anyone unilaterally but PII is still PII.

Sat, Jul 25, 14:34 · Configuration
RhinosF1 added a comment to T5965: Create a restricted local group allowing "abusefilter-privatedetails" and "abusefilter-privatedetails-log".

Both times there was unanimous rejection of the proposals.

Reading the closure, that was a lot to do with the format of the RfC. I'm not talking allowing wikis to just willy nilly give CU/OS to anyone. I'm talking allowing wikis with a sufficient enough community that they can elect at least 2 users who have signed an NDA by elections that are reviewed by the steward team and those users reaching support levels at lears higher than required for Steward.

Sat, Jul 25, 14:33 · Configuration
RhinosF1 renamed T5965: Create a restricted local group allowing "abusefilter-privatedetails" and "abusefilter-privatedetails-log" from Wanted to request a custom usergroup with the abusefilter-privatedetails and abusefilter-privatedetails-log permissions to Create a restricted local group allowing "abusefilter-privatedetails" and "abusefilter-privatedetails-log".
Sat, Jul 25, 14:30 · Configuration
RhinosF1 closed T5966: Extension:Page_Forms/Input_types/Datepicker as Invalid.

Based on what I've seen and Amanda's comment, closing as invalid.

Sat, Jul 25, 14:26 · Extensions
RhinosF1 added a comment to T5965: Create a restricted local group allowing "abusefilter-privatedetails" and "abusefilter-privatedetails-log".

These rights are fundamentally equivalent to CheckUser permissions. You would definitely need to sign the NDA if granted access to these permissions. Given that CheckUser and Oversight permissions are not given out to non-stewards, I don't see why an exception would be made here, unless there is some unusual/extenuating reason for this. If it is the case that the rights will be granted to anyone who has signed the NDA (which I doubt is the case), that is not reflected in the relevant policies/documentation.

Sat, Jul 25, 14:25 · Configuration
RhinosF1 added a comment to T5966: Extension:Page_Forms/Input_types/Datepicker.

I don't see any reason why this needs to be stalled. This is part of the Page Forms extension which I believe we already have, so I'm not sure if this needs a separate security review/install or not.

Sat, Jul 25, 14:17 · Extensions
RhinosF1 added a comment to T5967: cp7 health check flapping constantly and triggering DNS Critical alerts.

https://grafana.miraheze.org/d/iWQm-pOZz/nginx-appservers?viewPanel=12&orgId=1&from=now-15m&to=now&var-instance=mw5.miraheze.org:9113 shows mw5 nginx issues same as last night when it crashed?

Sat, Jul 25, 09:34 · Site Reliability Engineering
RhinosF1 renamed T5967: cp7 health check flapping constantly and triggering DNS Critical alerts from Cache Proxy (cp7?) health check flapping constantly to cp7 health check flapping constantly and triggering DNS Critical alerts.
Sat, Jul 25, 09:28 · Site Reliability Engineering
RhinosF1 triaged T5967: cp7 health check flapping constantly and triggering DNS Critical alerts as High priority.
Sat, Jul 25, 09:23 · Site Reliability Engineering
RhinosF1 added a comment to T5939: Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails.

Can this be closed?

Sat, Jul 25, 09:22 · MediaWiki, Amanda Catherine, Production Error, Site Reliability Engineering
RhinosF1 added a comment to T5966: Extension:Page_Forms/Input_types/Datepicker.

Why is this stalled?

Sat, Jul 25, 07:04 · Extensions

Fri, Jul 24

RhinosF1 edited projects for T5960: Enable MultiMaps on https://pointochinikia.miraheze.org/, added: Extensions; removed MediaWiki.
Fri, Jul 24, 14:23 · Extensions
RhinosF1 added a project to T5925: Polls and anonymous users: Upstream.

I'll talk to them

Fri, Jul 24, 10:53 · Upstream, Extensions
RhinosF1 added a comment to T5925: Polls and anonymous users.

From what I can see, it treats actor_id as unique but all IPs have actor_id set to 0 so when actor_id = 0 it needs to fallback to using actor_ip

Fri, Jul 24, 10:52 · Upstream, Extensions
RhinosF1 added a comment to T5925: Polls and anonymous users.

This stinks of a problem with how it handles actors + anonymous users. I'm betting Upstream.

Fri, Jul 24, 10:35 · Upstream, Extensions
RhinosF1 edited projects for T5959: Discord Webhook Integration, added: Configuration; removed MediaWiki.

Drop us an email to tech[at]miraheze[dot]org with the webhook url and we'll do it.

Fri, Jul 24, 08:11 · Configuration
RhinosF1 moved T5957: Polls are not working properly from Backlog to Deployed Extension Bugs on the Extensions board.
Fri, Jul 24, 08:10 · Extensions
RhinosF1 raised the priority of T5958: Error while creating blog entries from Low to Normal.
Fri, Jul 24, 08:09 · Extensions
RhinosF1 moved T5958: Error while creating blog entries from Backlog to Deployed Extension Bugs on the Extensions board.
Fri, Jul 24, 08:09 · Extensions
RhinosF1 moved T5024: TinyMCE alongside or instead of VisualEditor from Backlog to Review Needed on the Extensions board.

I don't see why this was taken out of the queue

Fri, Jul 24, 08:07 · Extensions
RhinosF1 added a comment to T5940: Problem with previously-autopatrolled revisions of administrators or autopatrollers become unpatrolled upon losing that status.

@RhinosF1 Should we file upstream?

Fri, Jul 24, 05:17 · Upstream, MediaWiki

Thu, Jul 23

RhinosF1 edited projects for T5951: Instant Commons, added: Extensions; removed MediaWiki.
Thu, Jul 23, 22:12 · Extensions
RhinosF1 edited projects for T5952: Extension:Header_Footer, added: Extensions; removed MediaWiki.
Thu, Jul 23, 22:11 · Extensions
RhinosF1 triaged T5955: Feature Request in my wiki the feature name 'Request for admin' And the wiki is tutorialwiki.miraheze.org as Normal priority.
Thu, Jul 23, 22:10 · Configuration
RhinosF1 triaged T5954: How can i change my logo? as Normal priority.
Thu, Jul 23, 22:10 · Configuration
RhinosF1 moved T5925: Polls and anonymous users from Backlog to Deployed Extension Bugs on the Extensions board.

This stinks of a problem with how it handles actors + anonymous users. I'm betting Upstream.

Thu, Jul 23, 22:09 · Upstream, Extensions
RhinosF1 assigned T5956: Unwanted Skin Change to Lakelimbo.
Thu, Jul 23, 22:08 · Configuration
RhinosF1 edited projects for T5953: Enable these file types on fanonpedia.miraheze.org, added: Configuration; removed MediaWiki.
Thu, Jul 23, 13:27 · Configuration

Wed, Jul 22

RhinosF1 triaged T5950: Add PDFHandler Prerequisites as Normal priority.
Wed, Jul 22, 19:55 · Extensions, Site Reliability Engineering
RhinosF1 edited projects for T5948: Enable Extension:Iframe on g12.miraheze.org, added: Extensions; removed MediaWiki.
Wed, Jul 22, 17:25 · Extensions