Page MenuHomeMiraheze
Create Task
Maniphest T8608

MediaWiki performance issues
Closed, ResolvedPublic
Actions

Description

This needs looking into ASAP (as in drop what ever you doing and look into it).

This has been a long running issue, but now is the point it just needs fixing.

Revisions and Commits

R9 MediaWiki configuration
R9:da50161a3434 Redo caching configuration (#4393)
R9:6fa5745c9e6d Set luasandbox maxLangCacheSize to 30
R9:0d6ea52f64bb Set shell limits
R9:4f440e66894b Set shell limits (on mw101 only for now) (#4388)
R9:250a4b530b48 All servers
R9:c459f7328559 Lower $wgMaxShellMemory to 210MB
R9:5f6fa2a23771 Set $wgShellCgroup for use of control group (#4389)
rPUPC Puppet Configuration
rPUPC23623dda41ca MediaWiki: support control groups (#2341)
rPUPC1379f9469711 ImageMagick: limit memory usage to 128MB (#2340)
rPUPC381f44c1e96c Add more to ImageMagick policy file (#2339)

Event Timeline

Paladox triaged this task as Unbreak Now! priority.Jan 15 2022, 20:54
Paladox created this task.
Herald added a subscriber: Agent_Isai. · View Herald TranscriptJan 15 2022, 20:54
Void subscribed.Jan 15 2022, 23:38
Dmehus subscribed.Jan 15 2022, 23:54
Edward_Chernenko subscribed.Jan 16 2022, 01:56
Unknown Object (User) added a comment.Jan 16 2022, 04:35

I've done quite a bit of work on this today, adjusting shell limits, and changing ImageMagick limits. I think it helped a bit, as load seemed to go down and numerous reports of it loading faster now. I also plan to do some additional work on this by using control groups for memory limiting to hopefully resolve some of the remaining issues as well.

Unknown Object (User) removed RhinosF1 as the assignee of this task.Jan 16 2022, 05:25
Unknown Object (User) moved this task from Backlog to Short Term on the MediaWiki (SRE) board.
Unknown Object (User) added a project: Performance.
Unknown Object (User) added a subscriber: RhinosF1.
Unknown Object (User) claimed this task.Jan 17 2022, 03:16

Most of my changes had a huge positive impact on the performance if it maintains as current for another day or so I will likely close this task as resolved.

Meta's main page loads in under 500ms consistently for me now.

Herald added a project: Universal Omega. · View Herald TranscriptJan 17 2022, 03:16
Unknown Object (User) moved this task from Unsorted to Short Term on the Universal Omega board.Jan 17 2022, 03:16
Unknown Object (User) added a comment.Jan 17 2022, 03:54

Just to note this seems to have also resolved large pages not loading (resulting in 502s) as well.

Unknown Object (User) added a comment.EditedJan 17 2022, 04:05

I did alot of changes for performance, but https://github.com/miraheze/mw-config/pull/4393 seems to have had the most positive impact, followed by https://github.com/miraheze/puppet/pull/2341, and https://github.com/miraheze/mw-config/pull/4389.

I've also set some limits for shell execution, at https://github.com/miraheze/mw-config/blob/963c10a/Defines.php#L408-L410 which seems to have had a pretty positive effect on load as well.

Additionally, I made some changes to ImageMagick (https://github.com/miraheze/puppet/pull/2339, and https://github.com/miraheze/puppet/pull/2340) which may or may not have had a huge impact by themselves but they couldn't have hurt.

Dmehus awarded a token.Jan 17 2022, 07:16
Unknown Object (User) closed this task as Resolved.Jan 17 2022, 16:41
Unknown Object (User) added commits: rPUPC381f44c1e96c: Add more to ImageMagick policy file (#2339), rPUPC1379f9469711: ImageMagick: limit memory usage to 128MB (#2340), R9:5f6fa2a23771: Set $wgShellCgroup for use of control group (#4389), rPUPC23623dda41ca: MediaWiki: support control groups (#2341), R9:c459f7328559: Lower $wgMaxShellMemory to 210MB, R9:250a4b530b48: All servers, R9:4f440e66894b: Set shell limits (on mw101 only for now) (#4388), R9:0d6ea52f64bb: Set shell limits, R9:6fa5745c9e6d: Set luasandbox maxLangCacheSize to 30, R9:da50161a3434: Redo caching configuration (#4393).

Marking as resolved since this improvement lasted for awhile. If the issues come up again it can be reopened.