Page MenuHomeMiraheze
Create Task
Maniphest T2355

Purchase mw3
Closed, ResolvedPublic
Actions

Description

Well, I think it's time to purchase another server again. See T1979#37483, T2335

mw1 and mw2 capacities are weak lately, causing intermittent 503s on wikis.

The other reason would be the jobrunner. Currently, it has been moved to test1 (mw1 almost got shut down because it), but there are still a lot of jobs and only one job runner doesn't seem to be enough. I think we would need to add another jobrunner to a potential mw3 so that the jobs can be balanced out.

Therefore, I think a mw3 should be considered to resolve mainly the intermittent 503 issue, but also the jobrunner issue.

Related Objects
Search...

Event Timeline

Reception123 created this task.Oct 30 2017, 08:15
Herald added subscribers: PokestarFan, Brynda1231. · View Herald TranscriptOct 30 2017, 08:15
CnocBride added a subscriber: CnocBride.Oct 30 2017, 10:26

@Reception123 How much do you think Miraheze would have to spend to purchase a new MediaWiki server?

Reception123 added a comment.Oct 30 2017, 10:43

@CnocBride Labster approves the budgets. If I remember correctly, if it's a 512 MB server it should be an extra $5/month.

Though before the budget approval, @Southparkfan should determine for sure whether mw3 is necessary.

John added a comment.Oct 30 2017, 11:48

It's 1024MB at $10/mo.

Southparkfan added a comment.Oct 30 2017, 11:49

Looking at the load graphs, I do not see major peaks that show a shortage of capacity. Are you sure the 503 issues are not caused by network issues or varnish configuration (in this case the backend probe)?

The jobrunner seems like a valid reason, though I would still prefer a second 1GB SVZ as mw3, and a jobrunner daemon on all servers. We can adjust the amount of simultaneous runJobs processes if needed.

Southparkfan assigned this task to labster.Nov 1 2017, 15:04

When I depooled mw2 on the 30th for maintenance, mw1's load became very high (besides no redundancy), so we definitely want an extra server now.

@labster what do you think about an extra server, 1GB SVZ for $10/mo?

labster added a comment.Nov 2 2017, 01:34

Yeah, the lack of redundancy is troubling. I think we're big enough now -- and have enough usage -- that we ought to have some level of redundancy, so let's go ahead and buy a new mw server.

So far as making it a jobrunner, I don't know. I'd like redundancy on this, too, even though it's less critical than mediawiki rendering.

Reception123 reassigned this task from labster to Southparkfan.Nov 2 2017, 09:12
Reception123 closed this task as Resolved.Nov 2 2017, 18:23

mw3 has been purchased by @Southparkfan

I have done the DNS and puppet configuration and have added it to Varnish. Therefore this task is done.

On a sidenode, jobrunner has not yet been enabled on mw3, yet. If there are still issues with jobs running I will discuss this with Ops.