Page MenuHomeMiraheze

Error: 503 Backend fetch failed
Closed, ResolvedPublic

Description

This incident holded on for some minutes. Text of the error message:

503 Backend fetch failed

Our servers are having problems at the moment. Please try again in a few minutes. There may be additional information on our Facebook and Twitter pages.

Please provide the details below if you report this error to the system administrators via https://phabricator.miraheze.org:

Error 503 Backend fetch failed, forwarded for 83.247.102.164, 127.0.0.1 (Varnish XID 1377520) via cp1 at Fri, 07 Jul 2017 08:49:25 GMT.

Event Timeline

Then some minutes of normality. Then, again, the error 503:

Our servers are having problems at the moment. Please try again in a few minutes. There may be additional information on our Facebook and Twitter pages.

Please provide the details below if you report this error to the system administrators via https://phabricator.miraheze.org:

Error 503 Backend fetch failed, forwarded for 83.247.102.164, 127.0.0.1 (Varnish XID 1737850) via cp1 at Fri, 07 Jul 2017 08:52:38 GMT.

Reception123 triaged this task as High priority.
Reception123 added a project: MediaWiki.

Way less than before but still happening (can confirm).

And after one of two minutes back to normal.

Yes, issues are still intermittent, but still should be looked at.

Reception123 lowered the priority of this task from High to Normal.Jul 12 2017, 08:24

Have not seen any errors for 5 days

Reception123 raised the priority of this task from Normal to High.Jul 15 2017, 08:21

and they're back :(

Had this error on my wiki in the once in the past week. As Reception said the error isn't as common as it was and sometimes the wiki just magically comes back out from the 503 errors. Can be a pain sometimes but manageable.

In ITIL-terms: there is a "problem" underlying and causing the incidents. I too had some 503 errors on yesterday 15th of july. Happily, in one or two minutes Mediawiki turns back to normal functioning.

John lowered the priority of this task from High to Normal.Jul 18 2017, 20:55
John subscribed.

Happens way to infrequently to warrant more than normal priority. Potentially a capacity issue so a mw3 may be in order to be looked into.

By virtue of above.

Reception123 raised the priority of this task from Normal to High.

I have personally not been experiencing this again, but multiple reports from users of this error being frequent again. @John said something about there not being a balance between mw1 and mw2, but I'm not sure how such balance can be achieved.

I don't know how varnish works but it naturally balances itself well. Unfortunately it's not aware of a fairly heavy process on mw1 which means varnish RR won't work.

Unsure if varnish has a weighted system.

I've been experiencing this for well over an hour today when attempting to access my own wiki sea.miraheze.org, or the meta wiki. The error message contains:

Error 503 Backend fetch failed, forwarded for 88.202.151.90, 127.0.0.1 (Varnish XID 3706769) via cp1 at Wed, 04 Oct 2017 11:16:23 GMT.
MacFan4000 raised the priority of this task from High to Unbreak Now!.Oct 4 2017, 12:15
MacFan4000 subscribed.

site is down so UBN

We are aware of the situation and we think we identified the cause (puppet1), but we currently have no sysadmins with access to puppet1 available, so we need to wait for one.

Error is due to db2 disk space. The temporary solution is dropping dbs in deleted.dblist.

UPDATE: I can now see the wikis :-)

Reception123 assigned this task to Southparkfan.

The permanent 503 issue was resolved by deletion of binary logs on db2
Please note that this and the occasional 503s is unrelated.

Ahmsaqib lowered the priority of this task from Unbreak Now! to Normal.Nov 25 2018, 21:42
MacFan4000 raised the priority of this task from Normal to Unbreak Now!.Nov 25 2018, 22:00

I been trying to create articles but I been seeing Error code 503

I been trying to create articles but I been seeing Error code 503

We're aware of the warning. It appears that the system is undergoing maintenance. It should be fixed in about an minutes or hour. Sorry for the inconvenience.

We should be back once mysql has finished booting back up, please open a new ticket though next time.

I been trying to create an article of NASCAR Xfinity Series but I been seeing this error but I been trying to see it 10 times

I been trying to create an article of NASCAR Xfinity Series but I been seeing this error but I been trying to see it 10 times

please open a new ticket

This is serious
503 Backend fetch failed
Our servers are having issues at the moment.

This is serious
503 Backend fetch failed
Our servers are having issues at the moment.

Per our twitter, which shows on the 503 page, we know but please also open a new ticket rather than using very old ones.

This error is getting on my nerves

This error is getting on my nerves

Please report them in a new ticket when they happen. This one is tracked at https://phabricator.miraheze.org/T5675

This error is getting on my nerves, when are you gonna get it fixed?

This error is getting on my nerves, when are you gonna get it fixed?

We are working on the incident now. Please remain calm.

This error has been popping up one hundred times 😡

This error has been popping up one hundred times 😡

We work as hard as we can. Please avoid letting frustrations get to you.

Can you get it fixed now, I don't want to wait

This error is getting on my nerves, How long until this error is resolved?

This error is getting on my nerves, How long until this error is resolved?

As soon as we can fix it we will, please allow us to handle the incident.

Can you fix it in one hour?

You can not demand we fix it in a set time. We are working as hard as we can to resolve the incident as soon as possible.

This error is getting on my nerves and you need to get it fixed NOW!!!!!!!!!!!