Page MenuHomeMiraheze

503 Backend fetch errors + uploads not working
Closed, ResolvedPublic

Description

EDIT: the site has been down for a few minutes (twice) due to 503 errors

When trying to upload a file on testwiki I get 504 Gateway error.

@MikeV has reported that he also gets the same error, as well as initially a 503 backend fetch.

Setting to UBN since this issue blocks a key feature of MediaWiki and it should be investigated ASAP.

Event Timeline

Reception123 renamed this task from Uploading files fails to 503 Backend fetch errors + uploads not working.Jun 15 2017, 17:12
Reception123 updated the task description. (Show Details)
Reception123 closed this task as Resolved.Jun 15 2017, 18:43
Reception123 claimed this task.

SPF restarted php5-fpm on mw* and issue looks fixed now.

Reception123 reopened this task as Open.Jun 15 2017, 19:13

Miraheze might be back up, but uploads still do not work.

Void added a comment.Jun 15 2017, 20:18

And, the site is back down (503s) again

Error 503 Backend fetch failed, forwarded for 2a02:2168:1bd5:e900:acbd:dfa9:14d6:6e14, 127.0.0.1 (Varnish XID 2294652) via cp1 at Thu, 15 Jun 2017 20:18:50 GMT.

Now working

MikeV added a comment.Jun 16 2017, 02:33

Not working for file upload. Tried twice...

Error 503 Backend fetch failed, forwarded for 73.245.63.95, 127.0.0.1 (Varnish XID 1345491) via cp2 at Fri, 16 Jun 2017 02:23:02 GMT.

Reception123 removed Reception123 as the assignee of this task.Jun 16 2017, 16:06

Miraheze is down. Error 503 again
Error 503 Backend fetch failed, forwarded for 2a02:2168:1bd5:e900:a879:b0b6:24f:ef79, 127.0.0.1 (Varnish XID 262177) via cp1 at Fri, 16 Jun 2017 19:56:24 GMT

Zppix added a subscriber: Zppix.Jun 16 2017, 21:14

Be advised while its an inconvience the 503 errors are known and a fix will hopefully come soon for the meantime SPF has tried to prevent them from happening as much as he can, usually the downtimes are less then ~2m tops!

Zppix edited projects, added Operations; removed MediaWiki.Jun 16 2017, 21:15
MikeV added a comment.EditedJun 16 2017, 21:21

The file upload failure does not appear to be limited to "~2M". It seems to be constant. Is that a different problem? Have not been able to upload any files in at least a day after many attempts.

"504 Gateway Time-out" keeps coming while uploading an image.

Jazzable removed a subscriber: Jazzable.Jun 16 2017, 22:22

Issues with MediaWiki as well as they seem to be coming from cp*

Corey added a subscriber: Corey.Jun 19 2017, 05:51
Redtmr1 moved this task from Backlog to Bugs on the MediaWiki board.Jun 21 2017, 00:33

504 Gateway Time-out

nginx/1.10.1

When uploading

NDKilla added a comment.Jun 26 2017, 03:03

Image uploads should now work, and I think I understand why [at least partly] they stopped working.

I'm not sure if this will fix all of the issues with 5xx errors, so I'm leaving this task open for now.

I just had the "503 Backend fetch failed" error again

Yes, same for me. Uploads are now working as expected, but there are still intermittent 503 Backend Fetch failed errors.

Reception123 closed this task as Resolved.Jun 28 2017, 17:38
Reception123 claimed this task.

After 2 days, I can confirm that there still are very infrequent 503s, and do not seem to be causing any downtime (if so, only one cache server at a time). If the issues resurfaces feel free to reopen this task.