Page MenuHomeMiraheze
Create Task
Maniphest T7112

JobQueueError from line 778 of /srv/mediawiki/w/includes/jobqueue/JobQueueRedis.php: Redis server error: socket error on read socket
Closed, ResolvedPublic
Actions

Description

When running importDump.php after a few seconds getting this error:

JobQueueError from line 778 of /srv/mediawiki/w/includes/jobqueue/JobQueueRedis.                                                                             php: Redis server error: socket error on read socket

#0 /srv/mediawiki/w/includes/jobqueue/JobQueueRedis.php(240): JobQueueRedis->han                                                                             dleErrorAndMakeException(Object(RedisConnRef), Object(RedisException))
#1 /srv/mediawiki/w/includes/jobqueue/JobQueue.php(374): JobQueueRedis->doBatchP                                                                             ush(Array, 0)
#2 /srv/mediawiki/w/includes/jobqueue/JobQueue.php(344): JobQueue->batchPush(Arr                                                                             ay, 0)
#3 /srv/mediawiki/w/includes/jobqueue/JobQueueGroup.php(171): JobQueue->push(Arr                                                                             ay)
#4 /srv/mediawiki/w/includes/jobqueue/JobQueueGroup.php(210): JobQueueGroup->pus                                                                             h(Array)
#5 /srv/mediawiki/w/includes/Title.php(4225): JobQueueGroup->lazyPush(Array)
#6 /srv/mediawiki/w/includes/page/WikiPage.php(3596): Title->touchLinks()
#7 /srv/mediawiki/w/includes/Storage/DerivedPageDataUpdater.php(1585): WikiPage:                                                                             :onArticleCreate(Object(Title))
#8 /srv/mediawiki/w/includes/page/WikiPage.php(2172): MediaWiki\Storage\DerivedP                                                                             ageDataUpdater->doUpdates()
#9 /srv/mediawiki/w/includes/import/ImportableOldRevisionImporter.php(218): Wiki                                                                             Page->doEditUpdates(Object(MediaWiki\Revision\RevisionStoreRecord), Object(User)                                                                             , Array)
#10 /srv/mediawiki/w/includes/import/WikiRevision.php(669): ImportableOldRevisio                                                                             nImporter->import(Object(WikiRevision))
#11 /srv/mediawiki/w/includes/import/WikiImporter.php(371): WikiRevision->import                                                                             OldRevision()
#12 /srv/mediawiki/w/maintenance/importDump.php(201): WikiImporter->importRevisi                                                                             on(Object(WikiRevision))
#13 /srv/mediawiki/w/includes/import/WikiImporter.php(507): BackupReader->handle                                                                             Revision(Object(WikiRevision), Object(WikiImporter))
#14 /srv/mediawiki/w/includes/import/WikiImporter.php(999): WikiImporter->revisi                                                                             onCallback(Object(WikiRevision))
#15 /srv/mediawiki/w/includes/import/WikiImporter.php(864): WikiImporter->proces                                                                             sRevision(Array, Array)
#16 /srv/mediawiki/w/includes/import/WikiImporter.php(802): WikiImporter->handle                                                                             Revision(Array)
#17 /srv/mediawiki/w/includes/import/WikiImporter.php(612): WikiImporter->handle                                                                             Page()
#18 /srv/mediawiki/w/maintenance/importDump.php(353): WikiImporter->doImport()
#19 /srv/mediawiki/w/maintenance/importDump.php(286): BackupReader->importFromHa                                                                             ndle(Resource id #890)
#20 /srv/mediawiki/w/maintenance/importDump.php(130): BackupReader->importFromFi                                                                             le('/home/reception...')
#21 /srv/mediawiki/w/maintenance/doMaintenance.php(107): BackupReader->execute()
#22 /srv/mediawiki/w/maintenance/importDump.php(358): require_once('/srv/mediawi                                                                             ki/...')
#23 {main}

Related Objects

Event Timeline

Reception123 triaged this task as High priority.Thu, Apr 8, 07:24
Reception123 created this task.
Herald added a subscriber: RhinosF1. · View Herald TranscriptThu, Apr 8, 07:24
John edited projects, added Infrastructure (SRE); removed Redis-JobRunner.Thu, Apr 8, 10:09

Redis software not the jobqueue software as this is manually ran, not a job

John added a comment.Thu, Apr 8, 10:17

Because of our monitoring, we’re doing fairly intensive Lua scripts on almost a 100k keys, this can take up to 2 seconds to run. We have set our connectTimeout in Redis has being 2s (https://github.com/miraheze/mw-config/blob/master/GlobalCache.php#L48).

Things we can do:

  • increase timeout
  • reduce number of keys being stored in Redis (already tasks)
  • reduce number of persistent client connections
John moved this task from Incoming to Short Term on the Infrastructure (SRE) board.Thu, Apr 8, 11:21
John closed this task as Resolved.Thu, Apr 8, 11:27
John claimed this task.

T7107

Reception123 reopened this task as Open.Fri, Apr 9, 09:34

@John I've run into the error again I'm afraid (though this time the dump has gone on for way longer, but eventually it happens)

15042 (5.33 pages/sec 11.13 revs/sec)
JobQueueError from line 778 of /srv/mediawiki/w/includes/jobqueue/JobQueueRedis.php: Redis server error: socket error on read socket

#0 /srv/mediawiki/w/includes/jobqueue/JobQueueRedis.php(240): JobQueueRedis->handleErrorAndMakeException(Object(RedisConnRef), Object(RedisException))
#1 /srv/mediawiki/w/includes/jobqueue/JobQueue.php(374): JobQueueRedis->doBatchPush(Array, 0)
#2 /srv/mediawiki/w/includes/jobqueue/JobQueue.php(344): JobQueue->batchPush(Array, 0)
#3 /srv/mediawiki/w/includes/jobqueue/JobQueueGroup.php(171): JobQueue->push(Array)
#4 /srv/mediawiki/w/includes/jobqueue/JobQueueGroup.php(210): JobQueueGroup->push(Array)
#5 /srv/mediawiki/w/includes/Title.php(4225): JobQueueGroup->lazyPush(Array)
#6 /srv/mediawiki/w/includes/page/WikiPage.php(3596): Title->touchLinks()
#7 /srv/mediawiki/w/includes/Storage/DerivedPageDataUpdater.php(1585): WikiPage::onArticleCreate(Object(Title))
#8 /srv/mediawiki/w/includes/page/WikiPage.php(2172): MediaWiki\Storage\DerivedPageDataUpdater->doUpdates()
#9 /srv/mediawiki/w/includes/import/ImportableOldRevisionImporter.php(218): WikiPage->doEditUpdates(Object(MediaWiki\Revision\RevisionStoreRecord), Object(User), Array)
#10 /srv/mediawiki/w/includes/import/WikiRevision.php(669): ImportableOldRevisionImporter->import(Object(WikiRevision))
#11 /srv/mediawiki/w/includes/import/WikiImporter.php(371): WikiRevision->importOldRevision()
#12 /srv/mediawiki/w/maintenance/importDump.php(201): WikiImporter->importRevision(Object(WikiRevision))
#13 /srv/mediawiki/w/includes/import/WikiImporter.php(507): BackupReader->handleRevision(Object(WikiRevision), Object(WikiImporter))
#14 /srv/mediawiki/w/includes/import/WikiImporter.php(992): WikiImporter->revisionCallback(Object(WikiRevision))
#15 /srv/mediawiki/w/includes/import/WikiImporter.php(857): WikiImporter->processRevision(Array, Array)
#16 /srv/mediawiki/w/includes/import/WikiImporter.php(795): WikiImporter->handleRevision(Array)
#17 /srv/mediawiki/w/includes/import/WikiImporter.php(612): WikiImporter->handlePage()
#18 /srv/mediawiki/w/maintenance/importDump.php(353): WikiImporter->doImport()
#19 /srv/mediawiki/w/maintenance/importDump.php(286): BackupReader->importFromHandle(Resource id #1000)
#20 /srv/mediawiki/w/maintenance/importDump.php(130): BackupReader->importFromFile('/home/reception...')
#21 /srv/mediawiki/w/maintenance/doMaintenance.php(107): BackupReader->execute()
#22 /srv/mediawiki/w/maintenance/importDump.php(358): require_once('/srv/mediawiki/...')
#23 {main}
John closed this task as Resolved.Fri, Apr 9, 10:22

Changes never got deployed on the server, this has been fixed now.