Shows an error.
[db1a142711000db3e921bfd1] 2021-10-17 09:37:21: Fatal exception of type "Error"
Description
Description
Related Objects
Related Objects
Event Timeline
Comment Actions
Call to undefined method HistoryBlobStub::uncompress()
from /srv/mediawiki/w/includes/historyblob/HistoryBlobStub.php(129)
#0 /srv/mediawiki/w/includes/Storage/SqlBlobStore.php(608): HistoryBlobStub->getText()
#1 /srv/mediawiki/w/includes/Storage/SqlBlobStore.php(520): MediaWiki\Storage\SqlBlobStore->decompressData(string, array)
#2 /srv/mediawiki/w/includes/Storage/SqlBlobStore.php(430): MediaWiki\Storage\SqlBlobStore->expandBlob(string, array, string)
#3 /srv/mediawiki/w/includes/Storage/SqlBlobStore.php(286): MediaWiki\Storage\SqlBlobStore->fetchBlobs(array, integer)
#4 /srv/mediawiki/w/includes/libs/objectcache/wancache/WANObjectCache.php(1707): MediaWiki\Storage\SqlBlobStore->MediaWiki\Storage\{closure}(boolean, integer, array, NULL, array)
#5 /srv/mediawiki/w/includes/libs/objectcache/wancache/WANObjectCache.php(1539): WANObjectCache->fetchOrRegenerate(string, integer, Closure, array, array)
#6 /srv/mediawiki/w/includes/Storage/SqlBlobStore.php(291): WANObjectCache->getWithSetCallback(string, integer, Closure, array)
#7 /srv/mediawiki/w/includes/Revision/RevisionStore.php(1191): MediaWiki\Storage\SqlBlobStore->getBlob(string, integer)
#8 /srv/mediawiki/w/includes/Revision/RevisionStore.php(1463): MediaWiki\Revision\RevisionStore->loadSlotContent(MediaWiki\Revision\SlotRecord, NULL, NULL, NULL, integer)
#9 [internal function]: MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}(MediaWiki\Revision\SlotRecord)
#10 /srv/mediawiki/w/includes/Revision/SlotRecord.php(324): call_user_func(Closure, MediaWiki\Revision\SlotRecord)
#11 /srv/mediawiki/w/includes/Revision/RevisionRecord.php(164): MediaWiki\Revision\SlotRecord->getContent()
#12 /srv/mediawiki/w/includes/parser/Parser.php(3697): MediaWiki\Revision\RevisionRecord->getContent(string)
#13 /srv/mediawiki/w/includes/parser/Parser.php(3547): Parser->statelessFetchTemplate(Title, Parser)
#14 /srv/mediawiki/w/extensions/Scribunto/includes/common/ScribuntoEngineBase.php(167): Parser->fetchTemplateAndTitle(Title)
#15 /srv/mediawiki/w/extensions/Scribunto/includes/engines/LuaCommon/LuaEngine.php(571): ScribuntoEngineBase->fetchModuleFromParser(Title)
#16 /srv/mediawiki/w/extensions/Scribunto/includes/engines/LuaSandbox/LuaSandboxCallback.php(26): Scribunto_LuaEngine->loadPackage(string)
#17 [internal function]: Scribunto_LuaSandboxCallback->__call(string, array)
#18 /srv/mediawiki/w/extensions/Scribunto/includes/engines/LuaSandbox/LuaSandboxInterpreter.php(113): LuaSandboxFunction->call(LuaSandboxFunction)
#19 /srv/mediawiki/w/extensions/Scribunto/includes/engines/LuaCommon/LuaEngine.php(295): Scribunto_LuaSandboxInterpreter->callFunction(LuaSandboxFunction, LuaSandboxFunction)
#20 /srv/mediawiki/w/extensions/Scribunto/includes/engines/LuaCommon/LuaModule.php(68): Scribunto_LuaEngine->executeFunctionChunk(LuaSandboxFunction, PPTemplateFrame_Hash)
#21 /srv/mediawiki/w/extensions/Scribunto/includes/common/Hooks.php(140): Scribunto_LuaModule->invoke(string, PPTemplateFrame_Hash)
#22 /srv/mediawiki/w/includes/parser/Parser.php(3356): ScribuntoHooks::invokeHook(Parser, PPTemplateFrame_Hash, array)
#23 /srv/mediawiki/w/includes/parser/Parser.php(3041): Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
#24 /srv/mediawiki/w/includes/parser/PPFrame_Hash.php(273): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#25 /srv/mediawiki/w/includes/parser/Parser.php(3230): PPFrame_Hash->expand(PPNode_Hash_Tree)
#26 /srv/mediawiki/w/includes/parser/PPFrame_Hash.php(273): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#27 /srv/mediawiki/w/includes/parser/Parser.php(3230): PPFrame_Hash->expand(PPNode_Hash_Tree)
#28 /srv/mediawiki/w/includes/parser/PPFrame_Hash.php(273): Parser->braceSubstitution(array, PPFrame_Hash)
#29 /srv/mediawiki/w/includes/parser/Parser.php(2879): PPFrame_Hash->expand(PPNode_Hash_Tree, integer)
#30 /srv/mediawiki/w/includes/parser/Parser.php(1549): Parser->replaceVariables(string)
#31 /srv/mediawiki/w/includes/parser/Parser.php(639): Parser->internalParse(string)
#32 /srv/mediawiki/w/includes/OutputPage.php(2115): Parser->parse(string, Title, ParserOptions, boolean, boolean, NULL)
#33 /srv/mediawiki/w/includes/OutputPage.php(1864): OutputPage->parseInternal(string, Title, boolean, boolean)
#34 /srv/mediawiki/w/includes/OutputPage.php(1795): OutputPage->addWikiTextTitleInternal(string, Title, boolean, boolean)
#35 /srv/mediawiki/w/includes/OutputPage.php(4128): OutputPage->addWikiTextAsInterface(string)
#36 /srv/mediawiki/w/includes/specials/SpecialContributions.php(368): OutputPage->wrapWikiMsg(string, array)
#37 /srv/mediawiki/w/includes/specialpage/SpecialPage.php(646): SpecialContributions->execute(string)
#38 /srv/mediawiki/w/includes/specialpage/SpecialPageFactory.php(1386): SpecialPage->run(string)
#39 /srv/mediawiki/w/includes/MediaWiki.php(309): MediaWiki\SpecialPage\SpecialPageFactory->executePath(Title, RequestContext)
#40 /srv/mediawiki/w/includes/MediaWiki.php(913): MediaWiki->performRequest()
#41 /srv/mediawiki/w/includes/MediaWiki.php(546): MediaWiki->main()
#42 /srv/mediawiki/w/index.php(53): MediaWiki->run()
#43 /srv/mediawiki/w/index.php(46): wfIndexMain()Comment Actions
MariaDB [famepediawiki]> SELECT * FROM text WHERE old_flags = 'object,utf-8' AND old_text NOT LIKE '%ConcatenatedGzipHistoryBlob%' LIMIT 3;
+--------+---------------------------------------------------------------------------------------------------------------------------------------+--------------+
| old_id | old_text | old_flags |
+--------+---------------------------------------------------------------------------------------------------------------------------------------+--------------+
| 50 | O:15:"HistoryBlobStub":3:{s:9:" * mOldId";s:5:"24648";s:8:" * mHash";s:32:"4fd6bef120de43768abbbd24da40051f";s:7:" * mRef";s:2:"50";} | object,utf-8 |
| 72 | O:15:"HistoryBlobStub":3:{s:9:" * mOldId";s:5:"24167";s:8:" * mHash";s:32:"17cbe2208cfbd26d5a5a052188c9b198";s:7:" * mRef";s:2:"72";} | object,utf-8 |
| 99 | O:15:"HistoryBlobStub":3:{s:9:" * mOldId";s:5:"24486";s:8:" * mHash";s:32:"72ca804c7ef14b83ff649bc3e013a9c5";s:7:" * mRef";s:2:"99";} | object,utf-8 |
+--------+---------------------------------------------------------------------------------------------------------------------------------------+--------------+If I had to guess, these 3 revision entries would be the problems.
Comment Actions
https://phabricator.wikimedia.org/T39882#410958
Seems to have happened sometimes when running compressOld, in MediaWiki, since 1.18/1.19 at least.
Comment Actions
I have a rough understanding of the potential problem, I'm going to try and create a script to hopefully identify actual troublesome revisions.
Comment Actions
universalomega@mwtask1:~$ sudo -u www-data php /srv/mediawiki/w/maintainance/storage/checkStorage.php --wiki=famepediawiki Checking... Error: Incorrect object header for concat bulk row 99 in old_id 1465, revisions 2508, 2509 Error: Incorrect object header for concat bulk row 201 in old_id 1452, revisions 2483, 2484 Error: Incorrect object header for concat bulk row 202 in old_id 1453, revisions 2485, 2486 Error: Incorrect object header for concat bulk row 209 in old_id 1455, revisions 2489, 2490 Error: Incorrect object header for concat bulk row 201 in old_id 19407, rev_id 26860 Error: Incorrect object header for concat bulk row 16785 in old_id 19464, rev_id 26976 Error: Incorrect object header for concat bulk row 209 in old_id 19477, revisions 27002, 27003 Error: Incorrect object header for concat bulk row 209 in old_id 22495, rev_id 30245 Error: Incorrect object header for concat bulk row 72 in text rows 24014, 24015, 24016, 24017, 24018, 24019, 24020, 24021, 24022, 24023, 24024, 24025, 24026, 24027, 24028, revisions 31856, 31857, 31858, 31859, 31860, 31861, 31862, 31863, 31864, 31865, 31866, 31867, 31868, 31869, 31870, 31871 Error: Incorrect object header for concat bulk row 50 in text rows 24634, 24635, 24636, 24637, 24638, 24639, 24640, 24641, 24642, 24643, 24644, 24645, 24646, 24647, revisions 32545, 32546, 32547, 32548, 32549, 32550, 32551, 32552, 32553, 32554, 32555, 32556, 32557, 32558, 32559, 32560 Error: Incorrect object header for concat bulk row 99 in text rows 1465, 24472, 24473, 24474, 24475, 24476, 24477, 24478, 24479, 24480, 24481, 24482, 24483, 24484, 24485, revisions 32363, 32344, 32345, 32346, 32347, 32348, 32349, 32350, 32351, 32352, 32353, 32354, 32355, 32356, 32357, 32358 Error: Incorrect object header for concat bulk row 201 in text rows 25975, 25976, 25977, 25978, 25979, 25980, 25981, 25982, 25983, 25984, 25985, 25986, 25987, revisions 33955, 33956, 33957, 33958, 33959, 33960, 33961, 33962, 33963, 33964, 33965, 33966, 33967 Error: Incorrect object header for concat bulk row 19470 in text rows 25766, 25767, 25768, 25769, 25770, 25771, 25772, 25773, 25774, 25775, 25776, 25777, 25778, 25779, 25780, 25781, 25782, revisions 33719, 33720, 33721, 33722, 33723, 33724, 33725, 33726, 33727, 33728, 33729, 33730, 33731, 33732, 33733, 33734, 33735, 33736 Error: Incorrect object header for concat bulk row 202 in text rows 1453, 26033, 26034, 26035, 26036, 26037, 26038, 26039, 26040, 26041, 26042, 26043, 26044, 26045, revisions 34045, 34019, 34020, 34021, 34022, 34023, 34024, 34025, 34026, 34027, 34028, 34029, 34030, 34031, 34032 Error: Incorrect object header for concat bulk row 209 in text rows 26105, 26106, 26107, 26108, 26109, 26110, 26111, 26112, 26113, 26114, 26115, 26116, revisions 34107, 34108, 34109, 34110, 34111, 34112, 34113, 34114, 34115, 34116, 34117, 34118 Error: Incorrect object header for concat bulk row 27981 in text rows 28589, 28590, 28591, 28592, 28593, 28594, 28595, 28596, 28597, 28598, 28599, 28600, 28601, 28602, 28603, 28604, 28605, 28606, revisions 36917, 36918, 36919, 36920, 36921, 36922, 36923, 36924, 36925, 36926, 36927, 36928, 36929, 36930, 36931, 36932, 36933, 36934 Error: Incorrect object header for concat bulk row 16785 in text rows 29343, 29345, revisions 37718, 37720 Errors: Damaged text, need to be restored from a backup: 2508,2509,2483,2484,2485,2486,2489,2490,26860,26976,27002,27003,30245,31856,31857,31858,31859,31860,31861,31862,31863,31864,31865,31866,31867,31868,31869,31870,31871,32545,32546,32547,32548,32549,32550,32551,32552,32553,32554,32555,32556,32557,32558,32559,32560,32363,32344,32345,32346,32347,32348,32349,32350,32351,32352,32353,32354,32355,32356,32357,32358,33955,33956,33957,33958,33959,33960,33961,33962,33963,33964,33965,33966,33967,33719,33720,33721,33722,33723,33724,33725,33726,33727,33728,33729,33730,33731,33732,33733,33734,33735,33736,34045,34019,34020,34021,34022,34023,34024,34025,34026,34027,34028,34029,34030,34031,34032,34107,34108,34109,34110,34111,34112,34113,34114,34115,34116,34117,34118,36917,36918,36919,36920,36921,36922,36923,36924,36925,36926,36927,36928,36929,36930,36931,36932,36933,36934,37718,37720 Flag statistics: object,utf-8 14657 33.44% utf-8 29174 66.56% Local object statistics: concatenatedgziphistoryblob 3032 20.69% historyblobstub 11625 79.31%
Comment Actions
checkStorage.php can resolve this with an XML dump. So hopefully, this will only require someone from the MW team to loop through all XML dumps with the checkStorage script
Comment Actions
https://gerrit.wikimedia.org/r/731819 will fix the checkStorage script. I had to make that change locally on mwtask1 last time to run it, which we can do again if necessary.
Comment Actions
Here's a script that can loop over backups, running --fix on checkStorage.php:
#! /bin/bash usage(){ echo "Usage: sh fixStorage.sh /path/to/backups/directory" exit 1 } if [ ! -d $1 ]; then echo "directory does not exist!" usage exit 1 fi for backup in $(ls $1); do wiki=`echo $backup | cut -d"." -f1` echo "Running checkStorage.php --fix $backup for $wiki" sudo -u www-data /usr/bin/php /srv/mediawiki/w/maintenance/storage/checkStorage.php --wiki=$wiki --fix $backup done
https://github.com/miraheze/mediawiki/pull/3802 fixes checkStorage.php
Comment Actions
I was able to finish compiling mwdumper and got that correctly working, after a few modifications to it (https://gerrit.wikimedia.org/r/732699), but now I have one final fatal in checkStorage.php, that I'm unsure with:
[aa5196c5f29f9d4d8282d8e1] [no req] MediaWiki\Revision\RevisionAccessException: No such slot: 3
Backtrace:
from /srv/mediawiki/w/includes/Revision/RevisionSlots.php(114)
#0 /srv/mediawiki/w/includes/Revision/RevisionSlots.php(95): MediaWiki\Revision\RevisionSlots->getSlot(integer)
#1 /srv/mediawiki/w/includes/import/WikiRevision.php(495): MediaWiki\Revision\RevisionSlots->getContent(integer)
#2 /srv/mediawiki/w/maintenance/storage/checkStorage.php(518): WikiRevision->getContent(integer)
#3 /srv/mediawiki/w/includes/import/WikiImporter.php(571): CheckStorage->importRevision(WikiRevision, WikiImporter)
#4 /srv/mediawiki/w/includes/import/WikiImporter.php(1059): WikiImporter->revisionCallback(WikiRevision)
#5 /srv/mediawiki/w/includes/import/WikiImporter.php(926): WikiImporter->processRevision(array, array)
#6 /srv/mediawiki/w/includes/import/WikiImporter.php(861): WikiImporter->handleRevision(array)
#7 /srv/mediawiki/w/includes/import/WikiImporter.php(678): WikiImporter->handlePage()
#8 /srv/mediawiki/w/maintenance/storage/checkStorage.php(513): WikiImporter->doImport()
#9 /srv/mediawiki/w/maintenance/storage/checkStorage.php(371): CheckStorage->restoreText(array, string)
#10 /srv/mediawiki/w/maintenance/storage/checkStorage.php(35): CheckStorage->check(boolean, string)
#11 {main}Comment Actions
https://gerrit.wikimedia.org/r/732342 is another patch that was necessary to do on mwtask1 today. Thought it'd be a good idea to document that here.
Comment Actions
I have fixed famepediawiki, testwiki, and test3wiki now. I can probably take over and do this on all wikis now, if that is wanted. But it would be easier for Reception123 to, as I would have to copy all XML files to my home directory to extract them. For now, only famepediawiki seemed to be blocking all edits, and at least that one is fixed.
Here is the script again, which I made specifically for this. It should loop over all files in a specified directory, running checkStorage.php --fix with them:
#! /bin/bash usage(){ echo "Usage: bash fixStorage.sh /path/to/backups/directory" exit 1 } if [ ! -d $1 ]; then echo "Directory does not exist!" usage exit 1 fi /usr/local/bin/logsalmsg "Started running checkStorage.php --fix on all wikis" for backup in $(ls $1); do wiki=`echo $backup | cut -d"." -f1` sudo -u www-data gunzip $1/$backup sleep 2 echo "Running checkStorage.php --fix $wiki.xml for $wiki" sudo -u www-data /usr/bin/php /srv/mediawiki/w/maintenance/storage/checkStorage.php --wiki=$wiki --fix $1/$wiki.xml sleep 2 sudo -u www-data rm -f $1/$wiki.xml sleep 2 done /usr/local/bin/logsalmsg "Finished running checkStorage.php --fix on all wikis"
Comment Actions
@Universal_Omega will be working tomorrow (his time) to make the script also unzip and delete files upon running the script as there's not enough space for unzipped files