Page MenuHomeMiraheze

populateWikibaseSitesTable.php fails with Error: 1406 Data too long for column 'site_global_key' at row 1 (db11.miraheze.org)
Closed, ResolvedPublic

Description

rhinos@jobrunner2:~$ sudo -u www-data php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/populateWikibaseSitesTable.php --wiki=datatrekwiki
Wikimedia\Rdbms\DBQueryError from line 1603 of /srv/mediawiki/w/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 
Function: DBSiteStore::saveSites
Error: 1406 Data too long for column 'site_global_key' at row 1 (db11.miraheze.org)

#0 /srv/mediawiki/w/includes/libs/rdbms/database/Database.php(1574): Wikimedia\Rdbms\Database->getQueryExceptionAndLog('Data too long f...', 1406, 'INSERT  INTO `s...', 'DBSiteStore::sa...')
#1 /srv/mediawiki/w/includes/libs/rdbms/database/Database.php(1152): Wikimedia\Rdbms\Database->reportQueryError('Data too long f...', 1406, 'INSERT  INTO `s...', 'DBSiteStore::sa...', false)
#2 /srv/mediawiki/w/includes/libs/rdbms/database/Database.php(2140): Wikimedia\Rdbms\Database->query('INSERT  INTO `s...', 'DBSiteStore::sa...')
#3 /srv/mediawiki/w/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->insert('`sites`', Array, 'DBSiteStore::sa...')
#4 /srv/mediawiki/w/includes/libs/rdbms/database/DBConnRef.php(371): Wikimedia\Rdbms\DBConnRef->__call('insert', Array)
#5 /srv/mediawiki/w/includes/site/DBSiteStore.php(217): Wikimedia\Rdbms\DBConnRef->insert('sites', Array, 'DBSiteStore::sa...')
#6 /srv/mediawiki/w/includes/site/CachingSiteStore.php(162): DBSiteStore->saveSites(Array)
#7 /srv/mediawiki/w/extensions/Wikibase/lib/includes/Sites/SitesBuilder.php(62): CachingSiteStore->saveSites(Array)
#8 /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/populateWikibaseSitesTable.php(56): Wikibase\Lib\Sites\SitesBuilder->buildStore(Array, 'wikipedia', 'datatrekwiki')
#9 /srv/mediawiki/w/maintenance/doMaintenance.php(99): PopulateWikibaseSitesTable->execute()
#10 /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/populateWikibaseSitesTable.php(158): require_once('/srv/mediawiki/...')
#11 {main}
[222af3ff62544b0a2b96b5a9] [no req]   Wikimedia\Rdbms\DBTransactionError from line 1438 of /srv/mediawiki/w/includes/libs/rdbms/database/Database.php: Explicit transaction still active. A caller may have caught an error. Open transactions: DBSiteStore::saveSites
Backtrace:
#0 /srv/mediawiki/w/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1602): Wikimedia\Rdbms\Database->assertNoOpenTransactions()
#1 /srv/mediawiki/w/includes/libs/rdbms/loadbalancer/LoadBalancer.php(2109): Wikimedia\Rdbms\LoadBalancer->Wikimedia\Rdbms\{closure}(Wikimedia\Rdbms\DatabaseMysqli)
#2 /srv/mediawiki/w/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1621): Wikimedia\Rdbms\LoadBalancer->forEachOpenMasterConnection(Closure)
#3 /srv/mediawiki/w/includes/libs/rdbms/lbfactory/LBFactory.php(208): Wikimedia\Rdbms\LoadBalancer->approveMasterChanges(array, string, integer)
#4 /srv/mediawiki/w/includes/libs/rdbms/lbfactory/LBFactoryMulti.php(241): Wikimedia\Rdbms\LBFactory->Wikimedia\Rdbms\{closure}(Wikimedia\Rdbms\LoadBalancer, string, array)
#5 /srv/mediawiki/w/includes/libs/rdbms/lbfactory/LBFactory.php(210): Wikimedia\Rdbms\LBFactoryMulti->forEachLB(Closure, array)
#6 /srv/mediawiki/w/includes/libs/rdbms/lbfactory/LBFactory.php(269): Wikimedia\Rdbms\LBFactory->forEachLBCallMethod(string, array)
#7 /srv/mediawiki/w/maintenance/doMaintenance.php(127): Wikimedia\Rdbms\LBFactory->commitMasterChanges(string)
#8 /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/populateWikibaseSitesTable.php(158): require_once(string)
#9 {main}
rhinos@jobrunner2:~$

Event Timeline

RhinosF1 triaged this task as Normal priority.Aug 14 2020, 08:34
RhinosF1 created this task.

@Paladox: You've dealt with similar issues before, would you mind proposing whatever change needs to be done and/or applying it to our tables?

@Paladox: You've dealt with similar issues before, would you mind proposing whatever change needs to be done and/or applying it to our tables?

The easiest and more robust way to add sites to the sites table is using an XML file and importSites.php script as described here https://www.mediawiki.org/wiki/Manual:Sites_table#Adding_a_new_site .
I will provide a sample in T6041 for you to try.

The easiest and more robust way to add sites to the sites table is using an XML file and importSites.php script as described here https://www.mediawiki.org/wiki/Manual:Sites_table#Adding_a_new_site .
I will provide a sample in T6041 for you to try.

That won't help, the script that we use to populate the sites table of wikis is failing because the column isn't big enough for the data it needs to hold. We're going to have that issue whatever way we try and import the table.

That won't help, the script that we use to populate the sites table of wikis is failing because the column isn't big enough for the data it needs to hold. We're going to have that issue whatever way we try and import the table.

Well, this is very strange. I see the error is in field site_global_key which is defined as varbinary(32): while I cannot be sure of your naming convention, I assume you are identifying the wiki with an ID like datatrekwiki or datatrek so it should easily fit.
May I ask how you generate the site_global_key?

Thanks

Well, this is very strange. I see the error is in field site_global_key which is defined as varbinary(32): while I cannot be sure of your naming convention, I assume you are identifying the wiki with an ID like datatrekwiki or datatrek so it should easily fit.

The insert it's failing on is for a wiki with a db name of 38 characters which is over 32. We seem to allow db names up to 64 charecters.

@Paladox: could you run the alter on the tables now rather than waiting for upstream to merge?

That'll get this unbroke.

RhinosF1 assigned this task to Paladox.
RhinosF1 added subscribers: Hispano76, Unknown Object (User).

Patch is upstream to make this work for all wikis and disabled for wikis with longer db names than this supports.