section.execute-GitInfo->getHeadCommitDate is very slow
Closed, ResolvedPublic
Actions

Assigned To

Unknown Object (User)

Authored By

	BrandonWM
	Nov 19 2020, 20:10

Description

See comments, 92% of the time to load siteinfo Api with extensions is taken doing this. It should be cached.

https://phabricator.wikimedia.org/T131003 ?

Related Objects

Mentioned In: R9:9af6e36c5c78: T6471: Set wgGitInfoCacheDirectory
T6431: cp9 extremely latent for a number of users
Mentioned Here: P421 curl -H 'X-Miraheze-Debug: test3.miraheze.org' 'https://meta.miraheze.org/w/api.php?action=query&meta=siteinfo&siprop=general|extensions&forceprofile=1' | grep '1 - main()' -A 450 | less

Event Timeline

BrandonWM triaged this task as Normal priority.Nov 19 2020, 20:10

BrandonWM created this task.

Herald added subscribers: Unknown Object (User), RhinosF1, Reception123. · View Herald TranscriptNov 19 2020, 20:10

BrandonWM mentioned this in T6431: cp9 extremely latent for a number of users.Nov 19 2020, 20:11

Wiki-Bot#2998 is not operated or maintained by Miraheze, and as such there is nothing we can do about it unfortunately

@Reception123 The dev is saying that it's Miraheze, not the bot that's the issue.

drop the stick

@BlackWidowMovie0000Editor Who is the dev, and how would it be our fault?

Zppix closed this task as Declined.Nov 19 2020, 20:17

In T6471#126889, @Reception123 wrote:

@BlackWidowMovie0000Editor Who is the dev, and how would it be our fault?

MarkusRost. He's on the support server. I can get the invite if you want.

BrandonWM updated the task description. (Show Details)Nov 19 2020, 20:22

I literally just went over this. And we're discussing in IRC right now, with the dev even saying it's an issue on Miraheze's end.

Reception123 removed Reception123 as the assignee of this task.Nov 20 2020, 12:18

Reception123 edited projects, added Site Reliability Engineering; removed Upstream.

BrandonWM moved this task from Radar to Bugs on the Site Reliability Engineering board.Nov 20 2020, 16:54

Even if this is somehow related to Miraheze, I do not see how we would really fix it. In addition, this is not a service relating to Miraheze what-so-ever, and has no impact on our own services, as such once again, closing this task.

I've investigated the problem: the reason for timeouts when using Wiki-Bot is because it asks for the whole list of extensions. Miraheze wikis respond to that specific request (https://meta.miraheze.org/w/api.php?action=query&meta=siteinfo&siprop=general|extensions with |extensions) too slowly, slower than most MediaWiki sites, and that is why the bot has problems. Asking simply for general is fine. If I had to guess, it's probably something to do with ManageWiki. (Leaving this here for people searching this in the future.)

P421 curl -H 'X-Miraheze-Debug: test3.miraheze.org' 'https://meta.miraheze.org/w/api.php?action=query&meta=siteinfo&siprop=general|ext 2 98.68% 9101.326 3 98.65% 9099.154 4 98.65% 9099.153 5 98.63% 9097.408 6 95.63% 8820.098 7 93.74% 8646.296 8 93.74% 8645.895 9 93.55% 8628.697 10 93.44% 8617.873 11 92.71% 8550.627 12 92.17% 8501.164 13 5.83% 537.685 14 5.77% 532.221 15 4.79% 441.908 16 4.77% 440.163 17 3.11% 287.291 18 3.11% 287.188 19 3.09% 284.896 20 3.09% 284.872 21 3.09% 284.648 22 3.01% 277.703 23 3.00% 276.926 24 1.60% 147.541 25 1.34% 123.619 26 1.28% 117.606 27 1.27% 117.559 28 1.27% 117.542 29 1.27% 116.785 30 1.27% 116.716 31 1.27% 116.712 32 1.26% 116.632 33 1.26% 116.441 34 1.25% 115.337 35 1.25% 115.307 36 1.25% 115.250 37 1.25% 115.147 38 1.25% 115.137 39 1.25% 115.100 40 1.25% 115.098 41 1.25% 115.080 42 1.25% 115.070 43 1.25% 115.068 44 1.25% 115.064 45 1.25% 115.036 46 1.25% 114.983 47 1.24% 114.726 48 1.13% 104.494 49 1.13% 104.325 50 1.13% 104.009 51 1.13% 103.811 52 1.10% 101.630 53 1.10% 101.308 54 1.09% 100.795 55 1.09% 100.791 56 1.09% 100.790 57 1.08% 99.642 58 1.08% 99.530 59 1.03% 94.716 60 0.83% 76.152 61 0.79% 72.922 62 0.76% 70.518 63 0.73% 67.631 64 0.56% 51.573 65 0.51% 46.591 66 0.51% 46.582 67 0.50% 46.563 68 0.38% 34.785 69 0.34% 31.061 70 0.28% 25.394 71 0.27% 25.222 72 0.26% 23.678 73 0.23% 21.267 74 0.21% 19.774 75 0.20% 18.101 76 0.19% 17.224 77 0.19% 17.190 78 0.18% 16.833 79 0.18% 16.369 80 0.15% 14.212 81 0.15% 13.959 82 0.14% 13.082 83 0.14% 12.980 84 0.14% 12.736 85 0.14% 12.588 86 0.13% 12.406 87 0.13% 12.251 88 0.13% 11.607 89 0.12% 11.231 90 0.11% 10.510 91 0.11% 9.993 92 0.11% 9.874 93 0.10% 9.645 94 0.10% 9.632 95 0.10% 9.448 96 0.10% 9.235 97 0.10% 9.219 98 0.10% 8.905 99 0.09% 8.307 100 0.09% 8.275 101 0.09% 7.967 102 0.09% 7.929 103 0.08% 7.801 104 0.08% 7.438 105 0.08% 7.363 106 0.08% 7.334 107 0.08% 7.329 108 0.08% 7.168 109 0.08% 7.150 110 0.08% 7.087 111 0.08% 7.073 112 0.08% 7.061 113 0.08% 6.992 114 0.08% 6.955 115 0.07% 6.878 116 0.07% 6.829 117 0.07% 6.811 118 0.07% 6.726 119 0.07% 6.660 120 0.07% 6.598 121 0.07% 6.430 122 0.07% 6.379 123 0.07% 6.002 124 0.06% 5.991 125 0.06% 5.984 126 0.06% 5.965 127 0.06% 5.960 128 0.06% 5.917 129 0.06% 5.898 130 0.06% 5.756 131 0.06% 5.651 132 0.06% 5.637 133 0.06% 5.534 134 0.06% 5.356 135 0.06% 5.328 136 0.06% 5.231 137 0.06% 5.127 138 0.06% 5.121 139 0.05% 5.040 140 0.05% 4.951 141 0.05% 4.947 142 0.05% 4.906 143 0.05% 4.882 144 0.05% 4.878 145 0.05% 4.790 146 0.05% 4.738 147 0.05% 4.725 148 0.05% 4.713 149 0.05% 4.702 150 0.05% 4.634 151 0.05% 4.632 152 0.05% 4.622 153 0.05% 4.613 154 0.05% 4.574 155 0.05% 4.531 156 0.05% 4.500 157 0.05% 4.483 158 0.05% 4.420 159 0.05% 4.375 160 0.05% 4.248 161 0.05% 4.225 162 0.05% 4.199 163 0.04% 4.137 164 0.04% 4.080 165 0.04% 4.078 166 0.04% 4.040 167 0.04% 4.014 168 0.04% 3.911 169 0.04% 3.741 170 0.04% 3.695 171 0.04% 3.657 172 0.04% 3.629 173 0.04% 3.608 174 0.04% 3.573 175 0.04% 3.555 176 0.04% 3.554 177 0.04% 3.455 178 0.04% 3.449 179 0.04% 3.432 180 0.04% 3.232 181 0.03% 3.210 182 0.03% 3.184 183 0.03% 3.051 184 0.03% 3.044 185 0.03% 3.042 186 0.03% 3.023 187 0.03% 2.991 188 0.03% 2.957 189 0.03% 2.956 190 0.03% 2.929 191 0.03% 2.913 192 0.03% 2.901 193 0.03% 2.871 194 0.03% 2.857 195 0.03% 2.809 196 0.03% 2.730 197 0.03% 2.729 198 0.03% 2.651 199 0.03% 2.640 200 0.03% 2.620 201 0.03% 2.602 202 0.03% 2.579 203 0.03% 2.573 204 0.03% 2.563 205 0.03% 2.515 206 0.03% 2.509 207 0.03% 2.497 208 0.03% 2.493 209 0.03% 2.491 210 0.03% 2.440 211 0.03% 2.434 212 0.03% 2.423 213 0.03% 2.373 214 0.03% 2.366 215 0.03% 2.352 216 0.03% 2.352 217 0.03% 2.322 218 0.02% 2.299 219 0.02% 2.261 220 0.02% 2.257 221 0.02% 2.250 222 0.02% 2.225 223 0.02% 2.204 224 0.02% 2.156 225 0.02% 2.062 226 0.02% 2.059 227 0.02% 2.045 228 0.02% 2.022 229 0.02% 2.008 230 0.02% 2.008 231 0.02% 2.000 232 0.02% 1.991 233 0.02% 1.978 234 0.02% 1.973 235 0.02% 1.965 236 0.02% 1.955 237 0.02% 1.953 238 0.02% 1.942 239 0.02% 1.939 240 0.02% 1.921 241 0.02% 1.920 242 0.02% 1.893 243 0.02% 1.892 244 0.02% 1.888 245 0.02% 1.877 246 0.02% 1.858 247 0.02% 1.812 248 0.02% 1.803 249 0.02% 1.787 250 0.02% 1.774 251 0.02% 1.772 252 0.02% 1.769 253 0.02% 1.769 254 0.02% 1.744 255 0.02% 1.735 256 0.02% 1.707 257 0.02% 1.690 258 0.02% 1.669 259 0.02% 1.669 260 0.02% 1.664 261 0.02% 1.663 262 0.02% 1.642 263 0.02% 1.641 264 0.02% 1.621 265 0.02% 1.606 266 0.02% 1.585 267 0.02% 1.582 268 0.02% 1.581 269 0.02% 1.580 270 0.02% 1.580 271 0.02% 1.560 272 0.02% 1.558 273 0.02% 1.548 274 0.02% 1.545 275 0.02% 1.539 276 0.02% 1.534 277 0.02% 1.514 278 0.02% 1.510 279 0.02% 1.507 280 0.02% 1.500 281 0.02% 1.498 282 0.02% 1.498 283 0.02% 1.497 284 0.02% 1.490 285 0.02% 1.485 286 0.02% 1.461 287 0.02% 1.461 288 0.02% 1.459 289 0.02% 1.458 290 0.02% 1.438 291 0.02% 1.410 292 0.02% 1.405 293 0.02% 1.399 294 0.02% 1.397 295 0.01% 1.371 296 0.01% 1.362 297 0.01% 1.362 298 0.01% 1.338 299 0.01% 1.320 300 0.01% 1.318 301 0.01% 1.310 302 0.01% 1.309 303 0.01% 1.308 304 0.01% 1.306 305 0.01% 1.305 306 0.01% 1.297 307 0.01% 1.282 308 0.01% 1.270 309 0.01% 1.268 310 0.01% 1.249 311 0.01% 1.239 312 0.01% 1.226 313 0.01% 1.213 314 0.01% 1.211 315 0.01% 1.198 316 0.01% 1.182 317 0.01% 1.165 318 0.01% 1.162 319 0.01% 1.160 320 0.01% 1.156 321 0.01% 1.151 322 0.01% 1.124 323 0.01% 1.118 324 0.01% 1.109 325 0.01% 1.104 326 0.01% 1.101 327 0.01% 1.092 328 0.01% 1.080 329 0.01% 1.064 330 0.01% 1.063 331 0.01% 1.056 332 0.01% 1.046 333 0.01% 1.045 334 0.01% 1.038 335 0.01% 1.036 336 0.01% 1.031 337 0.01% 1.028 338 0.01% 1.024 339 0.01% 1.021 340 0.01% 1.020 341 0.01% 1.020 342 0.01% 1.015 343 0.01% 1.014 344 0.01% 1.011 345 0.01% 1.010 346 0.01% 1.008 347 0.01% 1.008 348 0.01% 1.004 349 350-->

ensions&forceprofile=1' | grep '1 - main()' -A 450 | less

1	100.00% 9223.385 1 - main() 1 - wfApiMain 1 - ApiMain::execute 1 - ApiMain::executeActionWithErrorHandling 1 - ApiMain::executeAction 108 - MediaWiki\Shell\Command::execute 1 - ApiQuery::execute 1 - ApiQuerySiteinfo::execute 1 - ApiQuerySiteinfo::appendExtensions 326 - stream_select 107 - GitInfo::getHeadCommitDate 107 - section.execute-GitInfo->getHeadCommitDate 581 - MediaWiki\HookContainer\HookContainer::run 271 - MediaWiki\HookContainer\HookContainer::callLegacyHook 1 - ApiMain::printResult 1 - ApiFormatBase::closePrinter 8 - WANObjectCache::getWithSetCallback 7 - WANObjectCache::fetchOrRegenerate 1 - MediaWiki\Api\ApiHookRunner::onApiFormatHighlight 1 - SyntaxHighlight::onApiFormatHighlight 1 - SyntaxHighlight::highlight 1 - SyntaxHighlight::{closure} 1 - section.execute-SyntaxHighlight::{closure} 1 - OutputPage::output 108 - proc_open 2 - CookieWarning\Decisions::shouldShowCookieWarning 2 - CookieWarning\Decisions::isInConfiguredRegion 2 - CookieWarning\Decisions::getCountryCodeFromIP 2 - CookieWarning\HttpGeoLocation::locate 2 - Http::get 2 - Http::request 2 - MediaWiki\Http\HttpRequestFactory::request 2 - GuzzleHttpRequest::execute 2 - GuzzleHttp\Client::send 2 - GuzzleHttp\Client::sendAsync 2 - GuzzleHttp\Client::transfer 2 - GuzzleHttp\HandlerStack::__invoke 10 - GuzzleHttp\Middleware::GuzzleHttp\{closure} 2 - GuzzleHttp\RedirectMiddleware::__invoke 2 - GuzzleHttp\PrepareBodyMiddleware::__invoke 2 - GuzzleHttp\Middleware::GuzzleHttp\{closure}@1 2 - GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure} 2 - GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure}@1 2 - GuzzleHttp\Handler\CurlHandler::__invoke 1 - MediaWiki\HookContainer\HookRunner::onSetupAfterCache 1 - CreateWikiHooks::onSetupAfterCache 2 - curl_exec 18 - Wikimedia\Rdbms\LoadBalancer::getConnection 18 - Wikimedia\Rdbms\LoadBalancer::getServerConnection 13 - wfGetDB 13 - Wikimedia\Rdbms\LoadBalancer::getMaintenanceConnectionRef 1 - CreateWikiJson::__construct 2 - Wikimedia\Rdbms\LoadBalancer::reallyOpenConnection 2 - Wikimedia\Rdbms\Database::initConnection 2 - Wikimedia\Rdbms\Database::doInitConnection 2 - Wikimedia\Rdbms\DatabaseMysqlBase::open 2 - Wikimedia\Rdbms\DatabaseMysqli::mysqlConnect 2 - mysqli::real_connect 2 - Wikimedia\Rdbms\LoadBalancer::getForeignConnection 1 - MediaWiki\HookContainer\HookRunner::onBeforePageDisplay 1 - CookieWarning\Hooks::onBeforePageDisplay 1 - SkinTemplate::outputPage 1 - SkinMustache::generateHTML 1 - SkinMustache::getTemplateData 1 - Skin::afterContentHook 1 - MediaWiki\HookContainer\HookRunner::onSkinAfterContent 1 - CookieWarning\Hooks::onSkinAfterContent 391 - Monolog\Logger::addRecord 146 - Monolog\Logger::debug 112 - Monolog\Handler\WhatFailureGroupHandler::handle 112 - Monolog\Handler\AbstractProcessingHandler::handle 40 - Message::toString 410 - spl_autoload_call 112 - Monolog\Handler\SyslogUdpHandler::write 112 - Monolog\Handler\SyslogUdp\UdpSocket::write 112 - Monolog\Handler\SyslogUdp\UdpSocket::send 1 - ApiQuerySiteinfo::appendGeneralInfo 112 - socket_sendto 35 - Message::text 2653 - Wikimedia\Services\ServiceContainer::getService 1 - OutputPage::headElement 434 - AutoLoader::autoload 111 - GitInfo::__construct 23 - Wikimedia\Services\ServiceContainer::createService 25 - Wikimedia\Services\ServiceContainer::{closure} 1 - SiteConfiguration::extractAllGlobals 827 - SiteConfiguration::extractGlobalSetting 831 - SiteConfiguration::processSetting 118 - Title::getLinkURL 3 - OutputPage::getRlClient 109 - MediaWiki\ExtensionInfo::getLicenseFileNames 218 - scandir 119 - SpecialPage::getTitleFor 54 - Message::fetchMessage 123 - Title::getLocalURL 47 - MessageCache::get 107 - GitInfo::getHeadViewUrl 3284 - Composer\Autoload\ClassLoader::loadClass 47 - MessageCache::getMessageFromFallbackChain 47 - MessageCache::getMessageForLang 107 - GitInfo::getRemoteUrl 115 - wfTimestamp 117 - Wikimedia\Timestamp\ConvertibleTimestamp::convert 47 - MessageCache::getMsgFromNamespace 120 - SpecialPage::getTitleValueFor 1 - ApiQueryBase::getDB 1 - ApiBase::getDB 2 - Message::parseText 3 - Title::newMainPage 2 - MessageCache::parse 37 - Message::transformText 2 - Parser::parse 37 - MessageCache::transform 1 - ApiMain::reportUnusedParams 16 - Wikimedia\Rdbms\LoadBalancer::getLocalConnection 1 - ApiBase::addWarning 1 - ApiErrorFormatter::addWarning 220 - file_get_contents 47 - MessageCache::load 1 - ApiErrorFormatter_BackCompat::addWarningOrError 1 - Message::parseAsBlock 6173 - preg_match 2 - MediumSpecificBagOStuff::add 2 - MemcachedPhpBagOStuff::doAdd 3499 - Composer\Autoload\ClassLoader::findFile 2 - MemcachedClient::add 2 - MemcachedClient::_set 107 - parse_ini_file 32 - MemcachedClient::_fgets 32 - fgets 110 - GitInfo::getHeadSHA1 1 - WANObjectCache::set 34 - LCStoreCDB::get 118 - Wikimedia\Timestamp\ConvertibleTimestamp::__construct 3 - OutputPage::getRlClientContext 118 - Wikimedia\Timestamp\ConvertibleTimestamp::setTimestamp 6 - OutputPage::getResourceLoader 1 - MediaWiki\MediaWikiServices::getResourceLoader 15 - LocalisationCache::loadItem 108 - MediaWiki\Shell\FirejailCommand::buildFinalCommand 218 - GitInfo::getHead 241 - wfDebugLog 144 - spl_autoload_call@1 112 - MediaWiki\Logger\Monolog\LogstashFormatter::format 434 - proc_get_status 15 - MemcachedClient::_load_items 8 - Wikimedia\Rdbms\Database::query 221 - LocalisationCache::getSubitem 118 - SectionProfileCallback::__destruct 120 - MediaWiki\SpecialPage\SpecialPageFactory::getLocalNameFor 8 - Wikimedia\Rdbms\Database::executeQuery 4 - ApiResult::addValue 109 - MediaWiki\ExtensionInfo::getAuthorsFileName 21 - LocalisationCache::loadSubitem 8 - Wikimedia\Rdbms\Database::executeQueryAttempt 3 - json_decode 11 - MediumSpecificBagOStuff::getMulti 11 - MemcachedPhpBagOStuff::doGetMulti 11 - MemcachedClient::get_multi 10 - WANObjectCache::get 10 - WANObjectCache::getMulti 1 - gzcompress 118 - SectionProfiler::profileOutInternal 1164 - Wikimedia\AtEase\AtEase::suppressWarnings 24 - ResourceLoader::register 73995 - array_key_exists 108 - MediaWiki\Shell\Shell::command 10 - WebRequest::getSession 4 - ApiResult::validateValue 2 - Parser::internalParse 169 - ApiResult::validateValue@1 8 - Wikimedia\Rdbms\DatabaseMysqli::doQuery 6 - MediaWiki\Session\SessionManager::getGlobalSession 1 - MediaWiki\Session\SessionManager::getSessionForRequest 8 - mysqli::query 1 - LocalisationCache::getSubitemList 108 - MediaWiki\Shell\Command::buildFinalCommand 258 - Title::prefix 676 - is_readable 3384 - Composer\Autoload\ClassLoader::findFileWithExtension 960 - array_filter 501 - fread 1306 - ApiResult::validateValue@2 1 - MediaWiki\Session\SessionManager::getEmptySession 1 - MediaWiki\Session\SessionManager::getEmptySessionInternal 2 - Parser::internalParseHalfParsed 68 - Wikimedia\Services\ServiceContainer::getService@1 15 - Wikimedia\Services\ServiceContainer::createService@1 241 - Monolog\Logger::info 34 - unserialize 256 - Title::getNsText 188 - call_user_func_array 472 - SectionProfiler::getTime 16 - Wikimedia\Services\ServiceContainer::{closure}@1 111 - GitInfo::getCacheFilePath 1 - Skin::initPage 1 - Skin::preloadExistence 6 - MessageCache::getParser 112 - MediaWiki\Logger\Monolog\WikiProcessor::__invoke 1 - MediaWiki\MediaWikiServices::getParser 18 - Wikimedia\Rdbms\LoadBalancer::isMasterConnectionReadOnly 2 - MWTidy::tidy 4 - Parser::transformMsg 4 - Parser::preprocess 2 - MediaWiki\Tidy\RemexDriver::tidy 87 - Composer\Autoload\includeFile 10 - MediaWiki\Session\SessionManager::getSessionFromInfo 1 - OutputPage::getJSVars 124 - Title::getPrefixedText 19 - BagOStuff::getWithSetCallback 582 - Wikimedia\AtEase\AtEase::restoreWarnings 121 - MediaWiki\HookContainer\HookRunner::onGetLocalURL__Article 118 - AutoLoader::autoload@1 34 - Cdb\Reader\DBA::get 48 - Wikimedia\Rdbms\LoadBalancer::Wikimedia\Rdbms\{closure} 112 - Monolog\Formatter\NormalizerFormatter::format 34 - dba_fetch 32 - MediumSpecificBagOStuff::get 118 - ProfilerXhprof::scopedProfileIn 112 - Monolog\Formatter\NormalizerFormatter::normalize 4 - Wikimedia\Rdbms\DatabaseMysqlBase::serverIsReadOnly 10 - Parser::replaceVariables 1081 - escapeshellarg 75 - Wikimedia\ScopedCallback::__destruct 118 - Wikimedia\Timestamp\ConvertibleTimestamp::getTimestamp 137 - Title::newFromLinkTarget 66 - Wikimedia\ScopedCallback::consume 4 - RemexHtml\Tokenizer\Tokenizer::execute 13 - Title::newFromText 10 - MediaWiki\Session\SessionBackend::MediaWiki\Session\{closure} 12 - MediaWiki\Session\SessionBackend::save 13 - Title::newFromTextThrow 1 - ParserFactory::create 1 - Parser::__construct 8 - Parser::firstCallInit 118 - SectionProfiler::scopedProfileIn 1227 - Language::normalize 40 - Wikimedia\Services\ServiceContainer::getService@2 4 - MemcachedPhpBagOStuff::doGet 4 - MemcachedClient::get 10 - Wikimedia\Services\ServiceContainer::createService@2 279 - MediaWiki\Logger\LoggerFactory::getInstance 2 - Parser::handleInternalLinks 132 - MediaWiki\SpecialPage\SpecialPageFactory::getAliasList 2 - Parser::handleInternalLinks2 10 - Wikimedia\Services\ServiceContainer::{closure}@2 4 - section.query-m: SELECT @@GLOBAL.read_only AS Value 36 - array_flip 108 - MediaWiki\Shell\FirejailCommand::params 18 - Wikimedia\Rdbms\DBConnRef::__call 700 - ResourceLoader::isFileModule 1 - Skin::getSiteNotice 12 - Title::secureAndSplit 260 - MediaWikiTitleCodec::getNamespaceName 1227 - UtfNormal\Validator::cleanUp 6 - MessageCache::getParserOptions 1 - MediaWiki\HookContainer\HookRunner::onSiteNoticeAfter 1 - ApiFormatJson::execute 784 - Monolog\Formatter\NormalizerFormatter::normalize@1 1 - LinkBatch::execute 1 - LinkBatch::executeInto 1 - ParserOptions::__construct 2 - ResourceLoaderWikiModule::preloadTitleInfo 12 - MediaWikiTitleCodec::splitTitleString 21 - Wikimedia\ObjectFactory::getObjectFromSpec 128 - is_subclass_of 236 - getrusage 124 - Title::makeTitle 1 - SkinMustache::buildSearchProps 327 - fclose 1 - MirahezeMagicHooks::onSiteNoticeAfter 1798 - MediaWiki\Shell\Command::MediaWiki\Shell\{closure} 1 - StubObject::unstub 1 - StubObject::_unstub 2 - Wikimedia\Rdbms\Database::select 21 - MediaWiki\User\UserOptionsManager::getOption 44 - RequestContext::getLanguage 1 - LinkBatch::doQuery 1 - StubUserLang::_newObject 1 - CookieWarning\Hooks::generateElements 1 - ApiMain::__construct 13 - User::getOption 370 - Monolog\DateTimeImmutable::__construct 1 - ResourceLoaderClientHtml::getHeadHtml 1 - RemoteWiki::__construct 22 - MediaWiki\User\UserOptionsManager::loadUserOptions 112 - Monolog\Formatter\NormalizerFormatter::toJson 113 - WikiMap::getCurrentWikiId 593 - MediaWiki\HookContainer\HookContainer::getHandlers 109 - Wikimedia\AtEase\AtEase::quietCall 118 - SectionProfiler::profileInInternal 438 - realpath 137 - Title::getPrefixedDBkey 133 - json_encode 227 - is_file 1 - ApiMain::checkExecutePermissions 112 - Monolog\Utils::jsonEncode 1 - CentralAuthSessionProvider::unpersistSession 2 - RemexHtml\TreeBuilder\Dispatcher::startDocument 123 - TitleValue::__construct 2 - ApiResult::getResultData 20 - ApiBase::extractRequestParams 108 - MediaWiki\Shell\Command::params 594 - MediaWiki\HookContainer\HookContainer::getLegacyHandlers 262 - wfUrlencode 1 - ApiResult::applyTransformations 2 - ApiResult::applyTransformations@1 3 - ApiResult::applyTransformations@2 208 - WebRequest::getVal 6 - PPFrame_Hash::expand 889 - Language::caseFold 1 - OutputPage::getHeadLinksArray 1160 - error_reporting 5 - MediaWiki\User\DefaultOptionsLookup::getDefaultOptions 398 - Hooks::runner 108 - MediaWiki\Shell\CommandFactory::create 1267 - MediaWiki\MediaWikiServices::getContentLanguage 13609 - strtr 370 - DateTimeImmutable::__construct 27 - LocalisationCache::getItem 4 - Monolog\Logger::error 10 - MediaWiki\MediaWikiServices::getPermissionManager 2 - ApiBase::getPermissionManager 108 - MediaWiki\Shell\Shell::escape 117 - ApiResult::applyTransformations@3 135 - TitleValue::assertValidSpec 260 - Language::getNsText 889 - Language::uc 5 - Parser::braceSubstitution 110 - wfGetCaller 13 - Wikimedia\ObjectFactory::createObject 279 - MediaWiki\Logger\MonologSpi::getLogger 1 - SkinMustache::tailElement 193 - file_exists 125 - MediaWiki\Interwiki\ClassicInterwikiLookup::fetch 2 - Title::getCanonicalURL 96 - call_user_func 1 - MediaWiki\HookContainer\HookRunner::onParserFirstCallInit 2 - MediaWiki\SpecialPage\SpecialPageFactory::getPageList 1 - ApiMain::lacksSameOriginSecurity 1 - Language::getMonthNamesArray 14 - ApiBase::getHookRunner 12 - Language::getMonthName 1 - CentralAuthUtils::setP3P 12 - Language::getMessageFromDB 3 - Skin::footerLinkTitle 1 - Skin::printSource 1 - MediaWiki\HookContainer\HookRunner::onUserGetDefaultOptions Redmin reopened this task as Open.May 7 2021, 19:00 Redmin edited subscribers, added: Redmin; removed: BrandonWM. RhinosF1 renamed this task from Wiki-Bot to section.execute-GitInfo->getHeadCommitDate is very slow.May 7 2021, 19:02 RhinosF1 updated the task description. (Show Details) RhinosF1 edited projects, added Performance, MediaWiki, MediaWiki (SRE); removed Site Reliability Engineering. Herald added a subscriber: NDKilla. · View Herald TranscriptMay 7 2021, 19:02 RhinosF1 added a comment.May 7 2021, 19:02 Comment Actions Updated title/description to actually state the issue MarkusRost subscribed.May 7 2021, 19:03 MarkusRost added a comment.May 7 2021, 19:16 Comment Actions In T6471#144193, @Lev wrote: I've investigated the problem: the reason for timeouts when using Wiki-Bot is because it asks for the whole list of extensions. Miraheze wikis respond to that specific request (`https://meta.miraheze.org/w/api.php?action=query&meta=siteinfo&siprop=general\|extensions` with `\|extensions`) too slowly, slower than most MediaWiki sites, and that is why the bot has problems. Asking simply for `general` is fine. If I had to guess, it's probably something to do with ManageWiki. (Leaving this here for people searching this in the future.) The bot is asking for extensions only for a few requests (setup and test) to make sure the wiki has all the extensions for full bot functionality. But those extensions are no longer as required as before (I have pretty well working fallback code if they are missing). Therefore I'll look into removing the extensions part from my requests over the next few days. Reception123 added a subscriber: Southparkfan.May 10 2021, 09:06 Comment Actions @Southparkfan Southparkfan added a comment.May 13 2021, 19:59 Comment Actions `GitInfo::getHeadCommitDate` tries to fetch the timestamp of the last commit for each extension. This can be very slow: southparkfan@mw8:/srv/mediawiki/w/extensions/VisualEditor$ time git show -s --format=format:%ct HEAD 1620421442 real 0m0.892s user 0m0.006s sys 0m0.012s southparkfan@mw8:/srv/mediawiki/w/extensions/Wikibase$ time git show -s --format=format:%ct HEAD 1616149709 real 0m1.892s user 0m0.008s sys 0m0.019s (subtract 0.0010s for overhead) The cause? Disk access (according to `strace`: `lstat`/`read`/`stat`/`openat`). However, fetching the timestamp a second time reduces the response time to 0m0.006s, because the files in the `.git/` directory are cached in memory. I am aware our HDDs are not fast, that will not be 'fixed' in the near future. MediaWiki shouldn't make so many I/O calls in the first place. Per https://phabricator.wikimedia.org/T131003#5228332, we could add $wgGitInfoCacheDirectory backport https://gerrit.wikimedia.org/r/c/mediawiki/core/+/513729/ to MirahezeMagic. Adding a json file for caching will reduce I/O by a huge margin. Unknown Object (User) added a comment.May 22 2021, 10:10 Comment Actions @Southparkfan I do not think we can backport that patch do the change to `includes/GitInfo.php`, unless the script would work without that change, in which case we could. Southparkfan added a comment.May 30 2021, 18:39 Comment Actions In T6471#146147, @Universal_Omega wrote: @Southparkfan I do not think we can backport that patch do the change to `includes/GitInfo.php`, unless the script would work without that change, in which case we could. Not sure the script can be used withour the custom patch. Other than that, this task's priority can be lowered, I think? Unknown Object (User) moved this task from Backlog to Long Term on the MediaWiki (SRE) board.Jun 15 2021, 17:08 Reception123 lowered the priority of this task from Normal to Low.Jun 20 2021, 16:54 Restricted Repository Identity mentioned this in R9:9af6e36c5c78: T6471: Set wgGitInfoCacheDirectory.Jun 28 2021, 01:53 Reception123 closed this task as Resolved.Aug 1 2021, 06:11 Reception123 assigned this task to Unknown Object (User). Comment Actions fixed by @Universal_Omega Herald added a project: Universal Omega. · View Herald TranscriptAug 1 2021, 06:11 Log In to Comment

100.00% 9223.385 1 - main() 1 - wfApiMain 1 - ApiMain::execute 1 - ApiMain::executeActionWithErrorHandling 1 - ApiMain::executeAction 108 - MediaWiki\Shell\Command::execute 1 - ApiQuery::execute 1 - ApiQuerySiteinfo::execute 1 - ApiQuerySiteinfo::appendExtensions 326 - stream_select 107 - GitInfo::getHeadCommitDate 107 - section.execute-GitInfo->getHeadCommitDate 581 - MediaWiki\HookContainer\HookContainer::run 271 - MediaWiki\HookContainer\HookContainer::callLegacyHook 1 - ApiMain::printResult 1 - ApiFormatBase::closePrinter 8 - WANObjectCache::getWithSetCallback 7 - WANObjectCache::fetchOrRegenerate 1 - MediaWiki\Api\ApiHookRunner::onApiFormatHighlight 1 - SyntaxHighlight::onApiFormatHighlight 1 - SyntaxHighlight::highlight 1 - SyntaxHighlight::{closure} 1 - section.execute-SyntaxHighlight::{closure} 1 - OutputPage::output 108 - proc_open 2 - CookieWarning\Decisions::shouldShowCookieWarning 2 - CookieWarning\Decisions::isInConfiguredRegion 2 - CookieWarning\Decisions::getCountryCodeFromIP 2 - CookieWarning\HttpGeoLocation::locate 2 - Http::get 2 - Http::request 2 - MediaWiki\Http\HttpRequestFactory::request 2 - GuzzleHttpRequest::execute 2 - GuzzleHttp\Client::send 2 - GuzzleHttp\Client::sendAsync 2 - GuzzleHttp\Client::transfer 2 - GuzzleHttp\HandlerStack::__invoke 10 - GuzzleHttp\Middleware::GuzzleHttp\{closure} 2 - GuzzleHttp\RedirectMiddleware::__invoke 2 - GuzzleHttp\PrepareBodyMiddleware::__invoke 2 - GuzzleHttp\Middleware::GuzzleHttp\{closure}@1 2 - GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure} 2 - GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure}@1 2 - GuzzleHttp\Handler\CurlHandler::__invoke 1 - MediaWiki\HookContainer\HookRunner::onSetupAfterCache 1 - CreateWikiHooks::onSetupAfterCache 2 - curl_exec 18 - Wikimedia\Rdbms\LoadBalancer::getConnection 18 - Wikimedia\Rdbms\LoadBalancer::getServerConnection 13 - wfGetDB 13 - Wikimedia\Rdbms\LoadBalancer::getMaintenanceConnectionRef 1 - CreateWikiJson::__construct 2 - Wikimedia\Rdbms\LoadBalancer::reallyOpenConnection 2 - Wikimedia\Rdbms\Database::initConnection 2 - Wikimedia\Rdbms\Database::doInitConnection 2 - Wikimedia\Rdbms\DatabaseMysqlBase::open 2 - Wikimedia\Rdbms\DatabaseMysqli::mysqlConnect 2 - mysqli::real_connect 2 - Wikimedia\Rdbms\LoadBalancer::getForeignConnection 1 - MediaWiki\HookContainer\HookRunner::onBeforePageDisplay 1 - CookieWarning\Hooks::onBeforePageDisplay 1 - SkinTemplate::outputPage 1 - SkinMustache::generateHTML 1 - SkinMustache::getTemplateData 1 - Skin::afterContentHook 1 - MediaWiki\HookContainer\HookRunner::onSkinAfterContent 1 - CookieWarning\Hooks::onSkinAfterContent 391 - Monolog\Logger::addRecord 146 - Monolog\Logger::debug 112 - Monolog\Handler\WhatFailureGroupHandler::handle 112 - Monolog\Handler\AbstractProcessingHandler::handle 40 - Message::toString 410 - spl_autoload_call 112 - Monolog\Handler\SyslogUdpHandler::write 112 - Monolog\Handler\SyslogUdp\UdpSocket::write 112 - Monolog\Handler\SyslogUdp\UdpSocket::send 1 - ApiQuerySiteinfo::appendGeneralInfo 112 - socket_sendto 35 - Message::text 2653 - Wikimedia\Services\ServiceContainer::getService 1 - OutputPage::headElement 434 - AutoLoader::autoload 111 - GitInfo::__construct 23 - Wikimedia\Services\ServiceContainer::createService 25 - Wikimedia\Services\ServiceContainer::{closure} 1 - SiteConfiguration::extractAllGlobals 827 - SiteConfiguration::extractGlobalSetting 831 - SiteConfiguration::processSetting 118 - Title::getLinkURL 3 - OutputPage::getRlClient 109 - MediaWiki\ExtensionInfo::getLicenseFileNames 218 - scandir 119 - SpecialPage::getTitleFor 54 - Message::fetchMessage 123 - Title::getLocalURL 47 - MessageCache::get 107 - GitInfo::getHeadViewUrl 3284 - Composer\Autoload\ClassLoader::loadClass 47 - MessageCache::getMessageFromFallbackChain 47 - MessageCache::getMessageForLang 107 - GitInfo::getRemoteUrl 115 - wfTimestamp 117 - Wikimedia\Timestamp\ConvertibleTimestamp::convert 47 - MessageCache::getMsgFromNamespace 120 - SpecialPage::getTitleValueFor 1 - ApiQueryBase::getDB 1 - ApiBase::getDB 2 - Message::parseText 3 - Title::newMainPage 2 - MessageCache::parse 37 - Message::transformText 2 - Parser::parse 37 - MessageCache::transform 1 - ApiMain::reportUnusedParams 16 - Wikimedia\Rdbms\LoadBalancer::getLocalConnection 1 - ApiBase::addWarning 1 - ApiErrorFormatter::addWarning 220 - file_get_contents 47 - MessageCache::load 1 - ApiErrorFormatter_BackCompat::addWarningOrError 1 - Message::parseAsBlock 6173 - preg_match 2 - MediumSpecificBagOStuff::add 2 - MemcachedPhpBagOStuff::doAdd 3499 - Composer\Autoload\ClassLoader::findFile 2 - MemcachedClient::add 2 - MemcachedClient::_set 107 - parse_ini_file 32 - MemcachedClient::_fgets 32 - fgets 110 - GitInfo::getHeadSHA1 1 - WANObjectCache::set 34 - LCStoreCDB::get 118 - Wikimedia\Timestamp\ConvertibleTimestamp::__construct 3 - OutputPage::getRlClientContext 118 - Wikimedia\Timestamp\ConvertibleTimestamp::setTimestamp 6 - OutputPage::getResourceLoader 1 - MediaWiki\MediaWikiServices::getResourceLoader 15 - LocalisationCache::loadItem 108 - MediaWiki\Shell\FirejailCommand::buildFinalCommand 218 - GitInfo::getHead 241 - wfDebugLog 144 - spl_autoload_call@1 112 - MediaWiki\Logger\Monolog\LogstashFormatter::format 434 - proc_get_status 15 - MemcachedClient::_load_items 8 - Wikimedia\Rdbms\Database::query 221 - LocalisationCache::getSubitem 118 - SectionProfileCallback::__destruct 120 - MediaWiki\SpecialPage\SpecialPageFactory::getLocalNameFor 8 - Wikimedia\Rdbms\Database::executeQuery 4 - ApiResult::addValue 109 - MediaWiki\ExtensionInfo::getAuthorsFileName 21 - LocalisationCache::loadSubitem 8 - Wikimedia\Rdbms\Database::executeQueryAttempt 3 - json_decode 11 - MediumSpecificBagOStuff::getMulti 11 - MemcachedPhpBagOStuff::doGetMulti 11 - MemcachedClient::get_multi 10 - WANObjectCache::get 10 - WANObjectCache::getMulti 1 - gzcompress 118 - SectionProfiler::profileOutInternal 1164 - Wikimedia\AtEase\AtEase::suppressWarnings 24 - ResourceLoader::register 73995 - array_key_exists 108 - MediaWiki\Shell\Shell::command 10 - WebRequest::getSession 4 - ApiResult::validateValue 2 - Parser::internalParse 169 - ApiResult::validateValue@1 8 - Wikimedia\Rdbms\DatabaseMysqli::doQuery 6 - MediaWiki\Session\SessionManager::getGlobalSession 1 - MediaWiki\Session\SessionManager::getSessionForRequest 8 - mysqli::query 1 - LocalisationCache::getSubitemList 108 - MediaWiki\Shell\Command::buildFinalCommand 258 - Title::prefix 676 - is_readable 3384 - Composer\Autoload\ClassLoader::findFileWithExtension 960 - array_filter 501 - fread 1306 - ApiResult::validateValue@2 1 - MediaWiki\Session\SessionManager::getEmptySession 1 - MediaWiki\Session\SessionManager::getEmptySessionInternal 2 - Parser::internalParseHalfParsed 68 - Wikimedia\Services\ServiceContainer::getService@1 15 - Wikimedia\Services\ServiceContainer::createService@1 241 - Monolog\Logger::info 34 - unserialize 256 - Title::getNsText 188 - call_user_func_array 472 - SectionProfiler::getTime 16 - Wikimedia\Services\ServiceContainer::{closure}@1 111 - GitInfo::getCacheFilePath 1 - Skin::initPage 1 - Skin::preloadExistence 6 - MessageCache::getParser 112 - MediaWiki\Logger\Monolog\WikiProcessor::__invoke 1 - MediaWiki\MediaWikiServices::getParser 18 - Wikimedia\Rdbms\LoadBalancer::isMasterConnectionReadOnly 2 - MWTidy::tidy 4 - Parser::transformMsg 4 - Parser::preprocess 2 - MediaWiki\Tidy\RemexDriver::tidy 87 - Composer\Autoload\includeFile 10 - MediaWiki\Session\SessionManager::getSessionFromInfo 1 - OutputPage::getJSVars 124 - Title::getPrefixedText 19 - BagOStuff::getWithSetCallback 582 - Wikimedia\AtEase\AtEase::restoreWarnings 121 - MediaWiki\HookContainer\HookRunner::onGetLocalURL__Article 118 - AutoLoader::autoload@1 34 - Cdb\Reader\DBA::get 48 - Wikimedia\Rdbms\LoadBalancer::Wikimedia\Rdbms\{closure} 112 - Monolog\Formatter\NormalizerFormatter::format 34 - dba_fetch 32 - MediumSpecificBagOStuff::get 118 - ProfilerXhprof::scopedProfileIn 112 - Monolog\Formatter\NormalizerFormatter::normalize 4 - Wikimedia\Rdbms\DatabaseMysqlBase::serverIsReadOnly 10 - Parser::replaceVariables 1081 - escapeshellarg 75 - Wikimedia\ScopedCallback::__destruct 118 - Wikimedia\Timestamp\ConvertibleTimestamp::getTimestamp 137 - Title::newFromLinkTarget 66 - Wikimedia\ScopedCallback::consume 4 - RemexHtml\Tokenizer\Tokenizer::execute 13 - Title::newFromText 10 - MediaWiki\Session\SessionBackend::MediaWiki\Session\{closure} 12 - MediaWiki\Session\SessionBackend::save 13 - Title::newFromTextThrow 1 - ParserFactory::create 1 - Parser::__construct 8 - Parser::firstCallInit 118 - SectionProfiler::scopedProfileIn 1227 - Language::normalize 40 - Wikimedia\Services\ServiceContainer::getService@2 4 - MemcachedPhpBagOStuff::doGet 4 - MemcachedClient::get 10 - Wikimedia\Services\ServiceContainer::createService@2 279 - MediaWiki\Logger\LoggerFactory::getInstance 2 - Parser::handleInternalLinks 132 - MediaWiki\SpecialPage\SpecialPageFactory::getAliasList 2 - Parser::handleInternalLinks2 10 - Wikimedia\Services\ServiceContainer::{closure}@2 4 - section.query-m: SELECT @@GLOBAL.read_only AS Value 36 - array_flip 108 - MediaWiki\Shell\FirejailCommand::params 18 - Wikimedia\Rdbms\DBConnRef::__call 700 - ResourceLoader::isFileModule 1 - Skin::getSiteNotice 12 - Title::secureAndSplit 260 - MediaWikiTitleCodec::getNamespaceName 1227 - UtfNormal\Validator::cleanUp 6 - MessageCache::getParserOptions 1 - MediaWiki\HookContainer\HookRunner::onSiteNoticeAfter 1 - ApiFormatJson::execute 784 - Monolog\Formatter\NormalizerFormatter::normalize@1 1 - LinkBatch::execute 1 - LinkBatch::executeInto 1 - ParserOptions::__construct 2 - ResourceLoaderWikiModule::preloadTitleInfo 12 - MediaWikiTitleCodec::splitTitleString 21 - Wikimedia\ObjectFactory::getObjectFromSpec 128 - is_subclass_of 236 - getrusage 124 - Title::makeTitle 1 - SkinMustache::buildSearchProps 327 - fclose 1 - MirahezeMagicHooks::onSiteNoticeAfter 1798 - MediaWiki\Shell\Command::MediaWiki\Shell\{closure} 1 - StubObject::unstub 1 - StubObject::_unstub 2 - Wikimedia\Rdbms\Database::select 21 - MediaWiki\User\UserOptionsManager::getOption 44 - RequestContext::getLanguage 1 - LinkBatch::doQuery 1 - StubUserLang::_newObject 1 - CookieWarning\Hooks::generateElements 1 - ApiMain::__construct 13 - User::getOption 370 - Monolog\DateTimeImmutable::__construct 1 - ResourceLoaderClientHtml::getHeadHtml 1 - RemoteWiki::__construct 22 - MediaWiki\User\UserOptionsManager::loadUserOptions 112 - Monolog\Formatter\NormalizerFormatter::toJson 113 - WikiMap::getCurrentWikiId 593 - MediaWiki\HookContainer\HookContainer::getHandlers 109 - Wikimedia\AtEase\AtEase::quietCall 118 - SectionProfiler::profileInInternal 438 - realpath 137 - Title::getPrefixedDBkey 133 - json_encode 227 - is_file 1 - ApiMain::checkExecutePermissions 112 - Monolog\Utils::jsonEncode 1 - CentralAuthSessionProvider::unpersistSession 2 - RemexHtml\TreeBuilder\Dispatcher::startDocument 123 - TitleValue::__construct 2 - ApiResult::getResultData 20 - ApiBase::extractRequestParams 108 - MediaWiki\Shell\Command::params 594 - MediaWiki\HookContainer\HookContainer::getLegacyHandlers 262 - wfUrlencode 1 - ApiResult::applyTransformations 2 - ApiResult::applyTransformations@1 3 - ApiResult::applyTransformations@2 208 - WebRequest::getVal 6 - PPFrame_Hash::expand 889 - Language::caseFold 1 - OutputPage::getHeadLinksArray 1160 - error_reporting 5 - MediaWiki\User\DefaultOptionsLookup::getDefaultOptions 398 - Hooks::runner 108 - MediaWiki\Shell\CommandFactory::create 1267 - MediaWiki\MediaWikiServices::getContentLanguage 13609 - strtr 370 - DateTimeImmutable::__construct 27 - LocalisationCache::getItem 4 - Monolog\Logger::error 10 - MediaWiki\MediaWikiServices::getPermissionManager 2 - ApiBase::getPermissionManager 108 - MediaWiki\Shell\Shell::escape 117 - ApiResult::applyTransformations@3 135 - TitleValue::assertValidSpec 260 - Language::getNsText 889 - Language::uc 5 - Parser::braceSubstitution 110 - wfGetCaller 13 - Wikimedia\ObjectFactory::createObject 279 - MediaWiki\Logger\MonologSpi::getLogger 1 - SkinMustache::tailElement 193 - file_exists 125 - MediaWiki\Interwiki\ClassicInterwikiLookup::fetch 2 - Title::getCanonicalURL 96 - call_user_func 1 - MediaWiki\HookContainer\HookRunner::onParserFirstCallInit 2 - MediaWiki\SpecialPage\SpecialPageFactory::getPageList 1 - ApiMain::lacksSameOriginSecurity 1 - Language::getMonthNamesArray 14 - ApiBase::getHookRunner 12 - Language::getMonthName 1 - CentralAuthUtils::setP3P 12 - Language::getMessageFromDB 3 - Skin::footerLinkTitle 1 - Skin::printSource 1 - MediaWiki\HookContainer\HookRunner::onUserGetDefaultOptions

Redmin reopened this task as Open.May 7 2021, 19:00

Redmin edited subscribers, added: Redmin; removed: BrandonWM.

RhinosF1 renamed this task from Wiki-Bot to section.execute-GitInfo->getHeadCommitDate is very slow.May 7 2021, 19:02

RhinosF1 updated the task description. (Show Details)

RhinosF1 edited projects, added Performance, MediaWiki, MediaWiki (SRE); removed Site Reliability Engineering.

Herald added a subscriber: NDKilla. · View Herald TranscriptMay 7 2021, 19:02

Updated title/description to actually state the issue

MarkusRost subscribed.May 7 2021, 19:03

In T6471#144193, @Lev wrote:

I've investigated the problem: the reason for timeouts when using Wiki-Bot is because it asks for the whole list of extensions. Miraheze wikis respond to that specific request (https://meta.miraheze.org/w/api.php?action=query&meta=siteinfo&siprop=general|extensions with |extensions) too slowly, slower than most MediaWiki sites, and that is why the bot has problems. Asking simply for general is fine. If I had to guess, it's probably something to do with ManageWiki. (Leaving this here for people searching this in the future.)

The bot is asking for extensions only for a few requests (setup and test) to make sure the wiki has all the extensions for full bot functionality. But those extensions are no longer as required as before (I have pretty well working fallback code if they are missing).
Therefore I'll look into removing the extensions part from my requests over the next few days.

@Southparkfan

GitInfo::getHeadCommitDate tries to fetch the timestamp of the last commit for each extension. This can be very slow:

southparkfan@mw8:/srv/mediawiki/w/extensions/VisualEditor$ time git show -s --format=format:%ct HEAD
1620421442

real    0m0.892s
user    0m0.006s
sys     0m0.012s

southparkfan@mw8:/srv/mediawiki/w/extensions/Wikibase$ time git show -s --format=format:%ct HEAD
1616149709

real    0m1.892s
user    0m0.008s
sys     0m0.019s

(subtract 0.0010s for overhead)

The cause? Disk access (according to strace: lstat/read/stat/openat). However, fetching the timestamp a second time reduces the response time to 0m0.006s, because the files in the .git/ directory are cached in memory. I am aware our HDDs are not fast, that will not be 'fixed' in the near future. MediaWiki shouldn't make so many I/O calls in the first place.

Per https://phabricator.wikimedia.org/T131003#5228332, we could add $wgGitInfoCacheDirectory backport https://gerrit.wikimedia.org/r/c/mediawiki/core/+/513729/ to MirahezeMagic. Adding a json file for caching will reduce I/O by a huge margin.

@Southparkfan I do not think we can backport that patch do the change to includes/GitInfo.php, unless the script would work without that change, in which case we could.

In T6471#146147, @Universal_Omega wrote:

@Southparkfan I do not think we can backport that patch do the change to includes/GitInfo.php, unless the script would work without that change, in which case we could.

Not sure the script can be used withour the custom patch.

Other than that, this task's priority can be lowered, I think?

Unknown Object (User) moved this task from Backlog to Long Term on the MediaWiki (SRE) board.Jun 15 2021, 17:08

Reception123 lowered the priority of this task from Normal to Low.Jun 20 2021, 16:54

Restricted Repository Identity mentioned this in R9:9af6e36c5c78: T6471: Set wgGitInfoCacheDirectory.Jun 28 2021, 01:53

fixed by @Universal_Omega

Herald added a project: Universal Omega. · View Herald TranscriptAug 1 2021, 06:11

section.execute-GitInfo->getHeadCommitDate is very slowClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

section.execute-GitInfo->getHeadCommitDate is very slow
Closed, ResolvedPublic
Actions