Page MenuHomePhabricator

MediaModeration: Increased thumbnail transform time since DC switchover
Closed, ResolvedPublic

Description

As part of a routine check of the MediaModeration dashboard in Grafana, I noticed that since the DC switchover on March 25, the time needed for image transformations increased (the "php" stat is now at 8.7s for p75).

image.png (1,579×531 px, 84 KB)

Given that PHP file transform is used only on errors in Thumbor response, the issue may stem from increase in error responses from Thumbor (filtered out 200 and 404):

image.png (1,582×543 px, 41 KB)

Event Timeline

I think you're requesting the 330px standard size. Can you switch to 500px instead? That is the size that is being advertised as 2x for default thumb size and hit ratio in swift should be much higher than the other standard size (330px)

note that thumbnails don't get replicated across swift clusters in different DCs, so changes to runtime after the switchover is actually expected.

I think you're requesting the 330px standard size. Can you switch to 500px instead?

We could, but ideally we wouldn't want to unless necessary as the larger the thumbnail the more likely it would be too large for PhotoDNA (they impose a upper limit on the file height)

I think you're requesting the 330px standard size. Can you switch to 500px instead?

We could, but ideally we wouldn't want to unless necessary as the larger the thumbnail the more likely it would be too large for PhotoDNA (they impose a upper limit on the file height)

Another good one is 250px (default thumb size, 1x), you could try to prefer those instead if height is large?

I think you're requesting the 330px standard size. Can you switch to 500px instead?

We could, but ideally we wouldn't want to unless necessary as the larger the thumbnail the more likely it would be too large for PhotoDNA (they impose a upper limit on the file height)

Another good one is 250px (default thumb size, 1x), you could try to prefer those instead if height is large?

Yeah, 250px should be fine

Change #1287869 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/MediaModeration@master] Change $wgMediaModerationThumbnailWidth to 250

https://gerrit.wikimedia.org/r/1287869

Change #1287869 merged by jenkins-bot:

[mediawiki/extensions/MediaModeration@master] Change $wgMediaModerationThumbnailWidth to 250

https://gerrit.wikimedia.org/r/1287869

Rate of using source image over thumbnail has dramatically decreased after the train reached all wikis with this patch:

image.png (1,358×625 px, 47 KB)

I think given this change it probably makes sense to do a rescan of all unscannable images because it seems to have improved the chance an image can successfully scan

Mentioned in SAL (#wikimedia-operations) [2026-05-26T21:27:45Z] <Dreamy_Jazz> Running /usr/local/bin/foreachwikiindblist "all.dblist - mediamoderation-continuous-scan.dblist - preinstall.dblist" extensions/MediaModeration/maintenance/scanFilesInScanTable.php --use-jobqueue --sleep=1 --poll-sleep=10 --verbose in tmux session - T421688

Will leave the rescan code to run and then will probably also target Wikimedia Commons with the same script, but no need to keep the task open for that