Bucketing should include registered users with 100 or fewer edits at first exposure.
Only users at participating Wikipedias as identified in T420664
All users, who are editing a mobile web or desktop main namespace page (NS:0) using the visual editor, at any of the Wikipedias should have a 50% chance of being included/bucketed into the A/B test's control or treatment group.
The treatment group should have Edit Suggestions enabled while the control group will not.
Each device should remain in the same test group for the duration of the test (and across sessions and pages). Note: If the cookie is cleared manually or automatically – by, for example, a privacy enhanced browser – the client is issued a new wmf-uniq cookie, and they may or may not end up enrolled or even assigned to the same group.

Instrumentation Requirements

Ensure Test Kitchen stream is configured appropriately
.
.

Open question(s)

1. Will non-cache splitting be available by the time the controlled experiment is scheduled to begin?
- No.
2. If the answer to "1." is "No." and we move forward with cache splitting, what % of traffic (at en.wiki) will we sample?
- We are limiting this experiment to logged-in users. Accordingly, 100% of traffic will be included in the experiment at all wikis.

Participating wikis

Only those with ~85% UI translation, and good configurations, and no concerns raised onwiki

ID	UI % transl.	json ready	on-wiki discussion(s)
en.wiki	-	good	link
fr.wiki	99%	good	link
de.wiki	100%	good	link
ja.wiki	99%	good	link
es.wiki	98%	good	link
ru.wiki	96%	good	link
zh.wiki	99%	good	link
pl.wiki	99%	good	link
nl.wiki	99%	good	link
he.wiki	99%	good enough (needs disambig)	link
ar.wiki	100%	good	link
cs.wiki	99%	good	link
ko.wiki	99%	TKTK	link
sv.wiki	99%	good	link
vi.wiki	100%	TKTK	link

NOTE: List of wikis was finalized via T420664.

Wikis for later consideration

Per offline discussion, we're going to omit pt.wiki, vi.wiki, it.wiki, and bn.wiki from the initial experiment start. In T425006, we'll work to get the UI translated and include them in the experiment soon after it start.

ID	UI % transl.	json ready	on-wiki discussion(s)
pt.wiki	28%/72%	needs work	link
it.wiki	49%	TKTK	link
bn.wiki	43%	TKTK	link

Related Objects
Search...

Status	Subtype	Assigned	Task
Open		None	T360489 [EPIC] Generate and present edit suggestions at scale
Open		None	T414853 [MILESTONE] Deploy Suggestion Mode as a default-on feature.
Open		None	T404600 [MILESTONE] Run controlled experiment of Suggestion Mode MVP
Resolved		medelius	T421189 Start Suggestion Mode experiment in TestKitchen
Resolved		Esanders	T422032 Update normal-enabled-flagged button colours in OOUI
Resolved		Esanders	T422031 Implement large/small buttons in OOUI
Open	BUG REPORT	medelius	T423740 Instrumentation to measure checks/suggestions shown within VisualEditor broken on mobile

Event Timeline

ppelberg created this task.Mar 24 2026, 11:11 PM

ppelberg updated the task description. (Show Details)

ppelberg mentioned this in T418530: [SPIKE] Decide whether the Suggestion Mode MVP controlled experiment will run on Test Kitchen.

ppelberg updated the task description. (Show Details)Mar 25 2026, 8:12 PM

ppelberg moved this task from Backlog to New Suggestions on the VisualEditor Suggestion Mode board.Mar 27 2026, 7:52 PM

ppelberg moved this task from New Suggestions to Backlog on the VisualEditor Suggestion Mode board.

ppelberg triaged this task as High priority.Mar 30 2026, 5:22 PM

MNeisler updated the task description. (Show Details)Apr 1 2026, 5:00 PM

MNeisler updated the task description. (Show Details)Apr 1 2026, 5:02 PM

ppelberg updated the task description. (Show Details)Apr 10 2026, 11:24 PM

Restricted Application added subscribers: hubaishan, Stang, revi. · View Herald TranscriptApr 10 2026, 11:24 PM

ppelberg edited projects, added: Editing-team (Editing-Q4-13Apr-24Apr-2026); removed: Editing-team (Editing-Q4-30Mar-10Apr-2026).Apr 10 2026, 11:25 PM

Restricted Application added a subscriber: alaa. · View Herald TranscriptApr 10 2026, 11:25 PM

ppelberg moved this task from Decision to be made to Release Prep on the Editing-team (Editing-Q4-13Apr-24Apr-2026) board.Apr 10 2026, 11:25 PM

ppelberg updated the task description. (Show Details)

ldelench_wmf moved this task from Backlog to WE1.7 on the OKR-Work (WE1 FY2025-26) board.Apr 14 2026, 6:08 PM

medelius subscribed.Apr 14 2026, 8:21 PM

Quiddity updated the task description. (Show Details)Apr 15 2026, 10:33 PM

ppelberg mentioned this in T419066: [Suggestion Mode] Report on leading indicators.Apr 16 2026, 10:05 PM

ppelberg mentioned this in T404600: [MILESTONE] Run controlled experiment of Suggestion Mode MVP.Apr 16 2026, 11:23 PM

ppelberg added a subtask: T422032: Update normal-enabled-flagged button colours in OOUI.Apr 17 2026, 4:14 AM

ppelberg added a subtask: T422031: Implement large/small buttons in OOUI.

Quiddity updated the task description. (Show Details)Apr 17 2026, 5:58 PM

Quiddity updated the task description. (Show Details)Apr 17 2026, 7:40 PM

ppelberg added a subtask: T423740: Instrumentation to measure checks/suggestions shown within VisualEditor broken on mobile.Apr 17 2026, 9:07 PM

Quiddity mentioned this in T420667: [Suggestion Mode] Announce proposed start of controlled experiment.Apr 17 2026, 11:42 PM

Quiddity updated the task description. (Show Details)Apr 18 2026, 12:07 AM

Quiddity updated the task description. (Show Details)Apr 24 2026, 9:00 PM

Quiddity updated the task description. (Show Details)Apr 24 2026, 9:22 PM

Quiddity updated the task description. (Show Details)Apr 27 2026, 9:11 PM

Quiddity updated the task description. (Show Details)Apr 28 2026, 4:48 PM

DLynch edited projects, added: Editing-team (Editing-Q4-27Apr-8May-2026); removed: Editing-team (Editing-Q4-13Apr-24Apr-2026).Apr 28 2026, 5:27 PM

ppelberg updated the task description. (Show Details)Apr 29 2026, 1:41 PM

Next steps

Editing Engineering: create JSON pages basic json pages at zh.wiki and ko.wiki
@ppelberg: decide which, if any, of the languages that have ≤85% of the UI translated (bn, it, pt, vi) will we include in the initial experiment start?

Peter's Notes

At Bn.wiki, the following "core" UI elements are missing: bold, tone, year link, caption, add citaiton, remove external link, duplicate link, disambiguation, heading level, image caption, convert ref,
At pt.wiki, the following "core" UI elements are missing:
- year link, bold, convert reference, caption, heading level, disambiguation link, duplicate link, fix year link
At it.wiki, the following "core" UI elements are missing:
- tone, fix year link, caption, convert reference, external link, duplicate link, headling level, convert reference,
At vi.wiki, the following "core" UI elements are missing:
- bold, fix year link, caption, convert ref, external link, duplicate link, heading levle, caption,

"core" here refers to copy within suggestions themselves and omits more meta aspects of the UX. E.g. success toasts, copy on Special:EditChecks, etc.//

ppelberg moved this task from Decision to be made to Blocked on the Editing-team (Editing-Q4-27Apr-8May-2026) board.Apr 29 2026, 2:22 PM

ppelberg closed subtask T422031: Implement large/small buttons in OOUI as Resolved.Apr 29 2026, 2:32 PM

ppelberg closed subtask T422032: Update normal-enabled-flagged button colours in OOUI as Resolved.

Editing Engineering: create JSON pages basic json pages at zh.wiki and ko.wiki

Is this necessary? A null config should be valid (i.e. use all defaults), and I'm seeing suggestions on kowki if enabled by querystring. cc @Quiddity

ppelberg renamed this task from Deploy config to start the Suggestion Mode MVP controlled experiment to Start Suggestion Mode experiment in TestKitchen.Apr 29 2026, 5:13 PM

ppelberg updated the task description. (Show Details)

ppelberg moved this task from Blocked to Dev Doing on the Editing-team (Editing-Q4-27Apr-8May-2026) board.Apr 29 2026, 5:16 PM

ppelberg moved this task from Dev Doing to Blocked on the Editing-team (Editing-Q4-27Apr-8May-2026) board.

ppelberg updated the task description. (Show Details)Apr 30 2026, 3:25 PM

ppelberg updated the task description. (Show Details)

ppelberg updated the task description. (Show Details)Apr 30 2026, 3:30 PM

In T421189#11872580, @Esanders wrote:

Editing Engineering: create JSON pages basic json pages at zh.wiki and ko.wiki

Is this necessary? A null config should be valid (i.e. use all defaults), and I'm seeing suggestions on kowki if enabled by querystring. cc @Quiddity

Zhwiki was done by a local admin.

Kowiki I've proposed a recommended basic config at https://ko.wikipedia.org/wiki/%EB%AF%B8%EB%94%94%EC%96%B4%EC%9C%84%ED%82%A4%ED%86%A0%EB%A1%A0:Editcheck-config.json#c-Quiddity_(WMF)-20260430194900-Quiddity_(WMF)-20260427201200 and noted that I would ask an engineer to implement that locally, so that they have something to start with. I believe it is acceptable for you/us to help them like that.

ppelberg updated the task description. (Show Details)May 6 2026, 5:13 PM

@medelius: could you please ADD vi.wiki to the experiment?

Thanks to help from @Geugeor and @PPham, 100% of the UI is now translated into Vietnamese 🙌🏼

Sure thing! It's been added.

In T421189#11895872, @medelius wrote:

Sure thing! It's been added.

Thank you, Caro!

Decided

Per today's offline discussion with @MNeisler, we have decided to limit the experiment to logged-in users at all wikis in the experiment.

Thinking

Limiting the experiment to logged in users will relieve of us the sampling constraint we'd need to take on for both groups (logged in and out) at en.wiki (0.1%) and all others (10%) had we included logged out users
While we've historically included both logged in and logged out users in Edit Check experiments so that we could make scaling decisions for both groups with the results from a single experiment, this experiment supports a KR (WE1.7) that is scoped to logged-in users only. Accordingly, we can treat the "deploy as default-on to logged out users" as a separate decision later on.

ppelberg assigned this task to medelius.May 8 2026, 5:26 AM

ppelberg updated the task description. (Show Details)

ppelberg moved this task from Blocked to Dev Doing on the Editing-team (Editing-Q4-27Apr-8May-2026) board.

medelius edited projects, added: Editing-team (Editing-Q4-11May-22May-2026); removed: Editing-team (Editing-Q4-27Apr-8May-2026).May 11 2026, 4:41 PM

medelius moved this task from Decision to be made to Ready for Sign off on the Editing-team (Editing-Q4-11May-22May-2026) board.

ppelberg closed this task as Resolved.May 12 2026, 4:57 AM

Quiddity updated the task description. (Show Details)Wed, May 13, 9:16 PM

Stang unsubscribed.Sun, May 17, 8:33 PM

ppelberg mentioned this in T421968: [Suggestion] Deploy "Add a link" suggestion as a default-on suggestion.Mon, May 18, 9:36 PM

ppelberg mentioned this in T426687: Retroactively revise start date of Suggestion Mode MVP experiment.Mon, May 18, 9:49 PM