⚓ T308141 Deploy "add a link" to 15th round of wikis

	Subject	Repo	Branch	Lines +/-
	GrowthExperiments: enable frontend for 15th round of wikis	operations/mediawiki-config	master	+21 -0
	GrowthExperiments: enable AddLink backend 15th round of wikis	operations/mediawiki-config	master	+21 -0

Status	Assigned	Task
Open	lbowmaker	T307881 Scaling of link suggestions service
In Progress	Trizek-WMF	T304110 [EPIC] Deploy "add a link" to all Wikipedias
Resolved	Sgs	T308141 Deploy "add a link" to 15th round of wikis

Trizek-WMF created this task.May 11 2022, 2:28 PM

Restricted Application added subscribers: jhsoby-WMNO, Aklapper. · View Herald TranscriptMay 11 2022, 2:28 PM

Trizek-WMF added a parent task: T304110: [EPIC] Deploy "add a link" to all Wikipedias.May 11 2022, 2:33 PM

kostajh moved this task from Inbox to Triaged on the Growth-Team board.May 12 2022, 10:28 AM

Trizek-WMF moved this task from To Triage to Not ready to announce on the User-notice board.May 16 2022, 12:30 PM

kevinbazira claimed this task.Mar 29 2023, 12:01 PM

kevinbazira added a project: Machine-Learning-Team.

kevinbazira moved this task from Unsorted to In Progress on the Machine-Learning-Team board.

22/22 models were trained successfully in the 15th round of wikis.

Model evaluation has been completed and below are the backtesting results:

	Precision@0.5	Recall@0.5
quwiki	0.75	0.16
rmwiki	0.84	0.65
rmywiki	0.89	0.76
rnwiki	0.98	0.67
roa_rupwiki	0.76	0.57
roa_tarawiki	0.97	0.76
ruewiki	0.84	0.57
rwwiki	0.83	0.65
sawiki	0.75	0.13
sahwiki	0.77	0.43
satwiki	0.76	0.52
scwiki	0.80	0.54
scnwiki	0.91	0.56
scowiki	0.84	0.54
sdwiki	0.75	0.45
sewiki	0.95	0.69
sgwiki	1.00	0.89
shwiki	0.90	0.51
shnwiki	0.50	0.02
siwiki	0.74	0.13
skwiki	0.91	0.59
slwiki	0.83	0.53

CCing @MGerlach, in case he would like to add comments on the backtesting evaluation.

The conclusion on the backtesting results is that most of the languages look fine besides:

shnwiki has a low precision (0.50) and recall (0.02)
siwiki's precision (0.74) is slightly lower than the recommended one (0.75)

Talked to @MGerlach about these results and agreed that siwiki should be published but shnwiki shouldn't.

@kostajh, we published datasets for all 21/22 models that passed the evaluation in this round.

elukey reassigned this task from kevinbazira to kostajh.May 15 2023, 3:04 PM

elukey moved this task from In Progress to Watching on the Machine-Learning-Team board.

elukey added a subscriber: kevinbazira.

kevinbazira mentioned this in T309263: Support languages whose add-a-link models were not published.Jul 4 2023, 9:44 AM

kevinbazira mentioned this in T343374: Create a single table with evaluation metrics from all trained add-a-link models.Aug 2 2023, 6:10 PM

I ran this script for adding the link-recommendation task type and populating the excluded sections entries:

PHAB=T308141
for WIKI in quwiki rmwiki rmywiki rnwiki roa_rupwiki roa_tarawiki ruewiki rwwiki sawiki sahwiki satwiki scwiki scnwiki scowiki sdwiki sewiki sgwiki shwiki siwiki skwiki slwiki; do
    ORIGIN=`mwscript getConfiguration.php $WIKI --settings 'wgCanonicalServer' --format json | jq --raw-output '.wgCanonicalServer'`
    mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \
            --page MediaWiki:NewcomerTasks.json \
            --create-only \
            --json \
            --summary "Growth features configuration boilerplate ([[phab:$PHAB]])" \
            link-recommendation \
            '{ "type": "link-recommendation", "group": "easy" }'
    jq "select(.wiki==\"$WIKI\" and .probability > 0.25) | .section" wiki_sections.jsonl \
        | jq --slurp --compact-output "unique" \
        | mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \
            --page MediaWiki:NewcomerTasks.json \
            --json \
            --summary "machine-generated configuration for excluding sections from link recommendations ([[phab:$PHAB]]), feel free to improve" \
            link-recommendation.excludedSections \
            "`cat`"
    echo "$ORIGIN/wiki/MediaWiki:NewcomerTasks.json"
    echo "$ORIGIN/w/index.php?title=MediaWiki:NewcomerTasks.json&diff=next"
    echo "Press <Enter> to continue"
    read # give time for manual verification
done

Note that the script didn't populate excludedSections for rnwiki, roa_rupwiki, roa_tarawiki and sgwiki because these were not present in the wiki_sections.jsonl, see T345562.

Change 964949 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: enable AddLink backend 15th round of wikis

https://gerrit.wikimedia.org/r/964949

gerritbot added a project: Patch-For-Review.Oct 10 2023, 4:33 PM

Sgs updated the task description. (Show Details)Oct 10 2023, 4:38 PM

Change 964949 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: enable AddLink backend 15th round of wikis

https://gerrit.wikimedia.org/r/964949

Mentioned in SAL (#wikimedia-operations) [2023-10-11T07:27:16Z] <sgimeno@deploy2002> Started scap: Backport for [[gerrit:964949|GrowthExperiments: enable AddLink backend 15th round of wikis (T308141)]]

Mentioned in SAL (#wikimedia-operations) [2023-10-11T07:28:35Z] <sgimeno@deploy2002> sgimeno: Backport for [[gerrit:964949|GrowthExperiments: enable AddLink backend 15th round of wikis (T308141)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Maintenance_bot removed a project: Patch-For-Review.Oct 11 2023, 7:31 AM

Mentioned in SAL (#wikimedia-operations) [2023-10-11T07:35:01Z] <sgimeno@deploy2002> Finished scap: Backport for [[gerrit:964949|GrowthExperiments: enable AddLink backend 15th round of wikis (T308141)]] (duration: 07m 45s)

Sgs changed the task status from Open to In Progress.Oct 11 2023, 4:45 PM

Sgs moved this task from Triaged to Sprint 0 (Growth Team) on the Growth-Team board.

Sgs edited projects, added Growth-Team (Sprint 0 (Growth Team)); removed Growth-Team.

Sgs moved this task from Incoming to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.

KStoller-WMF edited projects, added Growth-Team (Sprint 1 (Growth Team)); removed Growth-Team (Sprint 0 (Growth Team)).Oct 18 2023, 9:17 PM

KStoller-WMF moved this task from Incoming to Ready on the Growth-Team (Sprint 1 (Growth Team)) board.

Sgs moved this task from Ready to Doing on the Growth-Team (Sprint 1 (Growth Team)) board.Oct 30 2023, 4:04 PM

KStoller-WMF edited projects, added Growth-Team (Sprint 2 (Growth Team)); removed Growth-Team (Sprint 1 (Growth Team)).Oct 30 2023, 6:58 PM

Sgs moved this task from Incoming to Doing on the Growth-Team (Sprint 2 (Growth Team)) board.Oct 31 2023, 4:40 PM

KStoller-WMF triaged this task as Medium priority.Nov 7 2023, 6:26 PM

Trizek-WMF set Due Date to Nov 22 2023, 5:00 PM.Nov 8 2023, 5:22 PM

KStoller-WMF edited projects, added Growth-Team (Sprint 3 (Growth Team)); removed Growth-Team (Sprint 2 (Growth Team)).Nov 14 2023, 2:46 PM

Sgs moved this task from Incoming to Doing on the Growth-Team (Sprint 3 (Growth Team)) board.Nov 14 2023, 3:41 PM

Sgs set the point value for this task to 1.Nov 14 2023, 4:46 PM

All wikis present results now.

Notes

rnwiki and sgwiki present a very low number of suggestions, 4 and 1 respectively, probably due to the scarce amount of articles (639 and 318). I think we can enable their frontends anyways wishing for the model to generate more results as articles grow. cc @Trizek-WMF

rmwiki and shwiki were not present in wikis.txt file leading to 400 errors on the API requests with this domains as parameters. Probably due to the fact the datasets were published before we fixed the publish-dataset.sh script in T340944. I've added them manually cc @kevinbazira

Sgs updated the task description. (Show Details)Nov 15 2023, 2:27 PM

In T308141#9334257, @Sgs wrote:

I think we can enable their frontends anyways wishing for the model to generate more results as articles grow. cc @Trizek-WMF

Agreed!

Just to confirm for Tech News purposes: Is this releasing next week even though there isn't a deployment train next week (20 Nov)? Or does the Tech News entry need to be moved to the following week (27 Nov) instead?
(The current entry says "Starting on Wednesday, a new set of Wikipedias will get "Add a link" [...]" (meaning 22 Nov). Thanks!

In T308141#9335723, @Quiddity wrote:

Just to confirm for Tech News purposes: Is this releasing next week even though there isn't a deployment train next week (20 Nov)? Or does the Tech News entry need to be moved to the following week (27 Nov) instead?
(The current entry says "Starting on Wednesday, a new set of Wikipedias will get "Add a link" [...]" (meaning 22 Nov). Thanks!

Thanks for your question. All required code for Add a link tasks is already in all production wikis. We will enable the new set of Wikipedias by performing a config backport in one of the available windows, probably at 14:00 UTC.

And (IIRC) when this code will be backported, activation is a config change, which is not impacted by the absence of train.

Quiddity moved this task from In current Tech/News draft to Already announced/Archive on the User-notice board.Nov 23 2023, 9:31 PM

Change 977644 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: enable frontend for 15th round of wikis

https://gerrit.wikimedia.org/r/977644

gerritbot added a project: Patch-For-Review.Nov 27 2023, 12:12 PM

Change 977644 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: enable frontend for 15th round of wikis

https://gerrit.wikimedia.org/r/977644

Mentioned in SAL (#wikimedia-operations) [2023-11-27T14:10:22Z] <urbanecm@deploy2002> Started scap: Backport for [[gerrit:977644|GrowthExperiments: enable frontend for 15th round of wikis (T308141)]], [[gerrit:975378|zghwiki: add timezone, wgSitename (T350241)]], [[gerrit:975376|bbcwiki: add timezone, wgSitename (T350373)]]

Stashbot mentioned this in T350241: Post-creation work for zghwiki.Nov 27 2023, 2:10 PM

Stashbot mentioned this in T350373: Post-creation work for bbcwiki.

Maintenance_bot removed a project: Patch-For-Review.Nov 27 2023, 2:10 PM

Mentioned in SAL (#wikimedia-operations) [2023-11-27T14:11:36Z] <urbanecm@deploy2002> sgimeno and anzx and urbanecm: Backport for [[gerrit:977644|GrowthExperiments: enable frontend for 15th round of wikis (T308141)]], [[gerrit:975378|zghwiki: add timezone, wgSitename (T350241)]], [[gerrit:975376|bbcwiki: add timezone, wgSitename (T350373)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-11-27T14:21:45Z] <urbanecm@deploy2002> Finished scap: Backport for [[gerrit:977644|GrowthExperiments: enable frontend for 15th round of wikis (T308141)]], [[gerrit:975378|zghwiki: add timezone, wgSitename (T350241)]], [[gerrit:975376|bbcwiki: add timezone, wgSitename (T350373)]] (duration: 11m 23s)

Sgs moved this task from Doing to Test in Production on the Growth-Team (Sprint 3 (Growth Team)) board.Nov 27 2023, 3:00 PM

Etonkovidova updated the task description. (Show Details)Nov 28 2023, 11:00 PM

Checked some wikis from the list - all looks as expected. Leaving the task in the Test in Production columns to monitor it during this week.

Sgs updated the task description. (Show Details)Nov 29 2023, 12:59 PM

calbon moved this task from Watching to 2023-2024 Q3 Done on the Machine-Learning-Team board.Nov 29 2023, 2:18 PM

Etonkovidova closed this task as Resolved.Dec 5 2023, 5:37 PM

Maintenance_bot edited projects, added User-notice-archive; removed User-notice.Dec 15 2023, 6:30 PM

Deploy "add a link" to 15th round of wikis
Closed, ResolvedPublic1 Estimated Story Points

Actions

Description

Details

Related Objects

Search...

Event Timeline

Deploy "add a link" to 15th round of wikis Closed, ResolvedPublic1 Estimated Story Points Actions

Description

Details

Related Objects Search...

Event Timeline

Deploy "add a link" to 15th round of wikis
Closed, ResolvedPublic1 Estimated Story Points

Actions

Related Objects

Search...