| ngkountas | |
| Feb 8 2022, 8:53 AM |
| F35379006: frequent-languages.webm | |
| Aug 2 2022, 7:42 AM |
| F35378993: content-language-selector.webm | |
| Aug 2 2022, 7:21 AM |
| F35377696: recent-translation.webm | |
| Aug 1 2022, 6:55 PM |
| F34961549: image.png | |
| Feb 22 2022, 5:23 AM |
Some new entrypoints have been deployed during the recent months and some are still being developed (T286641). In order to track how effective these new entrypoints are, we need to instrument them.
This task is intended to both support this instrumentation within the SX application and keep all the related information in a single place.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T286641 Entry points to improve discoverability of Section Translation | |||
| Resolved | ngkountas | T301222 Instrumentation of new SX entrypoints | |||
| Resolved | PRODUCTION ERROR | None | T297527 eventgate_validation_error: ''should have required property 'content_translation_session_position'" | ||
| Resolved | MNeisler | T287403 Update content_translation_event schema to accomodate new Content Translation entry points |
The event names for each entrypoint should be the following:
Change 760890 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):
[mediawiki/extensions/ContentTranslation@master] SX: Instrumentation support for all entrypoints
Change 760890 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] SX: Instrumentation support for all entrypoints
I see validation errors from eventlogging
I am using mw cli and it has eventlogging support.
eventlogging_1 | {
eventlogging_1 | "$schema": "/analytics/mediawiki/content_translation_event/1.0.0",
eventlogging_1 | "translation_type": "section",
eventlogging_1 | "wiki_db": "es",
eventlogging_1 | "access_method": "desktop",
eventlogging_1 | "user_name": "Admin",
eventlogging_1 | "user_is_anonymous": false,
eventlogging_1 | "content_translation_session_id": "cx_sx_c18ed0109578987ac734_desktop_es",
eventlogging_1 | "event_type": "dashboard_open",
eventlogging_1 | "event_source": "direct",
eventlogging_1 | "content_translation_session_position": 0,
eventlogging_1 | "user_global_edit_count": 152,
eventlogging_1 | "user_global_edit_count_bucket": "100-999 edits",
eventlogging_1 | "meta": {
eventlogging_1 | "stream": "mediawiki.content_translation_event",
eventlogging_1 | "domain": "es.mediawiki.mwdd.localhost",
eventlogging_1 | "id": "eb1853c9-3930-4de5-9091-da604920d88f",
eventlogging_1 | "dt": "2022-02-22T05:10:02.494Z",
eventlogging_1 | "request_id": "b0ad2ad0-939d-11ec-902c-edb38b77fe06"
eventlogging_1 | },
eventlogging_1 | "dt": "2022-02-22T05:10:01.491Z",
eventlogging_1 | "http": {
eventlogging_1 | "request_headers": {
eventlogging_1 | "user-agent": "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Mobile Safari/537.36"
eventlogging_1 | }
eventlogging_1 | }
eventlogging_1 | }
eventlogging_1 | {"name":"eventgate-wikimedia","hostname":"766f50909c18","pid":1,"level":40,"err":{"message":"Event failed validation with schema at /analytics/mediawiki/content_translation_event/1.0.0","name":"ValidationError","stack":"ValidationError: Event failed validation with schema at /analytics/mediawiki/content_translation_event/1.0.0\n at validatorFor.then (/srv/service/node_modules/eventgate/lib/EventValidator.js:285:23)\n at tryCatcher (/srv/service/node_modules/bluebird/js/release/util.js:16:23)\n at Promise._settlePromiseFromHandler (/srv/service/node_modules/bluebird/js/release/promise.js:547:31)\n at Promise._settlePromise (/srv/service/node_modules/bluebird/js/release/promise.js:604:18)\n at Promise._settlePromiseCtx (/srv/service/node_modules/bluebird/js/release/promise.js:641:10)\n at_drainQueueStep (/srv/service/node_modules/bluebird/js/release/async.js:97:12)\n at _drainQueue (/srv/service/node_modules/bluebird/js/release/async.js:86:9)\n at Async._drainQueues (/srv/service/node_modules/bluebird/js/release/async.js:102:5)\n at Immediate.Async.drainQueues [as _onImmediate] (/srv/service/node_modules/bluebird/js/release/async.js:15:14)\n at runCallback (timers.js:705:18)\n at tryOnImmediate (timers.js:676:5)\n at processImmediate (timers.js:658:5)","errors":[{"keyword":"required","dataPath":"","schemaPath":"#/required","params":{"missingProperty":"content_translation_session_position"},"message":"should have required property 'content_translation_session_position'"}],"errorsText":"'' should have required property 'content_translation_session_position'"},"msg":"event failed schema validation.","time":"2022-02-22T05:16:52.785Z","v":0}
This is the payload:
$schema: "/analytics/mediawiki/content_translation_event/1.0.0"
access_method: "desktop"
content_translation_session_id: "cx_sx_c18ed0109578987ac734_desktop_es"
dt: "2022-02-22T05:16:51.779Z"
event_source: "suggestion_no_seed"
event_type: "dashboard_translation_start"
meta: {stream: "mediawiki.content_translation_event", domain: "es.mediawiki.mwdd.localhost"}
domain: "es.mediawiki.mwdd.localhost"
stream: "mediawiki.content_translation_event"
translation_type: "section"
user_global_edit_count: 152
user_global_edit_count_bucket: "100-999 edits"
user_is_anonymous: false
user_name: "Admin"
wiki_db: "es"
The same error has been reported before: T297527: eventgate_validation_error: ''should have required property 'content_translation_session_position'"
The instrumentation of the entrypoints basically refers to properly logging the dashboard_open event, which is logged with content_translation_session_position property always set to 0. This is fine for the instrumentation of the entrypoints, since the dashboard is always the first step inside SX application for a user coming from an entrypoint. The errors mentioned above refer to other events (e.g. dashboard_translation_start) where content_translation_session_position property is missing. For these reasons I think that this task can be closed.
For reference these are the following entrypoints currently supported with their respective event_source property:
| Entrypoint name | Event source |
| mflanguagesearcher | content_language_selector |
| mfrecenttranslation | recent_translation |
| mfrecentedit | recent_edit |
| mffrequentlanguages | frequent_languages |
This will be moved to done column as well as the effort is the same as task T295756. See the mentioned task for details
Recent translation entrypoint is instrumented properly, as shown in the screencast below. As we can observe, the campaign URL parameter is properly set to mfrecenttranslation
and the event source of dashboard_open and dashboard_translation_start events, is properly set to recent_translation.
The mobile language selector entrypoint is instrumented properly. As we can check in the screencast below, the campaign URL parameter is correctly set to mflanguagesearcher and the dashboard_open and dashboard_translation_start events are properly logged with the event source set to content_language_selector.
The missing frequent languages entrypoint is NOT properly instrumented as displayed in the following screencast. The campaign URL parameter is wrongfully set to "mfmissinglanguages" instead of mffrequentlanguages and as a result the dashboard_open and dashboard_translation_start events are logged with the generic direct_preselect event source instead of the frequent_languages event source. This will be fixed in a upcoming patch.
Change 819487 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):
[mediawiki/extensions/ContentTranslation@master] Fix campaign URL param for mffrequentlanguages entrypoint
The instrumentation of the entrypoint when creating a new article on mobile is currently missing and this issue will fixed in a following patch.
Change 819488 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):
[mediawiki/extensions/ContentTranslation@master] SX: Support instrumentation of newbytranslationmobile entrypoint
Following the latest patches and after the tests, the status of the instrumentation of the entrypoints is displayed in the following table:
| Entrypoint name | Event source | Status |
| mflanguagesearcher | content_language_selector | Done |
| mfrecenttranslation | recent_translation | Done |
| mfrecentedit | recent_edit | Entrypoint not yet deployed |
| mffrequentlanguages | frequent_languages | Entrypoint not properly instrumented (wrong URL campaign param) |
| newbytranslationmobile | creation_invite | Entrypoint instrumentation not supported |
Except for the recent edit entrypoint that has not yet been deployed, fixes have been submitted for the instrumentation of the other two pending entrypoints.
Change 819487 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] Fix campaign URL param for mffrequentlanguages entrypoint
Change 819488 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] SX: Support instrumentation of newbytranslationmobile entrypoint
Based on input from @MNeisler, it seems that it may be better to name entrypoins without referring to the platform. If the same entry point is available on both desktop and mobile, it can use the same name (since it is the same idea connected to the same activity) and other data cana be used in the analysis to separate the data per platform if needed.
So I guess that it is better to name this event as "newbytranslation" instead of "newbytranslationmobile".
@ngkountas
For this entry point, I believe we currently have a similar event_source already identified in content_translation_event for the desktop entry point created in T216500:
event_source = 'invite_new_article_creation' as a possible source for the dashboard_open event
As @Pginer-WMF mentioned above, I'd recommend we apply the same event source to the equivalent mobile entry point if possible and use the access_method field to distinguish between the two types of entry points. This would simplify analytics and align with how the other entry points have been instrumented (for example, event_source = content_language_selector was applied to both the mobile and desktop entry points).
Note: I will update the content_translation_event schema documentation to reflect these new event_source values as part of T287403.
Change 823521 had a related patch set uploaded (by Santhosh; author: Santhosh):
[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20220816
Change 823521 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20220816
Change 825717 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):
[mediawiki/extensions/ContentTranslation@master] Fix event source for "newbytranslationmobile" entrypoint
@MNeisler thank you for your input! I fixed the related event source to match the existing one
Change 825717 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] Fix event source for "newbytranslationmobile" entrypoint
Change 826546 had a related patch set uploaded (by Nik Gkountas; author: Santhosh):
[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20220825
Change 826341 had a related patch set uploaded (by KartikMistry; author: Santhosh):
[mediawiki/extensions/ContentTranslation@wmf/1.39.0-wmf.26] CX3 Build 0.2.0+20220825
Change 826546 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] CX3 Build 0.2.0+20220825
Change 826341 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@wmf/1.39.0-wmf.26] CX3 Build 0.2.0+20220825
Mentioned in SAL (#wikimedia-operations) [2022-08-25T13:32:13Z] <hashar@deploy1002> Started scap: Backport for [[gerrit:826341|CX3 Build 0.2.0+20220825 (T309986 T301222)]]
Mentioned in SAL (#wikimedia-operations) [2022-08-25T13:57:10Z] <hashar@deploy1002> Finished scap: Backport for [[gerrit:826341|CX3 Build 0.2.0+20220825 (T309986 T301222)]] (duration: 24m 56s)
Based on the ocnversation above it seems that the event source has been set to " 'invite_new_article_creation", but it is unclear which i the final entry point name for this one. Can anyone confirm the name to keep the documentation in sync?
Thanks!
@Pginer-WMF I updated the documentation setting the name of the entrypoint to newbytranslation. The name doesn't mean much in our code in any case, it's more a way to reference the entrypoint.