OS filtering for origin trials is not applied consistently |
||||
Issue description
When configuring origin trials in RuntimeEnabledFeatures.json5, it is possible to limit the exposure to specific platforms, using "origin_trial_os".
However, this OS-filtering is not applied consistently, depending on how the trial is enabled.
This was observed for the InstalledApp feature, when setup as follows in RuntimeEnabledFeatures.json5:
{
name: "InstalledApp",
origin_trial_feature_name: "InstalledApp",
origin_trial_os: ["android"],
status: "experimental",
},
Problems with this on non-Android (Linux):
1. In production, it seems to completely ignore origin_trial_os. Using the test trial token, I can still use the origin trial feature on Linux if I have a valid token. This works on both the static <meta> tag in the HTML file, as well as inserting a <meta> tag dynamically.
2. In the layout test, it ignores the origin_trial_os but only when inserted dynamically. Running the test on Linux, the trial feature isn't available in the static <meta> tag test, but it does succeed when inserted dynamically.
See bug 635741 for the history behind the introduction of origin_trial_os.
,
Apr 3 2017
Non-OT APIs use SetRuntimeFeatureDefaultsForPlatform (in https://cs.chromium.org/chromium/src/content/child/runtime_features.cc?type=cs&l=26) to enable/disable features on specific platforms.
,
Apr 3 2017
Re comment #1, I see two scenarios for limiting OTs per platform: A) The feature will be available on all/most platforms, but the trial is only available on one/few platforms. B) The feature is not intended to work on all platforms (or not for a long), regardless of origin trials. Some kind of feature detection is necessary. For scenario (A), this was actually the case for Web Bluetooth. I don't recall the details, but the trial was limited to a few platforms (e.g. Android, ChromeOS), and the feature was experimental only on other platforms. For scenario (B), i.e. WebShare, as Matt said, we don't have a precedent that I know of for handling this in IDL. I can see scenario (A) occurring in the future, so I think we should keep origin_trial_os, and fix it to work reliably. Ideally, this would include guidance to make it easier for feature teams to test appropriately. For scenario (B), I think we need policy/guidance as to when it's appropriate to use origin_trial_os. I think that guidance should be broader than just origin trials. With standard approach for the web platform, we can reflect that in the guidance for origin_trial_os. However, we should not be afraid to have different behaviour for origin trials, if it better serves the need to collect data in the experiment. That is, it should be expected to have breaking behaviour between the origin trial and the shipping feature. Given that InstalledApp is not using origin_trial_os, we'll focus on fixing this for future experiments. The OT team will take on figuring out the appropriate policy both for the web platform in general, and origin trials specifically. Re comment #2, that is helpful, but I think just deals with the mechanics to flip flags. It doesn't seem to address the policy questions.
,
Apr 3 2017
I think Scenario A should probably not be happening. The origin trial should run on all the platforms that we intend to launch on, no? Was Bluetooth launched on other platforms without an OT? Or did it become "available" in the InstalledApp sense that it's present but always returns an empty list of devices? In either case, I think the OT should capture the same platforms as the final release. #2: Aha so that's how you do it.
,
Apr 19 2017
,
Feb 1 2018
This should still be addressed. Adding to the code health hotlist for Feature Control.
,
Aug 3
Tentatively assigning this to loonybear@ who also owns 711300.
,
Aug 10
Addressing this issue will require specific understanding of the OT framework. Even with a general solution for specifying platforms in bug 711300, it's not necessarily the same expertise fix the OT framework. |
||||
►
Sign in to add a comment |
||||
Comment 1 by mgiuca@chromium.org
, Apr 3 2017