New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 707232 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 3
Type: Bug

Blocked on:
issue 711300


Participants' hotlists:
Feature-Control-Code-Health


Sign in to add a comment

OS filtering for origin trials is not applied consistently

Project Member Reported by cha...@chromium.org, Mar 31 2017

Issue description

When configuring origin trials in RuntimeEnabledFeatures.json5, it is possible to limit the exposure to specific platforms, using "origin_trial_os".

However, this OS-filtering is not applied consistently, depending on how the trial is enabled.

This was observed for the InstalledApp feature, when setup as follows in RuntimeEnabledFeatures.json5:

    {
      name: "InstalledApp",
      origin_trial_feature_name: "InstalledApp",
      origin_trial_os: ["android"],
      status: "experimental",
    },

Problems with this on non-Android (Linux):
1.  In production, it seems to completely ignore origin_trial_os. Using the test trial token, I can still use the origin trial feature on Linux if I have a valid token. This works on both the static <meta> tag in the HTML file, as well as inserting a <meta> tag dynamically.
2. In the layout test, it ignores the origin_trial_os but only when inserted dynamically. Running the test on Linux, the trial feature isn't available in the static <meta> tag test, but it does succeed when inserted dynamically.


See  bug 635741  for the history behind the introduction of origin_trial_os.
 
Cc: sa...@chromium.org mgiuca@chromium.org
This came up in a discussion on experimentation-dev:
https://groups.google.com/a/chromium.org/d/msg/experimentation-dev/PVWly7Xvkig/0WF6pDr-BAAJ

A summary of my response there:
- origin_trial_os is *only* used by Web Share (which is going out of origin trial in M58, although the implementation is still there). So we could in theory remove it if we don't want it any more. But how to deal with future experiments that need to be available only on one platform?
- Maybe we should not have OTs limited per platform, because when they go to full release that will change the behaviour. The OT should work as close as possible to the full release
- Therefore maybe we should remove origin_trial_os but add a way to make an IDL feature only available on a certain OS.

Here are the IDL switches:
https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/WebKit/Source/bindings/IDLExtendedAttributes.md

There is no OS-specific or platform-specific check that I can find. I'm not sure we actually have a way to make a (non-OT) API available on one platform but not the others -- this seems to be unique to Web Share. This may indicate a deficiency in the Web Share API; it doesn't seem normal to have an API available on certain platforms and maybe we need a richer feature-detection scheme so we can have it exposed on all platforms but the caller can find out if it will work or not. This was discussed extensively here:
https://github.com/WICG/web-share/pull/8
and the conclusion reached at the time is that Chrome on Android vs Chrome on Desktop are like two different browsers; some features are supported on one but not the other, so it is fine to have it undefined on one but not the other.

Comment 2 by mek@chromium.org, Apr 3 2017

Non-OT APIs use SetRuntimeFeatureDefaultsForPlatform (in  https://cs.chromium.org/chromium/src/content/child/runtime_features.cc?type=cs&l=26) to enable/disable features on specific platforms.
Re comment #1, I see two scenarios for limiting OTs per platform:
A) The feature will be available on all/most platforms, but the trial is only available on one/few platforms.
B) The feature is not intended to work on all platforms (or not for a long), regardless of origin trials. Some kind of feature detection is necessary.

For scenario (A), this was actually the case for Web Bluetooth. I don't recall the details, but the trial was limited to a few platforms (e.g. Android, ChromeOS), and the feature was experimental only on other platforms.

For scenario (B), i.e. WebShare, as Matt said, we don't have a precedent that I know of for handling this in IDL.

I can see scenario (A) occurring in the future, so I think we should keep origin_trial_os, and fix it to work reliably. Ideally, this would include guidance to make it easier for feature teams to test appropriately.

For scenario (B), I think we need policy/guidance as to when it's appropriate to use origin_trial_os. I think that guidance  should be broader than just origin trials. With standard approach for the web platform, we can reflect that in the guidance for origin_trial_os. However, we should not be afraid to have different behaviour for origin trials, if it better serves the need to collect data in the experiment. That is, it should be expected to have breaking behaviour between the origin trial and the shipping feature.

Given that InstalledApp is not using origin_trial_os, we'll focus on fixing this for future experiments. The OT team will take on figuring out the appropriate policy both for the web platform in general, and origin trials specifically.

Re comment #2, that is helpful, but I think just deals with the mechanics to flip flags. It doesn't seem to address the policy questions.
I think Scenario A should probably not be happening. The origin trial should run on all the platforms that we intend to launch on, no? Was Bluetooth launched on other platforms without an OT? Or did it become "available" in the InstalledApp sense that it's present but always returns an empty list of devices?

In either case, I think the OT should capture the same platforms as the final release.

#2: Aha so that's how you do it.

Comment 5 by cha...@chromium.org, Apr 19 2017

Blockedon: 711300
This should still be addressed. Adding to the code health hotlist for Feature Control.
Owner: loonyb...@chromium.org
Status: Assigned (was: Available)
Tentatively assigning this to loonybear@ who also owns 711300.

Owner: ----
Status: Available (was: Assigned)
Addressing this issue will require specific understanding of the OT framework. Even with a general solution for specifying platforms in bug 711300, it's not necessarily the same expertise fix the OT framework.

Sign in to add a comment