New issue
Advanced search Search tips

Issue 883676 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner: ----
Closed: Sep 25
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

rebaseline-cl created bizarre baseline changes due to platform differences in global-interface-listing-expected.txt

Project Member Reported by foolip@chromium.org, Sep 13

Issue description

In https://chromium-review.googlesource.com/c/chromium/src/+/1223149/1 (PS1) I updated TestExpectations and ran `./third_party/blink/tools/blink_tool.py rebaseline-cl`

The bots which ran can be seen in that CL, 9 different ones including linux-blink-rel, win7-blink-rel, and others.

After they finished, I ran `./third_party/blink/tools/blink_tool.py rebaseline-cl`. The exact output of that was never uploaded to review 1223149, but I uploaded it to a separate CL now to show it:
https://chromium-review.googlesource.com/c/chromium/src/+/1224055

There are a number of added baselines among the many modified ones:
platform/fuchsia/http/tests/worklet/webexposed/global-interface-listing-paint-worklet-expected.txt
platform/mac-retina/webexposed/global-interface-listing-expected.txt
platform/mac/webexposed/global-interface-listing-expected.txt
platform/win/webexposed/global-interface-listing-expected.txt
virtual/origin-trials-runtimeflags-disabled/http/tests/origin_trials/webexposed/animationworklet-origin-trial-interfaces-worklet-scope-expected.txt
virtual/threaded/http/tests/worklet/webexposed/global-interface-listing-paint-worklet-expected.txt

It looks like the cause of the changes is that the bots had different global-interface-listing-expected.txt. I downloaded the "actual" from each bot and named it after the bot. There were exactly two forms. linux-blink-rel.txt matches what I got locally but came from the bot. mac10_11-blink-rel, mac10_12-blink-rel and mac10_13-blink-rel had different results from the others:

--- linux-blink-rel.txt	2018-09-13 11:46:50.993033176 +0200
+++ mac10_11-blink-rel.txt	2018-09-13 11:47:45.389376379 +0200
@@ -4092,6 +4092,7 @@
     getter root
     getter rootMargin
     getter thresholds
+    getter trackVisibility
     method constructor
     method disconnect
     method observe
@@ -4103,6 +4104,7 @@
     getter intersectionRatio
     getter intersectionRect
     getter isIntersecting
+    getter isVisible
     getter rootBounds
     getter target
     getter time

These two attributes have [RuntimeEnabled=IntersectionObserverV2] in the IDL, and I really don't understand how they could have been enabled on these bots. szager@, is it expected that this flag should be enabled on just some mac platforms? I can't find any code to suggest that.

robertma@, do you know of any other reason why the results could be different between bots? My only hypothesis is that the bots tested different commits and got different behavior, but comparing the commits for two of the bots with different results I don't spot any changes that seems like plausible culprits.

For now, logging this issue to see if anyone has ideas of what might have gone wrong.
 
linux-blink-rel.txt
259 KB View Download
mac10_10-blink-rel.txt
259 KB View Download
mac10_11-blink-rel.txt
259 KB View Download
mac10_12-blink-rel.txt
259 KB View Download
mac10_13-blink-rel.txt
259 KB View Download
mac10_13_retina-blink-rel.txt
259 KB View Download
win7-blink-rel.txt
259 KB View Download
win10-blink-rel.txt
259 KB View Download
It looks like this wasn't a transient problem, it also showed up in some of the mac failures in https://chromium-review.googlesource.com/c/chromium/src/+/1223149/3. But not exactly the same constellation. It's very strange. Trying again since I had to make some other unrelated changes.
Philip, I think your investigation is on the right path.

rebaseline-cl (or more specifically, optimize-baselines in this case) is WAI. The weird placement of baselines is due to mac10_11-blink-rel, mac10_12-blink-rel and mac10_13-blink-rel being different from others.

This is what the fallback tree looks like. I use "*" and "+" to represent the two different baselines, and "|" means this platform does not have a baseline (i.e. falls back to its parent).

* fuschia
* win
    | linux
+ mac
  * mac-retina
  | mac10.12
    | mac10.11
      * mac10.10
Yeah, I also suspsec that rebaseline-cl is correct here. Why some of the mac bots had different results is the mystery. szager@, any hypothesis of why that might be?
This might be related to an issue I brought up on blink-dev recently:

https://groups.google.com/a/chromium.org/d/msg/blink-dev/ivkXlBB3-XY/XkGdH8HiBwAJ

Basically, there are some tests in LayoutTests/intersection-observer/v2 that do this:

if (window.internals && internals.runtimeFlags) {
  internals.runtimeFlags.intersectionObserverV2Enabled = true;
}

If that js runs before the IntersectionObserver and IntersectionObserverEntry binding initialize code runs, then those extra attributes are added to the type templates.

Because the layout test runner re-uses content_shell instances to run multiple tests, it may be that the global namespace was polluted because one of the intersection-observer/v2 tests ran before the global-interface-listing test.
Here is the recent CL that added the tests:

https://chromium-review.googlesource.com/c/chromium/src/+/1212452
Oh, that would make sense. I thought that settings were reset between tests, but maybe not? Can you add cleanup code that resets the state back to the original?
Yeah, I'm working on just such a patch.
Project Member

Comment 8 by bugdroid1@chromium.org, Sep 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/be431e79c6ecdc6a097e90f9defa077a0af7e2af

commit be431e79c6ecdc6a097e90f9defa077a0af7e2af
Author: Stefan Zager <szager@chromium.org>
Date: Fri Sep 14 21:39:33 2018

[IntersectionObserverV2] Move all tests to virtual test suite

Previously, IOv2 layout tests would enable IOv2 in script via:

if (window.internals && internals.runtimeFlags) {
  internals.runtimeFlags.intersectionObserverV2Enabled true;
}

There are two problems with this approach:

1. By the time the above code runs, it may be too late: the prototype
for the IntersectionObserverEntry javascript type may already have been
created *without* the isVisible attribute. This was causing flaky test
failures, because the exact timing of when the prototype is created may
vary between runs.

2. Because the layout test runner re-uses content_shell processes for
multiple tests, the side effects from turning on the feature flag
affect tests that later use the same renderer process (see bug).

This CL also adds DisableIntersectionObserverV2ThrottleForTesting,
which eliminates the need to add a bunch of 100ms timers to the tests.
Slow tests are bad.

Finally, this CL reorders the handshake for cross-origin tests that
rely on postMessage, to ensure that both the embedding frame and the
child frame are properly initialized before starting the test
procedure.

BUG= 882280 , 883676 
R=chrishtr@chromium.org,foolip@chromium.org

Change-Id: I18e62c2cf099bf74ef4afc43f46a794c7a322582
Reviewed-on: https://chromium-review.googlesource.com/1220044
Commit-Queue: Stefan Zager <szager@chromium.org>
Reviewed-by: Chris Harrelson <chrishtr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#591483}
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/VirtualTestSuites
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/http/tests/intersection-observer/resources/v2-subframe.html
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/http/tests/intersection-observer/v2/cross-origin-effects.html
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/http/tests/intersection-observer/v2/cross-origin-occlusion.html
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/intersection-observer/resources/intersection-observer-test-utils.js
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/intersection-observer/v2/animated-occlusion.html
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/intersection-observer/v2/box-shadow.html
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/intersection-observer/v2/iframe-target.html
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/intersection-observer/v2/simple-effects.html
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/intersection-observer/v2/simple-occlusion.html
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/intersection-observer/v2/text-shadow.html
[rename] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/virtual/intersection-observer-v2/http/tests/intersection-observer/v2/README.txt
[copy] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/WebKit/LayoutTests/virtual/intersection-observer-v2/intersection-observer/v2/README.txt
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/blink/renderer/core/testing/internals.cc
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/blink/renderer/core/testing/internals.h
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/blink/renderer/core/testing/internals.idl
[modify] https://crrev.com/be431e79c6ecdc6a097e90f9defa077a0af7e2af/third_party/blink/renderer/platform/runtime_enabled_features.json5

Status: WontFix (was: Untriaged)
rebaseline-cl was actually working as expected here, closing.

Sign in to add a comment