external/wpt/accelerometer/Accelerometer.https.html is flaky (in a strange way) |
|||||||||||
Issue descriptionDuring a recent triage, I noticed this test behaving strangely. It's pass-fail flaky on Linux. An example of failures: https://test-results.appspot.com/data/layout_results/Linux_Tests/69207/layout-test-results/results.html It's pass-timeout flaky on Mac 10.12 & 10.13 (including our Retina, which is currently 10.12). Notably, often when it times out, the output is actually complete and correct, but there are extra output in the head and tail (https://test-results.appspot.com/data/layout_results/Mac10_12_Tests/12101/layout-test-results/results.html): Content-Type: text/plain ...(normal output)... #EOF #EOF This looks extremely suspicious to me. Something is perhaps wrong with content_shell test runner of r-w-t. And I can't think of any reason it only happens on Mac 10.12 & 10.13.
,
Apr 16 2018
,
Apr 17 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1d0417b9d838c2208369ea5e00e5c91b44f986f3 commit 1d0417b9d838c2208369ea5e00e5c91b44f986f3 Author: Robert Ma <robertma@chromium.org> Date: Mon Apr 16 03:07:25 2018 Clean up TestExpectaions 1. Some expectations can be consolidated (some tryjobs did not finish during the original imports, and expectations were added in later imports). 2. Filed a separate issue for wpt/accelerometer/Accelerometer.https.html TBR=cbiesinger Bug: 833215, 833093 Change-Id: If23c7188c0c74e525e362f8fd39dccc1c5f11aab Reviewed-on: https://chromium-review.googlesource.com/1013739 Reviewed-by: Robert Ma <robertma@chromium.org> Commit-Queue: Robert Ma <robertma@chromium.org> Cr-Commit-Position: refs/heads/master@{#550929} [modify] https://crrev.com/1d0417b9d838c2208369ea5e00e5c91b44f986f3/third_party/WebKit/LayoutTests/TestExpectations
,
Apr 17 2018
The Linux flake looks like the Mojo mocks are getting into a bad start throwing "Error: Call reset() before initialize()."
,
Apr 17 2018
,
Apr 18 2018
Issue 833087 has been merged into this issue.
,
May 10 2018
Detected 4 new flakes for test/step "external/wpt/accelerometer/Accelerometer.https.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyPgsSBUZsYWtlIjNleHRlcm5hbC93cHQvYWNjZWxlcm9tZXRlci9BY2NlbGVyb21ldGVyLmh0dHBzLmh0bWwM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
May 11 2018
,
May 11 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c0e275eb8c0a351b3baca3a44dd5b3c04d649af5 commit c0e275eb8c0a351b3baca3a44dd5b3c04d649af5 Author: Max Morin <maxmorin@chromium.org> Date: Fri May 11 09:51:13 2018 Add expectations for flaky Accelerometer.https.html Tbr: cbiesinger No-Try: true Bug: 833215 Change-Id: I3807d48df11e6b4416357896b7af4564b0ef7c6e Reviewed-on: https://chromium-review.googlesource.com/1055448 Reviewed-by: Max Morin <maxmorin@chromium.org> Commit-Queue: Max Morin <maxmorin@chromium.org> Cr-Commit-Position: refs/heads/master@{#557828} [modify] https://crrev.com/c0e275eb8c0a351b3baca3a44dd5b3c04d649af5/third_party/WebKit/LayoutTests/TestExpectations
,
May 15 2018
Adding an owner for the Web Tests infrastructure. I'm similarly confused why the runner is expecting the output to contain the Content-Type and #EOF lines on this, and only this, test file.
,
May 15 2018
The runner isn't *expecting* Content-Type of EOF. Here's the expectation (baseline), which seems correct to me: https://chromium.googlesource.com/chromium/src/+/master/third_party/WebKit/LayoutTests/external/wpt/accelerometer/Accelerometer.https-expected.txt Rather, it's the actual test output from content_shell that sometimes contains Content-Type and EOF, which I don't understand.
,
May 15 2018
Strange. The extra text is the header and footer that content_shell writes for the test. Given that the harness thinks that the test is timing out, it's as if it doesn't see or recognize the text, but is getting it anyway. This suggests that maybe something is out of sync in port/driver.py, but just for that one test. The tests before and after run fine. I'm not sure what would cause this.
,
May 15 2018
Sorry, I realized my comment in 11 might be a bit confusing w.r.t. my use of "expecting". content_shell prints the Content-Type header and the EOF footer, and run_web_tests understands them and strips them away. (https://cs.chromium.org/chromium/src/third_party/blink/tools/blinkpy/web_tests/port/driver.py?l=471&rcl=a81931feabf43701dd4d8732e08276e376fccb57) So in that sense, run_web_tests runner does expect to see the header and footer, but they should be handled within run_web_tests and opaque to Blink engineers, i.e. they shouldn't appear in the *-actual output or *-expected baselines.
,
Jun 1 2018
The assigned owner "mikhail.pozdnyakov@intel.com" is not able to receive e-mails, please re-triage. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jun 1 2018
,
Jul 4
https://chromium-review.googlesource.com/c/chromium/src/+/1126388 should take care of the test timing out problem; the extra output needs to be taken care of separately.
,
Jul 6
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2dc478a481992a16af70f690611e2b7e08740721 commit 2dc478a481992a16af70f690611e2b7e08740721 Author: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Date: Fri Jul 06 20:49:22 2018 Mark external/wpt/accelerometer/Accelerometer.https.html as a slow test This test often times out on the Mac bots, but it doesn't seem to be really failing but rather just being slow: it currently tests 3 different accelerometer sensors and for each of them we make several asynchronous calls and wait for many events, which can often cause it to take longer than 6 seconds to run (even on a release Linux build it never takes less than 5 seconds to run). Bug: 833215 Change-Id: I105a04e797f68b91739b631387d8e7d8da36e6da Reviewed-on: https://chromium-review.googlesource.com/1126388 Commit-Queue: Raphael Kubo da Costa (CET) <raphael.kubo.da.costa@intel.com> Reviewed-by: Reilly Grant <reillyg@chromium.org> Cr-Commit-Position: refs/heads/master@{#573064} [modify] https://crrev.com/2dc478a481992a16af70f690611e2b7e08740721/third_party/WebKit/LayoutTests/SlowTests [modify] https://crrev.com/2dc478a481992a16af70f690611e2b7e08740721/third_party/WebKit/LayoutTests/TestExpectations
,
Jul 6
Reassignig to Robert for the infra side; the test itself should no longer be flaky. |
|||||||||||
►
Sign in to add a comment |
|||||||||||
Comment 1 by bugdroid1@chromium.org
, Apr 16 2018