Media Router integration tests are flaky |
|||||||||||||||||||||
Issue descriptionFindit has detected a flake at test MediaRouterIntegrationOneUABrowserTest.Basic. Culprit (100.0% confidence): https://chromium-review.googlesource.com/q/Ie5c2fdc9a838dec2906a73f7827fa6ceafdcce1b Regression range: https://crrev.com/a0a5c4046775f09dfff103972baf6ab12e33acc5..0d86ccd6bd11c42c0edb775c806b420f10193f66?pretty=fuller Analysis: https://findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVyvQELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCKGAWNocm9taXVtLm1lbW9yeS9MaW51eCBDaHJvbWl1bSBPUyBBU2FuIExTYW4gVGVzdHMgKDEpLzI2NTkwL2Jyb3dzZXJfdGVzdHMvVFdWa2FXRlNiM1YwWlhKSmJuUmxaM0poZEdsdmJrOXVaVlZCUW5KdmQzTmxjbFJsYzNRdVFtRnphV009DAsSE01hc3RlckZsYWtlQW5hbHlzaXMYAQw If this result was incorrect, apply the label Findit-Incorrect-Result, mark the bug as Untriaged and the component Tools>Test>Findit>Flakiness.
,
Mar 15 2018
Issue 822048 has been merged into this issue.
,
Mar 15 2018
Issue 822300 has been merged into this issue.
,
Mar 15 2018
Issue 822301 has been merged into this issue.
,
Mar 15 2018
Issue 822216 has been merged into this issue.
,
Mar 15 2018
Issue 822215 has been merged into this issue.
,
Mar 15 2018
Issue 822179 has been merged into this issue.
,
Mar 15 2018
Going to do another round of disabling, hopefully that will be the last.
,
Mar 15 2018
Issue 822337 has been merged into this issue.
,
Mar 15 2018
Ping since still getting alerts. For example on MediaRouterIntegrationOneUANoReceiverBrowserTest.Basic . Thanks for helping out!
,
Mar 15 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d42d1308cb2f7a1ca28af8f197524c80579cf0c6 commit d42d1308cb2f7a1ca28af8f197524c80579cf0c6 Author: mark a. foltz <mfoltz@chromium.org> Date: Thu Mar 15 20:33:14 2018 [Media Router] Disable flaky MR tests TBR=zhaobin Bug: 822231 Change-Id: Ibd6c654a1b5cdc2449ed091de1b1a64589df0746 Reviewed-on: https://chromium-review.googlesource.com/964729 Reviewed-by: mark a. foltz <mfoltz@chromium.org> Reviewed-by: Adam Parker <amp@chromium.org> Cr-Commit-Position: refs/heads/master@{#543482} [modify] https://crrev.com/d42d1308cb2f7a1ca28af8f197524c80579cf0c6/chrome/test/media_router/media_router_integration_browsertest.cc [modify] https://crrev.com/d42d1308cb2f7a1ca28af8f197524c80579cf0c6/chrome/test/media_router/media_router_integration_ui_browsertest.cc [modify] https://crrev.com/d42d1308cb2f7a1ca28af8f197524c80579cf0c6/chrome/test/media_router/media_router_one_ua_integration_browsertest.cc
,
Mar 15 2018
Issue 821717 has been merged into this issue.
,
Mar 15 2018
Let me know if you see any additional flakes after f0c6 landed.
,
Mar 15 2018
,
Mar 16 2018
The test failed on https://ci.chromium.org/buildbot/tryserver.chromium.linux/linux_chromium_cfi_rel_ng/426 which I think is past "f0c6" (which I suppose means d42d1308cb2f7a1ca28af8f197524c80579cf0c6 ? I think it might be more common to use the first few letters of the hash instead of the last few, and more than 4 is also needed to avoid collisions)
,
Mar 16 2018
Those are different tests than the one in the patch in C#11.
,
Mar 16 2018
Issue 822741 has been merged into this issue.
,
Mar 16 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3fa1b00388b667ff1c7df4a8a3e57e9a87a31f96 commit 3fa1b00388b667ff1c7df4a8a3e57e9a87a31f96 Author: mark a. foltz <mfoltz@chromium.org> Date: Fri Mar 16 19:23:00 2018 [Media Router] Disable two more flaky tests. Disables two more tests found to be flaky. TBR=zhaobin Bug: 822231 Change-Id: I0c6a36a7d448e942ffd191ea12471f33c2729c81 Reviewed-on: https://chromium-review.googlesource.com/966983 Reviewed-by: mark a. foltz <mfoltz@chromium.org> Commit-Queue: mark a. foltz <mfoltz@chromium.org> Cr-Commit-Position: refs/heads/master@{#543780} [modify] https://crrev.com/3fa1b00388b667ff1c7df4a8a3e57e9a87a31f96/chrome/test/media_router/media_router_integration_browsertest.cc [modify] https://crrev.com/3fa1b00388b667ff1c7df4a8a3e57e9a87a31f96/chrome/test/media_router/media_router_one_ua_integration_browsertest.cc
,
Mar 16 2018
,
Mar 16 2018
,
Mar 17 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7bd70e542e44ee67671cb5a64d6751ed1ed7ed03 commit 7bd70e542e44ee67671cb5a64d6751ed1ed7ed03 Author: Karan Bhatia <karandeepb@chromium.org> Date: Sat Mar 17 00:04:19 2018 Disable flaky MediaRouterIntegrationOneUANoReceiverBrowserTest.ReconnectSesion BUG=822231 TBR=mfoltz@chromium.org Change-Id: I22d30d5c0c6ba309492697605077a94ba994148c Reviewed-on: https://chromium-review.googlesource.com/967411 Commit-Queue: Karan Bhatia <karandeepb@chromium.org> Reviewed-by: Karan Bhatia <karandeepb@chromium.org> Cr-Commit-Position: refs/heads/master@{#543876} [modify] https://crrev.com/7bd70e542e44ee67671cb5a64d6751ed1ed7ed03/chrome/test/media_router/media_router_one_ua_integration_browsertest.cc
,
Mar 17 2018
,
Mar 17 2018
Detected 3 new flakes for test/step "MediaRouterIntegrationOneUANoReceiverBrowserTest.ReconnectSession". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyTAsSBUZsYWtlIkFNZWRpYVJvdXRlckludGVncmF0aW9uT25lVUFOb1JlY2VpdmVyQnJvd3NlclRlc3QuUmVjb25uZWN0U2Vzc2lvbgw. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Mar 19 2018
The test in c#23 should be disabled.
,
Mar 19 2018
Here is a summary of what's been disabled. I will start a doc with the failure output before it's lost. It's going to take some effort set aside in Q2 to fix all of these, so setting a future milestone. MediaRouterIntegrationBrowserTest::MANUAL_OpenLocalMediaFileFullscreen MediaRouterIntegrationBrowserTest::MANUAL_OpenLocalMediaFileInCurrentTab MediaRouterIntegrationBrowserTest::MANUAL_ReconnectSession MediaRouterIntegrationBrowserTest::MANUAL_Dialog_RouteCreationTimedOut MediaRouterIntegrationIncognitoBrowserTest::MANUAL_ReconnectSession MediaRouterIntegrationOneUABrowserTest::MANUAL_Basic MediaRouterIntegrationOneUABrowserTest::MANUAL_SendAndOnMessage MediaRouterIntegrationOneUABrowserTest::MANUAL_ReceiverCloseConnection MediaRouterIntegrationOneUABrowserTest::MANUAL_ReconnectSession MediaRouterIntegrationOneUABrowserTest::MANUAL_ReconnectSessionSameTab MediaRouterIntegrationOneUANoReceiverBrowserTest::MANUAL_Basic MediaRouterIntegrationOneUANoReceiverBrowserTest::MANUAL_Fail_SendMessage MediaRouterIntegrationOneUANoReceiverBrowserTest::MANUAL_ReconnectSession
,
Mar 19 2018
I dumped a bunch of output from the failed builders into this doc (tried to group by root cause). It seems like we can fix most of the Presentation API tests by handling pending Mojo callbacks better and working around a ChromeOS DCHECK. https://docs.google.com/document/d/1WS9YYlC-sgjdpFS1f8NtCWMrzUhEBALghL1bClwMxFc/edit?usp=sharing
,
Mar 19 2018
,
Mar 19 2018
,
Mar 20 2018
Findit identified the culprit r543005 with confidence 100.0% in the config "chromium.memory / Linux Chromium OS ASan LSan Tests (1)" based on the flakiness trend: https://findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVy3QELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCKmAWNocm9taXVtLm1lbW9yeS9MaW51eCBDaHJvbWl1bSBPUyBBU2FuIExTYW4gVGVzdHMgKDEpLzI2NTczL211c19icm93c2VyX3Rlc3RzL1RXVmthV0ZTYjNWMFpYSkpiblJsWjNKaGRHbHZiazl1WlZWQlRtOVNaV05sYVhabGNrSnliM2R6WlhKVVpYTjBMa1poYVd4ZlUyVnVaRTFsYzNOaFoyVT0MCxITTWFzdGVyRmxha2VBbmFseXNpcxgDDA Automatically posted by the findit-for-me app (https://goo.gl/Ot9f7N). Flake Analyzer is in alpha version. Feedback is welcome using component Tools>Test>FindIt>Flakiness !
,
Mar 21 2018
,
Mar 22 2018
Issue 824889 has been merged into this issue.
,
Mar 30 2018
Issue 818766 has been merged into this issue.
,
Mar 30 2018
,
Mar 30 2018
,
Apr 2 2018
Issue 828039 has been merged into this issue.
,
Apr 4 2018
Issue 828798 has been merged into this issue.
,
Apr 4 2018
WebContentsImplBrowserTest.PausePageScheduledTasks is Flaky
,
Apr 5 2018
,
Apr 6 2018
Findit identified the culprit r547403 with confidence 100.0% in the config "chromium.memory / Linux Chromium OS ASan LSan Tests (1)" based on the flakiness trend: https://findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVyzAELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCKVAWNocm9taXVtLm1lbW9yeS9MaW51eCBDaHJvbWl1bSBPUyBBU2FuIExTYW4gVGVzdHMgKDEpLzI2ODc2L2NvbnRlbnRfYnJvd3NlcnRlc3RzL1YyVmlRMjl1ZEdWdWRITkpiWEJzUW5KdmQzTmxjbFJsYzNRdVVHRjFjMlZRWVdkbFUyTm9aV1IxYkdWa1ZHRnphM009DAsSE01hc3RlckZsYWtlQW5hbHlzaXMYAgw Automatically posted by the findit-for-me app (https://goo.gl/Ot9f7N). Flake Analyzer is in alpha version. Feedback is welcome using component Tools>Test>FindIt>Flakiness !
,
Apr 19 2018
,
Apr 19 2018
,
Jun 18 2018
,
Jun 18 2018
Issue 840165 has been merged into this issue.
,
Jun 18 2018
,
Jul 24
,
Aug 2
,
Aug 2
,
Aug 14
Issue 873912 has been merged into this issue.
,
Sep 4
,
Oct 2
,
Nov 7
,
Jan 7
We are now done with converting the integration/e2e tests to use the Views UI instead of WebUI (issue 900248). There still are 7 flaky test cases that are either disabled or run only in our private waterfall: MediaRouterE2EBrowserTest.DISABLED_OpenLocalMediaFileFullscreen (issue 903016) MediaRouterIntegrationBrowserTest.DISABLED_OpenLocalMediaFileFullscreen (issue 903016) MediaRouterIntegrationBrowserTest.MANUAL_Fail_NoProvider MediaRouterIntegrationBrowserTest.MANUAL_Dialog_Basic MediaRouterIntegrationBrowserTest.MANUAL_Dialog_RouteCreationTimedOut MediaRouterIntegrationOneUABrowserTest.MANUAL_SendAndOnMessage MediaRouterIntegrationOneUABrowserTest.MANUAL_ReceiverCloseConnection |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by rogerta@chromium.org
, Mar 15 2018Owner: mfo...@chromium.org
Status: Assigned (was: Available)