New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 693498 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Feb 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug



Sign in to add a comment

android_n5x_swarming_rel try bot is flaky

Project Member Reported by sdefresne@chromium.org, Feb 17 2017

Issue description

I'm trying to land https://codereview.chromium.org/2680403005/ that is iOS only change (only touches files built on iOS) and is code that is only used by the test suite that test it (part of a multi CL refactoring).

However, android_n5x_swarming_rel has been failing flakily multiple times, preventing me from landing the CL. This is not the first time this bot has been flaking and preventing landing iOS only changes.

See for example https://bugs.chromium.org/p/chromium/issues/detail?id=666293.
 
Components: Infra
I don't think we have any UI showing what's flaky on a specific builder, so it would be very hard to find an owner for this. From your CL looks like the following tests are failing:
 - org.chromium.android_webview.test.PostMessageTest#testCanUseReceivedAwMessagePortFromJS
 - benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.browse:news:cnn

The flakes for them have been recorded by chromium-try-flakes:
 - http://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyYgsSBUZsYWtlIldvcmcuY2hyb21pdW0uYW5kcm9pZF93ZWJ2aWV3LnRlc3QuUG9zdE1lc3NhZ2VUZXN0I3Rlc3RDYW5Vc2VSZWNlaXZlZEF3TWVzc2FnZVBvcnRGcm9tSlMM
 - http://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyeQsSBUZsYWtlIm5iZW5jaG1hcmtzLnN5c3RlbV9oZWFsdGhfc21va2VfdGVzdC5TeXN0ZW1IZWFsdGhCZW5jaG1hcmtTbW9rZVRlc3Quc3lzdGVtX2hlYWx0aC5tZW1vcnlfbW9iaWxlLmJyb3dzZTpuZXdzOmNubgw

First one is not flaky enough to have a bug filed for it yet, but the second is clearly very flaky. I am not sure why the bug wasn't filed yet, but will investigate.
After looking at the logs, I now see what happened. Yesterday, I've fixed a bug in a recipe uploading test results, which helped chromium-try-flakes to file bugs for specific flaky tests instead of filing bugs for entire step. This prevents the problem when nobody wanted to own bugs for entire flaky step since it would mean they would have to fix all flaky tests there.

Unfortunately, this also means that many bugs for flaky tests need to be filed instead of a few bugs that were used to track flaky steps containing them. As a result chromium-try-flakes quickly hit the limit of 10 bugs per day, which was set up to prevent chromium-try-flakes from filing hundreds of bugs when a single change breaks lots of tests, so this is WAI. Over the next coming days the backlog of bugs should be exhausted and chromium-try-flakes should be quickly filing bugs for flaky tests.

Meanwhile, I recommend reaching out to the sheriffs or disabling the test yourself.
Labels: Sheriff-Chromium
I'm going to add Sheriff-Chromium label, so that this bug shows up on SoM.
Components: -Infra -Infra>Flakiness Tests>Flaky
This is not infra flakiness, so removing corresponding components and adding Tests>Flaky.
Cc: tdander...@chromium.org alph@chromium.org
Status: WontFix (was: Untriaged)
Looks like benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.browse:news:cnn has not flaked in the last ~170 builds. Marking this as WontFix, and as per #3 it looks like a separate bug will be auto-filed if this flakiness is observed again.

Sign in to add a comment