New issue
Advanced search Search tips

Issue 875130 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug

Blocking:
issue 668217



Sign in to add a comment

"org.chromium.chrome.browser.hardware_acceleration.ToastHWATest#testOpenedInBackgroundToast" is flaky

Project Member Reported by chromium...@appspot.gserviceaccount.com, Aug 17

Issue description

"org.chromium.chrome.browser.hardware_acceleration.ToastHWATest#testOpenedInBackgroundToast" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 17 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyZQsSBUZsYWtlIlpvcmcuY2hyb21pdW0uY2hyb21lLmJyb3dzZXIuaGFyZHdhcmVfYWNjZWxlcmF0aW9uLlRvYXN0SFdBVGVzdCN0ZXN0T3BlbmVkSW5CYWNrZ3JvdW5kVG9hc3QM.

Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
 
Blocking: 668217
Components: Test>Android
Labels: -Sheriff-Chromium OS-Android
Owner: dskiba@chromium.org
Status: Assigned (was: Untriaged)
Disabling this in https://crrev.com/c/1179665.

dskiba: Could you (here as well) please help investigate?
Project Member

Comment 2 by bugdroid1@chromium.org, Aug 17

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2db0839d6a455786e586c14cfcffa02fc96af665

commit 2db0839d6a455786e586c14cfcffa02fc96af665
Author: Friedrich Horschig <fhorschig@chromium.org>
Date: Fri Aug 17 10:06:09 2018

[Flaky][Android] Mark several ToastHWATests flaky

Theses tests were known to be flaky for a while and this is not the
first time they are disabled for unknown reasons. Reopened and linked
bugs with details.

TBR=dskiba@chromium.org, tedchoc@chromium.org

Bug: 668217, 875130,  875129 ,  875092 
Change-Id: Ifee7d29bbdf28349aea3837a6b1a2525484c6e9a
Reviewed-on: https://chromium-review.googlesource.com/1179665
Reviewed-by: Friedrich Horschig <fhorschig@chromium.org>
Commit-Queue: Friedrich Horschig <fhorschig@chromium.org>
Cr-Commit-Position: refs/heads/master@{#584010}
[modify] https://crrev.com/2db0839d6a455786e586c14cfcffa02fc96af665/chrome/android/javatests/src/org/chromium/chrome/browser/hardware_acceleration/ToastHWATest.java

Labels: Sheriff-Chromium
Owner: fhorschig@chromium.org
First, I'm not working on Chrome anymore, please find someone else.

Second, the failure doesn't seem to be test-specific, I checked two logs and they both have the following exception:

Traceback (most recent call last):
  File "/b/swarming/w/ir/build/android/test_runner.py", line 1040, in main
    return RunTestsCommand(args)
  File "/b/swarming/w/ir/build/android/test_runner.py", line 743, in RunTestsCommand
    return RunTestsInPlatformMode(args)
  File "/b/swarming/w/ir/build/android/test_runner.py", line 867, in RunTestsInPlatformMode
    with json_writer(), logcats_uploader, env, test_instance, test_run:
  File "/b/swarming/w/ir/build/android/pylib/base/test_run.py", line 39, in __enter__
    self.SetUp()
  File "/b/swarming/w/ir/build/android/pylib/local/device/local_device_gtest_run.py", line 351, in SetUp
    self._env.parallel_devices.pMap(
  File "/b/swarming/w/ir/build/android/pylib/local/device/local_device_environment.py", line 189, in parallel_devices
    return parallelizer.SyncParallelizer(self.devices)
  File "/b/swarming/w/ir/build/android/pylib/local/device/local_device_environment.py", line 180, in devices
    self._InitDevices()
  File "/b/swarming/w/ir/build/android/pylib/local/device/local_device_environment.py", line 137, in _InitDevices
    default_retries=self._max_tries - 1, device_arg=device_arg)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 2916, in HealthyDevices
    return _get_devices()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 2903, in _get_devices
    raise device_errors.NoDevicesError()
NoDevicesError: No devices attached.


Cc: dskiba@chromium.org
 Issue 875092  has been merged into this issue.
 Issue 875129  has been merged into this issue.
Components: Infra
Owner: ----
Status: Available (was: Assigned)
That's actually a good pointer, thanks!
An unattached device sounds like a bot config error ... Maybe someone from Infra can help?
Labels: -Sheriff-Chromium Infra-Troopers
Troopers, do you mind taking a look? Thank you! 
Components: -Infra
Labels: -Infra-Troopers Sheriff-Chromium
If it's failing at this rate, it seems extraordinarily unlikely that it's a device issue.

dskiba, I'm not sure which builds you were looking at, but I'm seeing the tests in question crash repeatedly with

[FATAL:persistent_histogram_allocator.cc(893)] Check failed: !subtle::NoBarrier_Load(&g_histogram_allocator)

e.g. in

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/63564
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/63553
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/63542
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/63528

Back to the sheriffs.
Owner: yosin@chromium.org
Status: Started (was: Available)
Investigating...
Labels: -Sheriff-Chromium
Owner: tedc...@chromium.org
Status: Assigned (was: Started)
Since test is already marked flaky, assigned to tedchoc@ for triage.
Cc: tedc...@chromium.org jbudorick@google.com
Owner: sadrul@chromium.org
@jbudorick, all the flakes were on the:
chrome_public_test_apk on Android device Nexus 5X (with patch, experimental)
step.

With that said, we did see that this all started after this patch landed:
https://chromium-review.googlesource.com/c/chromium/src/+/1174709

The problem the step is experimental, which means we don't revert changes as a result of that.

If we think this is likely still an issue, then I'm going to assign to sadrul@ to do some preliminary investigation.  Is there a way to force that type of setup locally?  Should we try to run these tests a bunch of times to see if they fail?
Cc: -jbudorick@google.com jbudorick@chromium.org
@ted: that just means that this is flaking on M and not K.
https://chromium-review.googlesource.com/c/chromium/src/+/1174709 was reverted in https://chromium-review.googlesource.com/c/chromium/src/+/1180741 because it was breaking android-go-perf bots (with the same callstack in persistent_histogram_allocator.cc). The CL was relanded with a fix in https://chromium-review.googlesource.com/c/chromium/src/+/1182041. The android-go-perf bots have been green after the fixed CL land. I suspect these tests are also no longer flaking after that.
Is it possible to check if the tests are still flaking?

Sign in to add a comment