New issue
Advanced search Search tips

Issue 594832 link

Starred by 0 users

Issue metadata

Status: Duplicate
Merged: issue 580919
Owner:
Closed: Mar 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug

Blocked on:
issue 588049



Sign in to add a comment

"telemetry_perf_unittests (with patch)" is flaky

Project Member Reported by chromium...@appspot.gserviceaccount.com, Mar 14 2016

Issue description

"telemetry_perf_unittests (with patch)" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 3 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA.

Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs

This flaky test/step was previously tracked in  issue 593747 .
 

Comment 1 by csharp@chromium.org, Mar 15 2016

Labels: -Sheriff-Chromium Infra-Troopers
All three failures look like bot issues, assigning to infra-troopers in case these failure are unknown.

Sample output:
Running ['/usr/bin/python', u'/b/build/slave/android/build/src/infra/scripts/runtest_wrapper.py', '--', '--target', 'Release', '--xvfb', '--builder-name', u'linux_android_rel_ng', '--slave-name', u'build80-b4', '--build-number', '37996', '--build-properties', '{"buildnumber": 37996, "slavename": "build80-b4", "target_platform": "android", "mastername": "tryserver.chromium.android", "buildername": "linux_android_rel_ng"}', '--test-type', 'telemetry_perf_unittests', '--run-python-script', '/b/build/slave/android/build/src/tools/perf/run_tests', '--retry-limit', '3', '--browser', 'android-chromium', '--device', 'android', '--android-blacklist-file', u'/b/build/slave/android/build/src/out/bad_devices.json', '--write-full-results-to', '/tmp/tmp1GNWC3']

/usr/bin/python /b/build/slave/android/build/src/tools/perf/run_tests --retry-limit 3 --browser android-chromium --device android --android-blacklist-file /b/build/slave/android/build/src/out/bad_devices.json --write-full-results-to /tmp/tmp1GNWC3
WARNING:root:Device 03d27ca10ae59472 is blacklisted.
WARNING:root:Device 05fcf061006ade74 is blacklisted.
WARNING:root:Device 06085ed10062306c is blacklisted.
WARNING:root:Device 06b8250f00622716 is blacklisted.
WARNING:root:Device 06085d6d006230af is blacklisted.
WARNING:root:Device 06b6f30c006227c3 is blacklisted.
WARNING:root:Device 06c330d8006b0000 is blacklisted.
Usage: run_tests.py [test_name ...] [<options>]

run_tests.py: error: No browser found of type android-chromium. Cannot run tests.
Re-run with --browser=list to see available browser types.
[Running on builder: "linux_android_rel_ng"]
DBUS_SESSION_BUS_ADDRESS env var not found, starting dbus-launch
 setting DBUS_SESSION_BUS_ADDRESS to unix:abstract=/tmp/dbus-3eidtuBUEi,guid=af61ecf7e5a51244ee21203e56e6f984
 setting DBUS_SESSION_BUS_PID to 29349
Window manager (openbox) started.
Additional test environment:
  CHROME_DEVEL_SANDBOX=/opt/chromium/chrome_sandbox
  LD_LIBRARY_PATH=/b/build/slave/android/build/src/out/Release:/b/build/slave/android/build/src/out/Release/lib:/b/build/slave/android/build/src/out/Release/lib.target
Stopping Xvfb with pid 29351 ...
Xvfb pid file removed
 killed dbus-daemon with PID 29349
 cleared DBUS_SESSION_BUS_ADDRESS environment variable
Traceback (most recent call last):
Command ['/usr/bin/python', u'/b/build/slave/android/build/src/infra/scripts/runtest_wrapper.py', '--', '--target', 'Release', '--xvfb', '--builder-name', u'linux_android_rel_ng', '--slave-name', u'build80-b4', '--build-number', '37996', '--build-properties', '{"buildnumber": 37996, "slavename": "build80-b4", "target_platform": "android", "mastername": "tryserver.chromium.android", "buildername": "linux_android_rel_ng"}', '--test-type', 'telemetry_perf_unittests', '--run-python-script', '/b/build/slave/android/build/src/tools/perf/run_tests', '--retry-limit', '3', '--browser', 'android-chromium', '--device', 'android', '--android-blacklist-file', u'/b/build/slave/android/build/src/out/bad_devices.json', '--write-full-results-to', '/tmp/tmp1GNWC3'] returned exit code 2
  File "/b/build/slave/android/build/src/testing/scripts/telemetry_perf_unittests.py", line 65, in <module>
    sys.exit(common.run_script(sys.argv[1:], funcs))
  File "/b/build/slave/android/build/src/testing/scripts/common.py", line 59, in run_script
    return args.func(args)
  File "/b/build/slave/android/build/src/testing/scripts/telemetry_perf_unittests.py", line 39, in main_run
    results = json.load(f)
  File "/usr/lib/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
step returned non-zero exit code: 1
@@@STEP_LOG_LINE@json.output (invalid)@null@@@
@@@STEP_LOG_END@json.output (invalid)@@@
@@@STEP_FAILURE@@@
Cc: sullivan@chromium.org
Labels: -Infra-Troopers Infra-Android OS-Android
Looks like something for Infra-Android! But probably the speed infra team as well?

Kinda looks like the lack of devices is badly interacting with the test runner script...
Failing to run tests when there are no devices available on which tests can be run is WAI. Probably not a speed infra issue.
I would agree, but it would be nice to give a nicer error than "No JSON object could be decoded"
Owner: bpastene@chromium.org
Status: Assigned (was: Untriaged)
All recent flakes have been on build80-b4, looks like a problem with that slave specifically. I'll take it offline to investigate.
Blockedon: 588049
Project Member

Comment 7 by chromium...@appspot.gserviceaccount.com, Mar 25 2016

Labels: Sheriff-Chromium
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Components: Tests>Telemetry
Owner: ----
Status: Untriaged (was: Assigned)
build80-b4 has been fixed and all recent flakes are actual flakes:

https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/43153
https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/43226
https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/43299

The build times-out due to lack of output from the test for 40min. Looks like telemetry_perf_unittests has a deadlock of sorts at the very end?
Owner: nedngu...@google.com
Status: Started (was: Untriaged)
Yes, there is currently a deadlock that happen very often on win platform.

Comment 10 by grt@chromium.org, Mar 30 2016

Labels: -Sheriff-Chromium
Removing Sheriff-Chromium since this has an owner.
Mergedinto: 580919
Status: Duplicate (was: Started)
Components: Infra>Client>Android
Labels: -Infra-Android

Sign in to add a comment