Test failures with ValueError("No JSON object could be decoded") on ChromeOS |
||||||||||
Issue description
Following tests are failing: testExecutingTestsInSortedOrder, testFilteringAfterSharding, testJsonOutputFormatNegativeFilter, testJsonOutputFormatPositiveFilter, testJsonOutputWhenSetupClassFailed, testJsonOutputWhenTearDownClassFailed, testSetUpProcessCalledOnce, testShardedTestRun, testSplittingShardsByTimes, testTearDownProcessCalledOnce.
Stacks all look like this:
telemetry.testing.browser_test_runner_unittest.BrowserTestRunnerTest.testExecutingTestsInSortedOrder failed unexpectedly 0.4128s:
02/22 18:54:59.584 INFO |run_chromeos_tests:0052| Traceback (most recent call last):
02/22 18:54:59.584 INFO |run_chromeos_tests:0052| File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/testing/browser_test_runner_unittest.py", line 137, in testExecutingTestsInSortedOrder
02/22 18:54:59.584 INFO |run_chromeos_tests:0052| 'Alphabetical', [], alphabetical_tests)
02/22 18:54:59.585 INFO |run_chromeos_tests:0052| File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/testing/browser_test_runner_unittest.py", line 68, in baseTest
02/22 18:54:59.585 INFO |run_chromeos_tests:0052| test_result = json.load(f)
02/22 18:54:59.585 INFO |run_chromeos_tests:0052| File "/usr/local/lib/python2.7/json/__init__.py", line 290, in load
02/22 18:54:59.586 INFO |run_chromeos_tests:0052| **kw)
02/22 18:54:59.586 INFO |run_chromeos_tests:0052| File "/usr/local/lib/python2.7/json/__init__.py", line 338, in loads
02/22 18:54:59.587 INFO |run_chromeos_tests:0052| return _default_decoder.decode(s)
02/22 18:54:59.587 INFO |run_chromeos_tests:0052| File "/usr/local/lib/python2.7/json/decoder.py", line 366, in decode
02/22 18:54:59.587 INFO |run_chromeos_tests:0052| obj, end = self.raw_decode(s, idx=_w(s, 0).end())
02/22 18:54:59.588 INFO |run_chromeos_tests:0052| File "/usr/local/lib/python2.7/json/decoder.py", line 384, in raw_decode
02/22 18:54:59.588 INFO |run_chromeos_tests:0052| raise ValueError("No JSON object could be decoded")
02/22 18:54:59.588 INFO |run_chromeos_tests:0052| ValueError: No JSON object could be decoded
First failing build:
https://uberchromegw.corp.google.com/i/chromiumos.chromium/builders/x86-generic-telemetry/builds/11797
Catapult log of failing build has 3 commits:
https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/16ba85e7d1d5..2242f1ab32ba
,
Feb 27 2017
+cc gardeners for this week.
,
Feb 27 2017
That CL changes the perf dashboard code. How could it cause CrOS test failures?
,
Feb 27 2017
These tests don't fail when I call telemetry/bin/run_tests testJsonOutput, but fail when I use the telmetry_UnitTests wrapper, so maybe there's a problem in run_chromeos_tests.py? https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/testing/run_chromeos_tests.py Annie - you're right, Dave's CL is not the culprit; my bisect was incorrect.
,
Feb 27 2017
Looks like it's this CL: https://codereview.chromium.org/2708403004 Perhaps run_chromeos_tests.py also needs to be updated?
,
Feb 28 2017
The CL is too complex for me to fix unfortunately. The cros FYI bots for telemetry have been red for almost 2 weeks due to 2 cascading issues. CL to disable these tests: https://codereview.chromium.org/2720313002/
,
Feb 28 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2194663c24756def6a7747f8bbfc12b2614f6143 commit 2194663c24756def6a7747f8bbfc12b2614f6143 Author: catapult-deps-roller <catapult-deps-roller@chromium.org> Date: Tue Feb 28 19:39:06 2017 Roll src/third_party/catapult/ c81620124..ad3cf3625 (2 commits). https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/c81620124f0f..ad3cf362537b $ git log c81620124..ad3cf3625 --date=short --no-merges --format='%ad %ae %s' 2017-02-28 achuith Disable failing tests in browser_test_runner_unittest.py 2017-02-28 jessimb Added a show more button. Created with: roll-dep src/third_party/catapult BUG=696553 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=catapult-sheriff@chromium.org Review-Url: https://codereview.chromium.org/2722603004 Cr-Commit-Position: refs/heads/master@{#453669} [modify] https://crrev.com/2194663c24756def6a7747f8bbfc12b2614f6143/DEPS
,
Mar 8 2017
Sorry, I am not familiar with how cros is setup to run the benchmark. Reassigned to achuith@
,
Mar 9 2017
Ned, I don't mind fixing cros failures when they are relatively straightforward. In this case, I'll have to understand your CL and debug tests that you introduced with this CL. Do you have any time to meet with me next week to go over this together? We can abandon the effort if it takes more than 30 min, but there's a good chance the fix is easy.
,
Mar 9 2017
P2 since the test is currently disabled and the bots are green.
,
Jan 16
,
Jan 16
|
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by achuith@chromium.org
, Feb 27 2017Status: Assigned (was: Untriaged)