telemetry_perf_unittests failing on 6 builders (Mac10.1{0,1,2}, Linux Tests, Win 7 Tests {,x64}) |
|||||||
Issue descriptiontelemetry_perf_unittests failing on 3 builders Builders failed on: - Mac10.10 Tests: https://build.chromium.org/p/chromium.mac/builders/Mac10.10%20Tests - Mac10.11 Tests: https://build.chromium.org/p/chromium.mac/builders/Mac10.11%20Tests - Mac10.12 Tests: https://build.chromium.org/p/chromium.mac/builders/Mac10.12%20Tests
,
May 24 2017
I had a look at 2 "sampled" failing builds per each of 3 builds, the failures there are as described above.
,
May 24 2017
The same test fails on 3 more builders: https://build.chromium.org/p/chromium.linux/builders/Linux%20Tests https://build.chromium.org/p/chromium.win/builders/Win%207%20Tests%20x64%20%281%29 https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20%281%29
,
May 24 2017
On Linux Tests FindIt reports https://codereview.chromium.org/2899123002 as a culprit. This CL is also present in the first failing builds. The CL disables "webrtc.multiple_peerconnections story" (see issue 725502).
,
May 24 2017
Hi rnephew@ & nednguyen@, your CL (https://codereview.chromium.org/2899123002) seems to break benchmarks.benchmark_smoke_unittest.BenchmarkSmokeTest.webrtc.
,
May 24 2017
I've contacted people cc'ed in issue 725502 via chat. ehmaldonado@ confirmed that revert of https://codereview.chromium.org/2899123002 will cause failing of multiple_peerconnections tests.
,
May 24 2017
ehmaldonado@ effectively reverted https://codereview.chromium.org/2899123002, while disabling the multiple_peerconnections test in a different way. The CL is https://codereview.chromium.org/2896253004 and we are landing it.
,
May 24 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a1db6b7d87922f09904037e82447170cc97c0a2e commit a1db6b7d87922f09904037e82447170cc97c0a2e Author: ehmaldonado <ehmaldonado@chromium.org> Date: Wed May 24 07:39:44 2017 [Telemetry] Disable webrtc.multiple_peerconnections story. It is failing on all platforms. The previous CL (https://codereview.chromium.org/2899123002) caused builders to break. BUG=725502, 725796 # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true TBR=rnephew@chromium.org, nednguyen@google.com, vitaliii@chromium.org Review-Url: https://codereview.chromium.org/2896253004 Cr-Commit-Position: refs/heads/master@{#474198} [modify] https://crrev.com/a1db6b7d87922f09904037e82447170cc97c0a2e/tools/perf/benchmarks/webrtc.py [modify] https://crrev.com/a1db6b7d87922f09904037e82447170cc97c0a2e/tools/perf/page_sets/webrtc_cases.py
,
May 24 2017
I forcelanded the CL without waiting for trybots (it is effectively a revert of a recent CL with a trivial fix on top).
,
May 24 2017
Linux Tests builder is green now (https://build.chromium.org/p/chromium.linux/builders/Linux%20Tests/builds/56978). Waiting for other builders to pickup the revert.
,
May 24 2017
They all are green now. Thanks ehmaldonado@ for the fix.
,
May 24 2017
,
Jan 16
,
Jan 16
|
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by vitaliii@chromium.org
, May 24 2017I had a look at first failing builds on all three builders. The situation seems the same - benchmarks.benchmark_smoke_unittest.BenchmarkSmokeTest.webrtc fails. The log does not look helpful: === LOG STARTS === Trace sizes in bytes: {'traceEvents': 5435932, 'telemetry': 34426, 'tabIds': 40} Try printing formatted exception: None None None Traceback (most recent call last): RunBenchmark at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py:389 benchmark.ValidateExpectations(stories) ValidateExpectations at /b/s/w/ir/third_party/catapult/telemetry/telemetry/benchmark.py:273 self._expectations.ValidateAgainstStorySet(story_set) ValidateAgainstStorySet at /b/s/w/ir/third_party/catapult/telemetry/telemetry/story/expectations.py:31 raise TypeError('Story %s is not in the story set.' % expectation) TypeError: Story multiple_peerconnections is not in the story set. Locals: expectation : 'multiple_peerconnections' s : <page_sets.webrtc_cases.GetUserMedia object at 0x10bbca990> stories : ['hd_local_stream_10s'] story_set : <page_sets.webrtc_cases.WebrtcPageSet object at 0x10bbcae50> Traceback (most recent call last): File "/b/s/w/ir/tools/perf/benchmarks/benchmark_smoke_unittest.py", line 84, in BenchmarkSmokeTest msg='Failed: %s' % benchmark) AssertionError: Failed: <class 'benchmarks.webrtc.WebrtcPerfBenchmark'> Retrying failed tests (attempt #3 of 3)... [ RUN ] /b/s/w/itGGHOHx/tmp3DxdSC.html [ OK ] /b/s/w/itGGHOHx/tmp3DxdSC.html (3565 ms) [1/1] benchmarks.benchmark_smoke_unittest.BenchmarkSmokeTest.webrtc failed unexpectedly 34.0436s: === LOG ENDS ===