"benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.load:media:soundcloud" is flaky |
||||||||||
Issue description"benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.load:media:soundcloud" is flaky. This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label. We have detected 8 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNygAELEgVGbGFrZSJ1YmVuY2htYXJrcy5zeXN0ZW1faGVhbHRoX3Ntb2tlX3Rlc3QuU3lzdGVtSGVhbHRoQmVuY2htYXJrU21va2VUZXN0LnN5c3RlbV9oZWFsdGgubWVtb3J5X2Rlc2t0b3AubG9hZDptZWRpYTpzb3VuZGNsb3VkDA. Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs This flaky test/step was previously tracked in issue 720210 .
,
May 26 2017
,
May 26 2017
I'm confused. I went to disable this test in tools/perf/benchmarks/system_health_smoke_test.py, but IT IS ALREADY DISABLED: # Disable media tests in CQ. crbug.com/649392 'benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.play:media:soundcloud', # pylint: disable=line-too-long Why is it running? I don't see any recent changes to that file that would cause test disablement to be ignored.
,
May 26 2017
Failing test is load:media:soundcloud, whereas disabled test is play:media:soundcloud
,
May 26 2017
#4 oops yes I just noticed this.
,
May 26 2017
,
May 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a89c40e112c18b20534d84678228acbcde10620e commit a89c40e112c18b20534d84678228acbcde10620e Author: Matt Giuca <mgiuca@chromium.org> Date: Fri May 26 08:24:01 2017 Disabled soundcloud load test in SystemHealthBenchmarkSmokeTest. Test is timing out. The 'play' test was already disabled; now the 'load' test is also disabled. TBR=nednguyen@chromium.org Bug: 649392 , 726439 Change-Id: I6eca533f0a9348adb0d567be1f6af907a603e0ae Reviewed-on: https://chromium-review.googlesource.com/516923 Reviewed-by: Matt Giuca <mgiuca@chromium.org> Commit-Queue: Matt Giuca <mgiuca@chromium.org> Cr-Commit-Position: refs/heads/master@{#474965} [modify] https://crrev.com/a89c40e112c18b20534d84678228acbcde10620e/tools/perf/benchmarks/system_health_smoke_test.py
,
May 26 2017
,
May 26 2017
Should the source of the problem be identified, fixed, and the story re-enabled before closing this issue?
,
May 26 2017
,
May 26 2017
perezju@: yes, this should be closed once the actual problem is fixed. Assigning to you since you are listed as owner for system health memory benchmarks. Please help find a better owner if necessary. Since the test is disabled, lowering priority to 2.
,
May 26 2017
<meme title="I don't know what I expected" /> From the swarming logs: https://chromium-swarm.appspot.com/task?id=36555c94e4b68f10&refresh=10&show_raw=1&wide_logs=true The error appears to be a timeout while forcing garbage collection: File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 793, in ForceGarbageCollection self._tab.CollectGarbage() File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/tab.py", line 242, in CollectGarbage self._inspector_backend.CollectGarbage() [...] File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 511, in CollectGarbage self._page.CollectGarbage() File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_page.py", line 162, in CollectGarbage res = self._inspector_websocket.SyncRequest(request, timeout) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 110, in SyncRequest res = self._Receive(timeout) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 149, in _Receive data = self._socket.recv() And the browser logs don't appear to contain anything useful. The error leads me to suspect an issue communicating with devtools, but I cannot be sure. Strangely, the story is not failing on the regular bots: config story build history 0 linux memory load:media:soundcloud 717 -------------------- 1 linux common load:media:soundcloud 717 -------------------- Ned any ideas on how to debug this further?
,
May 26 2017
I suspect it's https://github.com/catapult-project/catapult/issues/3218 I am not an expert on this area, but I suspect that we can't really assume a long like socket connection between Telemetry & browser will always be stable. Andrey: how do you keep the socket connection between devtool & browser stable?
,
May 26 2017
This is the first time I hear of websockets being divided into long-lived and short-lived. This terms seem to be specific to the python library that telemetry is using, and even there seems to be specific to particular examples. I suggest you check the code websocket-client. We don't do anything special to keep the connection stable, considering is's usually on top of rather reliable media, I don't expect it to be unstable.
,
May 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/31217030d60f5b0250fd3841be65de6e6f4359a9 commit 31217030d60f5b0250fd3841be65de6e6f4359a9 Author: Matt Giuca <mgiuca@chromium.org> Date: Wed May 31 03:39:59 2017 SystemHealthSmokeTest: Fixed disable message typo. Bug: 726439 Change-Id: Ia688e6b35c33e48353d371476991c35559eb1b7d Reviewed-on: https://chromium-review.googlesource.com/516865 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Matt Giuca <mgiuca@chromium.org> Cr-Commit-Position: refs/heads/master@{#475776} [modify] https://crrev.com/31217030d60f5b0250fd3841be65de6e6f4359a9/tools/perf/benchmarks/system_health_smoke_test.py
,
Sep 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/359bafdab6c2f69932618e2879f1f5689e08af59 commit 359bafdab6c2f69932618e2879f1f5689e08af59 Author: Juan A. Navarro Perez <perezju@chromium.org> Date: Fri Sep 08 11:09:38 2017 [tools/perf] Re-enable meadia system health smoke tests These are running fine on bots now. Bug: 726439 Change-Id: I97a5f9e85d873c685af14085f9534081fd3a5ee5 Reviewed-on: https://chromium-review.googlesource.com/655299 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org> Cr-Commit-Position: refs/heads/master@{#500564} [modify] https://crrev.com/359bafdab6c2f69932618e2879f1f5689e08af59/tools/perf/benchmarks/system_health_smoke_test.py
,
Sep 8 2017
,
Sep 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ed15e687ba59854b2124a8f6c2bda9e4bc5616e1 commit ed15e687ba59854b2124a8f6c2bda9e4bc5616e1 Author: Roger McFarlane <rogerm@chromium.org> Date: Fri Sep 08 15:06:59 2017 Revert "[tools/perf] Re-enable meadia system health smoke tests" This reverts commit 359bafdab6c2f69932618e2879f1f5689e08af59. Reason for revert: Seeing telemetry-perf bot failures. Original change's description: > [tools/perf] Re-enable meadia system health smoke tests > > These are running fine on bots now. > > Bug: 726439 > Change-Id: I97a5f9e85d873c685af14085f9534081fd3a5ee5 > Reviewed-on: https://chromium-review.googlesource.com/655299 > Reviewed-by: Ned Nguyen <nednguyen@google.com> > Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org> > Cr-Commit-Position: refs/heads/master@{#500564} TBR=perezju@chromium.org,nednguyen@google.com Change-Id: Ib3d38babf138c04ea32fa62cf257d20727593888 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 726439 , 763379 Reviewed-on: https://chromium-review.googlesource.com/657917 Reviewed-by: Roger McFarlane <rogerm@chromium.org> Commit-Queue: Roger McFarlane <rogerm@chromium.org> Cr-Commit-Position: refs/heads/master@{#500594} [modify] https://crrev.com/ed15e687ba59854b2124a8f6c2bda9e4bc5616e1/tools/perf/benchmarks/system_health_smoke_test.py
,
Sep 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/71921771dba55d265b7f953a4c0809c36ee180a0 commit 71921771dba55d265b7f953a4c0809c36ee180a0 Author: Juan Antonio Navarro Pérez <perezju@chromium.org> Date: Tue Sep 26 13:30:46 2017 Reland "[tools/perf] Re-enable meadia system health smoke tests" This reverts commit ed15e687ba59854b2124a8f6c2bda9e4bc5616e1. Reason for revert: Tests should now fit within the allotted time. Original change's description: > Revert "[tools/perf] Re-enable meadia system health smoke tests" > > This reverts commit 359bafdab6c2f69932618e2879f1f5689e08af59. > > Reason for revert: > > Seeing telemetry-perf bot failures. > > Original change's description: > > [tools/perf] Re-enable meadia system health smoke tests > > > > These are running fine on bots now. > > > > Bug: 726439 > > Change-Id: I97a5f9e85d873c685af14085f9534081fd3a5ee5 > > Reviewed-on: https://chromium-review.googlesource.com/655299 > > Reviewed-by: Ned Nguyen <nednguyen@google.com> > > Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#500564} > > TBR=perezju@chromium.org,nednguyen@google.com > > Change-Id: Ib3d38babf138c04ea32fa62cf257d20727593888 > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Bug: 726439 , 763379 > Reviewed-on: https://chromium-review.googlesource.com/657917 > Reviewed-by: Roger McFarlane <rogerm@chromium.org> > Commit-Queue: Roger McFarlane <rogerm@chromium.org> > Cr-Commit-Position: refs/heads/master@{#500594} TBR=rogerm@chromium.org,perezju@chromium.org,nednguyen@google.com # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 726439 , 763379 Change-Id: I309dfdd0f7b32bd4dc275904fc24ef1e251f8c85 Reviewed-on: https://chromium-review.googlesource.com/684024 Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org> Cr-Commit-Position: refs/heads/master@{#504352} [modify] https://crrev.com/71921771dba55d265b7f953a4c0809c36ee180a0/tools/perf/benchmarks/system_health_smoke_test.py
,
Sep 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2f37bb9a989ac2130617f13b47984a8e972b7e81 commit 2f37bb9a989ac2130617f13b47984a8e972b7e81 Author: Marc Treib <treib@chromium.org> Date: Tue Sep 26 15:48:01 2017 Revert "Reland "[tools/perf] Re-enable meadia system health smoke tests"" This reverts commit 71921771dba55d265b7f953a4c0809c36ee180a0. Reason for revert: telemetry_perf_unittests failing again: https://uberchromegw.corp.google.com/i/chromium.linux/builders/Linux%20Tests Original change's description: > Reland "[tools/perf] Re-enable meadia system health smoke tests" > > This reverts commit ed15e687ba59854b2124a8f6c2bda9e4bc5616e1. > > Reason for revert: Tests should now fit within the allotted time. > > Original change's description: > > Revert "[tools/perf] Re-enable meadia system health smoke tests" > > > > This reverts commit 359bafdab6c2f69932618e2879f1f5689e08af59. > > > > Reason for revert: > > > > Seeing telemetry-perf bot failures. > > > > Original change's description: > > > [tools/perf] Re-enable meadia system health smoke tests > > > > > > These are running fine on bots now. > > > > > > Bug: 726439 > > > Change-Id: I97a5f9e85d873c685af14085f9534081fd3a5ee5 > > > Reviewed-on: https://chromium-review.googlesource.com/655299 > > > Reviewed-by: Ned Nguyen <nednguyen@google.com> > > > Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org> > > > Cr-Commit-Position: refs/heads/master@{#500564} > > > > TBR=perezju@chromium.org,nednguyen@google.com > > > > Change-Id: Ib3d38babf138c04ea32fa62cf257d20727593888 > > No-Presubmit: true > > No-Tree-Checks: true > > No-Try: true > > Bug: 726439 , 763379 > > Reviewed-on: https://chromium-review.googlesource.com/657917 > > Reviewed-by: Roger McFarlane <rogerm@chromium.org> > > Commit-Queue: Roger McFarlane <rogerm@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#500594} > > TBR=rogerm@chromium.org,perezju@chromium.org,nednguyen@google.com > > # Not skipping CQ checks because original CL landed > 1 day ago. > > Bug: 726439 , 763379 > Change-Id: I309dfdd0f7b32bd4dc275904fc24ef1e251f8c85 > Reviewed-on: https://chromium-review.googlesource.com/684024 > Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> > Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org> > Cr-Commit-Position: refs/heads/master@{#504352} TBR=rogerm@chromium.org,perezju@chromium.org,nednguyen@google.com Change-Id: Ib027b20abc609bab8a6269fe658ad5b135eb66b7 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 726439 , 763379 Reviewed-on: https://chromium-review.googlesource.com/685094 Reviewed-by: Marc Treib <treib@chromium.org> Commit-Queue: Marc Treib <treib@chromium.org> Cr-Commit-Position: refs/heads/master@{#504378} [modify] https://crrev.com/2f37bb9a989ac2130617f13b47984a8e972b7e81/tools/perf/benchmarks/system_health_smoke_test.py |
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by mgiuca@chromium.org
, May 26 2017Labels: OS-Linux
Owner: mgiuca@chromium.org
Status: Started (was: Untriaged)