Get telemetry test coverage on ChromeOS back on the CQ |
|||||||||||||||||||
Issue description"benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.multitab:misc:typical24" is flaky. This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label. We have detected 4 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyggELEgVGbGFrZSJ3YmVuY2htYXJrcy5zeXN0ZW1faGVhbHRoX3Ntb2tlX3Rlc3QuU3lzdGVtSGVhbHRoQmVuY2htYXJrU21va2VUZXN0LnN5c3RlbV9oZWFsdGgubWVtb3J5X2Rlc2t0b3AubXVsdGl0YWI6bWlzYzp0eXBpY2FsMjQM. Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs This flaky test/step was previously tracked in issue 698499 .
,
Apr 25 2018
,
Apr 25 2018
,
Apr 25 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8998796c733c0018c34648eabe99d5b3aac231b0 commit 8998796c733c0018c34648eabe99d5b3aac231b0 Author: Ned Nguyen <nednguyen@google.com> Date: Wed Apr 25 20:00:02 2018 Disable multitab benchmark on ChromeOS TBR=perezju@chromium.org NOTRY=true Bug: 836447 Change-Id: I8058c3f219831333b7539e5d015d825cf53620e8 Reviewed-on: https://chromium-review.googlesource.com/1028715 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#553708} [modify] https://crrev.com/8998796c733c0018c34648eabe99d5b3aac231b0/tools/perf/expectations.config
,
Apr 26 2018
Issue 837315 has been merged into this issue.
,
Apr 26 2018
We're still seeing chromeos failures despite disabling(https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyggELEgVGbGFrZSJ3YmVuY2htYXJrcy5zeXN0ZW1faGVhbHRoX3Ntb2tlX3Rlc3QuU3lzdGVtSGVhbHRoQmVuY2htYXJrU21va2VUZXN0LnN5c3RlbV9oZWFsdGgubWVtb3J5X2Rlc2t0b3AvbXVsdGl0YWI6bWlzYzp0eXBpY2FsMjQM), can you take a look Ned?
,
Apr 26 2018
Hmhh, I think this is because linux-chromeos-rel integration of Telemetry isn't working properly (issue 799950) I propose that we take telemetry_perf_unittest & telemetry_unittest out of this linux-chromeos-rel config until issue 799950 is fixed. +Dirk: who owns linux-chromeos-rel?
,
Apr 27 2018
Detected 6 new flakes for test/step "benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop/multitab:misc:typical24". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyggELEgVGbGFrZSJ3YmVuY2htYXJrcy5zeXN0ZW1faGVhbHRoX3Ntb2tlX3Rlc3QuU3lzdGVtSGVhbHRoQmVuY2htYXJrU21va2VUZXN0LnN5c3RlbV9oZWFsdGgubWVtb3J5X2Rlc2t0b3AvbXVsdGl0YWI6bWlzYzp0eXBpY2FsMjQM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Apr 27 2018
sheriff checking in. #7 sounds reasonable enough to me. dpranke, +bpastene, thoughts?
,
Apr 27 2018
SGTM, though I'm wondering if we know if something regressed or if it's never worked right?
,
Apr 28 2018
+cywang: FYI
,
Apr 30 2018
,
Apr 30 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/01dc60bac33dbae9c3f249d3c7278418b4617d26 commit 01dc60bac33dbae9c3f249d3c7278418b4617d26 Author: John Budorick <jbudorick@chromium.org> Date: Mon Apr 30 16:17:16 2018 Remove telemetry_{perf_,}unittests from linux-chromeos-rel. TBR=nednguyen@chromium.org,dpranke@chromium.org Bug: 836447 Change-Id: I9d102995fc94cf4950b44df91595f26709cf7624 Reviewed-on: https://chromium-review.googlesource.com/1034018 Reviewed-by: John Budorick <jbudorick@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: John Budorick <jbudorick@chromium.org> Cr-Commit-Position: refs/heads/master@{#554774} [modify] https://crrev.com/01dc60bac33dbae9c3f249d3c7278418b4617d26/testing/buildbot/chromium.chromiumos.json [modify] https://crrev.com/01dc60bac33dbae9c3f249d3c7278418b4617d26/testing/buildbot/test_suites.pyl [modify] https://crrev.com/01dc60bac33dbae9c3f249d3c7278418b4617d26/testing/buildbot/waterfalls.pyl
,
Apr 30 2018
,
May 2 2018
After #13, I don't believe we have any telemetry tests running on our cros cq bots. + some cros folks FYI. I know that telemetry is often a pain-point in the pfq, so this may make things more turbulent.
,
May 2 2018
I agree with the concerns expressed in comment #15. Especially since issue 799950 is only a P3. Can we please decide to prioritize issue 799950 or come up with a better short term solution? Otherwise I fear we are just going to be pushing the problem elsewhere (i.e. the cros PFQ or CQ).
,
May 2 2018
I removed the suite based on nednguyen's comment in #7 as part of sheriffing. If you'd like to discuss those concerns, he's likely the person to start with, not me.
,
May 2 2018
I assigned it to you since you put up the CL, but reassigning to nednguyen@ is fine. Leaving this unowned and available, i.e. forgotton, not so much :)
,
May 2 2018
I don't have bandwidth to work on this
,
May 2 2018
IIUC, Ned mentioned in comment #7 that telemetry's browser selection would need improvement before reenabling telemetry tests on linux-chromeos bots (bug 799950) Incidentally, I'm working on spinning up a true chromeos bot in our CI/CQ (via VMs) in bug 832374 . Since that's a real cros environment, I don't think bug 799950 would be relevant there. I'm pretty close to getting telemetry up and running in those VMs, so I propose the solution to this is to: - get the vm bot on the CQ - add telemetry tests to the vm bot Does that sg to everyone? FWIW, getting that bot up and running is my highest priority task right now, so I promise it won't be forgotten :)
,
May 2 2018
Plan in #20 SGTM.
,
May 3 2018
Thank you Ben for picking this up! Your plan SGTM. Feel free to close this and reference issue 832374 instead!
,
May 3 2018
I'll keep this open for further motivation for me, but repurpose it a little
,
May 4 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8ab6954de550aebdf36d72e899910731fe9b6186 commit 8ab6954de550aebdf36d72e899910731fe9b6186 Author: Ben Pastene <bpastene@chromium.org> Date: Fri May 04 04:08:24 2018 Add telemetry_perf_unittests to the cros vm test bot. Will all likely fail until I fix crbug.com/839187 This'll at least get them running. Bug: 836447 , 832374 Change-Id: I1d206ab931230652b4ba3dc8ba519f386a80da96 Reviewed-on: https://chromium-review.googlesource.com/1043258 Commit-Queue: Ben Pastene <bpastene@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#555974} [modify] https://crrev.com/8ab6954de550aebdf36d72e899910731fe9b6186/testing/buildbot/chromium.fyi.json [modify] https://crrev.com/8ab6954de550aebdf36d72e899910731fe9b6186/testing/buildbot/test_suites.pyl [modify] https://crrev.com/8ab6954de550aebdf36d72e899910731fe9b6186/testing/buildbot/waterfalls.pyl [modify] https://crrev.com/8ab6954de550aebdf36d72e899910731fe9b6186/tools/mb/mb.py
,
May 4 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e149db1b7ca4b5b3ca1e90156d2246a2200bf9b7 commit e149db1b7ca4b5b3ca1e90156d2246a2200bf9b7 Author: Ben Pastene <bpastene@chromium.org> Date: Fri May 04 23:20:43 2018 Skip testTelemetryUsesChromeClockDomain on CrOS VM test bot. Magically, this is the only failure: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/chromeos-amd64-generic-rel-vm-tests/1864 TBR=dpranke@chromium.org, nednguyen@google.com Bug: 836447 Change-Id: Iccce216208b418ea7b34b1793a902400ed084c97 Reviewed-on: https://chromium-review.googlesource.com/1044877 Reviewed-by: Ben Pastene <bpastene@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Ben Pastene <bpastene@chromium.org> Cr-Commit-Position: refs/heads/master@{#556239} [modify] https://crrev.com/e149db1b7ca4b5b3ca1e90156d2246a2200bf9b7/testing/buildbot/chromium.fyi.json [modify] https://crrev.com/e149db1b7ca4b5b3ca1e90156d2246a2200bf9b7/testing/buildbot/test_suites.pyl
,
May 7 2018
Detected 3 new flakes for test/step "benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop/multitab:misc:typical24". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyggELEgVGbGFrZSJ3YmVuY2htYXJrcy5zeXN0ZW1faGVhbHRoX3Ntb2tlX3Rlc3QuU3lzdGVtSGVhbHRoQmVuY2htYXJrU21va2VUZXN0LnN5c3RlbV9oZWFsdGgubWVtb3J5X2Rlc2t0b3AvbXVsdGl0YWI6bWlzYzp0eXBpY2FsMjQM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
May 8 2018
,
May 9 2018
Should you disable benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop/multitab:misc:typical24 as well, while you are working on it?
,
May 9 2018
,
May 18 2018
We've got telemetry tests running on the fyi bot: https://chromium-review.googlesource.com/c/chromium/src/+/1054590 It recently started failing core.perf_benchmark_unittest.PerfBenchmarkTest.testVariationArgs on chromeos due to a change in src (https://chromium-review.googlesource.com/c/chromium/src/+/1054590) If the test was on the CQ (I'm still working on that, should be doable very soon) then we would have prevented that from landing. As it is though, that change got in. I'm curious how that will now affect the PFQ. If it ends up preventing Chrome from getting rolled forward, what would the next steps for a gardener be? Revert the chromium-side change? Disable the test?
,
May 18 2018
Ben, the PFQ doesn't run telemetry unittests. We have an FYI bot that's monitored by the gardeners that runs telemetry/perf unittests. Autotests use telemetry for controlling chrome, so if there's a telemetry breakage on ChromeOS that's more fundamental, the PFQ will break, but a perf unit test failing won't affect the PFQ.
,
May 18 2018
Ideally we would also have chromeos telemetry unittests also running in the catapult CQ, otherwise a CL that had a green run on the catapult CQ will cause the catapult auto-roller to stall when it tries to roll that CL into chromium.
,
May 18 2018
re #31: Ah, alright. Thanks for clearing that up. re #32: And yeah, adding a cros bot to catapult's CQ is a good idea. It's got one for every other OS: https://build.chromium.org/p/tryserver.client.catapult/builders I'll get to that once I've sorted chromium's CQ out.
,
Jun 19 2018
Issue 827520 has been merged into this issue.
,
Jun 20 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/58fb9451e182cd2da15ec9ba66043784ad544f43 commit 58fb9451e182cd2da15ec9ba66043784ad544f43 Author: jonross <jonross@chromium.org> Date: Wed Jun 20 14:28:01 2018 Removing broken telemetry from linux-chromiumos bot For a while there was work looking into getting telemetry_perf_unittests and telemetry_unittests up and running on linux-chromiumos. However instead there is now a desire to just get an actual ChromeOS device on the CQ. This will resolve our coverage needs. We've been using the Mojo Chromiumos FYI bot to investigate this. But the tests have been broken there for a while. This change removes those configs. TEST=telemetry_perf_unittests, telemetry_unittests Bug: 836447 Change-Id: I8870c69dae811101d4869e3c7ede28771ecbf905 Reviewed-on: https://chromium-review.googlesource.com/1107838 Reviewed-by: Scott Violet <sky@chromium.org> Cr-Commit-Position: refs/heads/master@{#568834} [modify] https://crrev.com/58fb9451e182cd2da15ec9ba66043784ad544f43/testing/buildbot/chromium.fyi.json [modify] https://crrev.com/58fb9451e182cd2da15ec9ba66043784ad544f43/testing/buildbot/test_suites.pyl [modify] https://crrev.com/58fb9451e182cd2da15ec9ba66043784ad544f43/testing/buildbot/waterfalls.pyl
,
Jun 20 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6f8491c3e7ad966421f083a04c26fc192fe5a065 commit 6f8491c3e7ad966421f083a04c26fc192fe5a065 Author: Ben Pastene <bpastene@chromium.org> Date: Wed Jun 20 20:20:41 2018 Add telemetry_unittests to cros VM fyi bot. Bug: 836447 Change-Id: I08e90548e90e8f4c14e2551dd736a4fcf00b7b20 Reviewed-on: https://chromium-review.googlesource.com/1107262 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: John Budorick <jbudorick@chromium.org> Commit-Queue: Ben Pastene <bpastene@chromium.org> Cr-Commit-Position: refs/heads/master@{#568975} [modify] https://crrev.com/6f8491c3e7ad966421f083a04c26fc192fe5a065/testing/buildbot/chromium.fyi.json [modify] https://crrev.com/6f8491c3e7ad966421f083a04c26fc192fe5a065/testing/buildbot/test_suites.pyl
,
Jun 25 2018
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/8d2c28ec7666a034afe50a59f862ea94cef42d9a commit 8d2c28ec7666a034afe50a59f862ea94cef42d9a Author: Ben Pastene <bpastene@chromium.org> Date: Mon Jun 25 16:57:52 2018 telemetry: Fix cros_interface_unittest.testGetFileNonExistent. On swarming, $TMPDIR gets set to some path like /b/s/w/ir/... That path doesn't exist on a remote cros device, so the test here tries pushing to it and fails unexpectedly. This hardcodes /tmp/ instead, which should exist on the device. Bug: chromium:836447 Change-Id: I1f76063d4f459f7c25aa6470563818aaa35e0a53 Reviewed-on: https://chromium-review.googlesource.com/1112736 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ben Pastene <bpastene@chromium.org> [modify] https://crrev.com/8d2c28ec7666a034afe50a59f862ea94cef42d9a/telemetry/telemetry/core/cros_interface_unittest.py
,
Jun 26 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fcb4ff9c7d4616b235ce4dbe782e81fc41ce36fb commit fcb4ff9c7d4616b235ce4dbe782e81fc41ce36fb Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Tue Jun 26 00:22:11 2018 Roll src/third_party/catapult 7d423fc5614b..8d2c28ec7666 (1 commits) https://chromium.googlesource.com/catapult.git/+log/7d423fc5614b..8d2c28ec7666 git log 7d423fc5614b..8d2c28ec7666 --date=short --no-merges --format='%ad %ae %s' 2018-06-25 bpastene@chromium.org telemetry: Fix cros_interface_unittest.testGetFileNonExistent. Created with: gclient setdep -r src/third_party/catapult@8d2c28ec7666 The AutoRoll server is located here: https://catapult-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:836447 TBR=sullivan@chromium.org Change-Id: I0e98c2e411d82aed75acfcd7d5fb44cc6ad82b80 Reviewed-on: https://chromium-review.googlesource.com/1113821 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#570259} [modify] https://crrev.com/fcb4ff9c7d4616b235ce4dbe782e81fc41ce36fb/DEPS
,
Jun 27 2018
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/d96c633790000d036c5bbcf943d2c4f1f06d9820 commit d96c633790000d036c5bbcf943d2c4f1f06d9820 Author: Ben Pastene <bpastene@chromium.org> Date: Wed Jun 27 22:16:08 2018 Restore battor_wrapper.IsBattOrConnected in battor_tracing_agent_unittest. After the unittest ran, the mocked function would remain for the remainder of the test. This restores it to the true implementation in tear-down. Bug: chromium:836447 Change-Id: Iea62bb69236a5ae75668f1f6ace1934c98bf95c6 Reviewed-on: https://chromium-review.googlesource.com/1114189 Reviewed-by: Charlie Andrews <charliea@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ben Pastene <bpastene@chromium.org> [modify] https://crrev.com/d96c633790000d036c5bbcf943d2c4f1f06d9820/telemetry/telemetry/internal/platform/tracing_agent/battor_tracing_agent_unittest.py
,
Jun 28 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d32ce0619f16d5776147111e8cbb53a46b8019da commit d32ce0619f16d5776147111e8cbb53a46b8019da Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Thu Jun 28 02:27:19 2018 Roll src/third_party/catapult e82ab88bd0dd..d96c63379000 (8 commits) https://chromium.googlesource.com/catapult.git/+log/e82ab88bd0dd..d96c63379000 git log e82ab88bd0dd..d96c63379000 --date=short --no-merges --format='%ad %ae %s' 2018-06-27 bpastene@chromium.org Restore battor_wrapper.IsBattOrConnected in battor_tracing_agent_unittest. 2018-06-27 simonhatch@chromium.org Dashboard - Remove stats_around_revision 2018-06-27 simonhatch@chromium.org Dashboard - Remove some MR related references in yaml files. 2018-06-27 benjhayden@chromium.org Fix TypeError in Anomaly.QueryAsync. 2018-06-27 simonhatch@chromium.org Dashboard - Remove monitored property when migrating. 2018-06-27 tedchoc@google.com Add FOREGROUND_SERVICE to the permission blacklist. 2018-06-27 simonhatch@chromium.org Dashboard - Master doesn't need to derive from InternalOnlyModel 2018-06-27 eyaich@google.com Outputting a valid histogram set when running disabled benchmark Created with: gclient setdep -r src/third_party/catapult@d96c63379000 The AutoRoll server is located here: https://catapult-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:836447 TBR=sullivan@chromium.org Change-Id: I7e81c46db4309715a8c010bd7e590224e7583b84 Reviewed-on: https://chromium-review.googlesource.com/1117819 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#571006} [modify] https://crrev.com/d32ce0619f16d5776147111e8cbb53a46b8019da/DEPS
,
Jun 28 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f60c42119b59b0816f723fcfd9070cef0ea4e3f7 commit f60c42119b59b0816f723fcfd9070cef0ea4e3f7 Author: Ben Pastene <bpastene@chromium.org> Date: Thu Jun 28 21:09:48 2018 Reenable CrOSInterfaceTest in telemetry_unittests on VM test bot. TBR=jbudorick@chromium.org Bug: 836447 Change-Id: I0eb9eaf50b369c401072be20456558f66ed86ab7 Reviewed-on: https://chromium-review.googlesource.com/1118761 Reviewed-by: Ben Pastene <bpastene@chromium.org> Commit-Queue: Ben Pastene <bpastene@chromium.org> Cr-Commit-Position: refs/heads/master@{#571258} [modify] https://crrev.com/f60c42119b59b0816f723fcfd9070cef0ea4e3f7/testing/buildbot/chromium.fyi.json [modify] https://crrev.com/f60c42119b59b0816f723fcfd9070cef0ea4e3f7/testing/buildbot/test_suites.pyl
,
Jul 2
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/342af4f41f32d3703133fa2a5d046ac558a76af2 commit 342af4f41f32d3703133fa2a5d046ac558a76af2 Author: Ben Pastene <bpastene@chromium.org> Date: Mon Jul 02 22:00:19 2018 Disable telemetry's testBattOrTracing on cros VM test bot. TBR=jbudorick@chromium.org Bug: 836447 Change-Id: Ief0cbfbfe135265115d46e777d7701fe9515d485 Reviewed-on: https://chromium-review.googlesource.com/1123113 Reviewed-by: Ben Pastene <bpastene@chromium.org> Reviewed-by: John Budorick <jbudorick@chromium.org> Commit-Queue: Ben Pastene <bpastene@chromium.org> Cr-Commit-Position: refs/heads/master@{#572010} [modify] https://crrev.com/342af4f41f32d3703133fa2a5d046ac558a76af2/testing/buildbot/chromium.fyi.json [modify] https://crrev.com/342af4f41f32d3703133fa2a5d046ac558a76af2/testing/buildbot/test_suites.pyl
,
Aug 3
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2bc796bb69753a2e2b9ef22000513e3e58141852 commit 2bc796bb69753a2e2b9ef22000513e3e58141852 Author: Ben Pastene <bpastene@chromium.org> Date: Fri Aug 03 03:02:00 2018 Create a separate target for deps needed to deploy chrome to cros device. These deps were embedded in //:chromiumos_preflight. This moves them to a separate target at //chromeos:cros_chrome_deploy for easier reuse. And add it to telemetry tests. Bug: 836447 Change-Id: I786a6827c610c33b19c8027fb227026c22f3d319 Reviewed-on: https://chromium-review.googlesource.com/1157645 Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: John Budorick <jbudorick@chromium.org> Commit-Queue: Ben Pastene <bpastene@chromium.org> Cr-Commit-Position: refs/heads/master@{#580422} [modify] https://crrev.com/2bc796bb69753a2e2b9ef22000513e3e58141852/BUILD.gn [modify] https://crrev.com/2bc796bb69753a2e2b9ef22000513e3e58141852/chromeos/BUILD.gn [modify] https://crrev.com/2bc796bb69753a2e2b9ef22000513e3e58141852/tools/perf/chrome_telemetry_build/BUILD.gn
,
Aug 6
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/22d25c856a70980c251952e873d923d9a3516b52 commit 22d25c856a70980c251952e873d923d9a3516b52 Author: Ben Pastene <bpastene@chromium.org> Date: Mon Aug 06 20:18:34 2018 Deploy a locally built browser when running telemetry tests in cros VMs. This adds a --deploy option to the vm test runner and binds it to true every time we build the cros_vm_launcher. This assumes that everything that uses that target also builds chrome and also wants it deployed to the VM (which is true atm). Also rebalance telemetry unit test shard count on the FYI bot. Bug: 836447 Change-Id: I012932edac7bf40c369f69378e3ad365dd9e1ab6 Reviewed-on: https://chromium-review.googlesource.com/1155833 Commit-Queue: Ben Pastene <bpastene@chromium.org> Reviewed-by: John Budorick <jbudorick@chromium.org> Cr-Commit-Position: refs/heads/master@{#580969} [modify] https://crrev.com/22d25c856a70980c251952e873d923d9a3516b52/build/chromeos/create_vm_test_script.py [modify] https://crrev.com/22d25c856a70980c251952e873d923d9a3516b52/build/chromeos/run_vm_test.py [modify] https://crrev.com/22d25c856a70980c251952e873d923d9a3516b52/build/config/chromeos/rules.gni [modify] https://crrev.com/22d25c856a70980c251952e873d923d9a3516b52/chromeos/BUILD.gn [modify] https://crrev.com/22d25c856a70980c251952e873d923d9a3516b52/testing/buildbot/chromium.fyi.json [modify] https://crrev.com/22d25c856a70980c251952e873d923d9a3516b52/testing/buildbot/test_suites.pyl
,
Dec 10
is this fixed?
,
Dec 10
Yep, telemetry_unittests and telemetry_perf_unittests are running on the vm bot: https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.try/chromeos-amd64-generic-rel Was using bug 876494 mainly there at the end. Forgot this one was filed as well. |
|||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||
Comment 1 by thestig@chromium.org
, Apr 25 2018Components: Tests>Telemetry
Labels: -Sheriff-Chromium OS-Chrome
[21/21] benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.multitab:misc:typical24 failed unexpectedly 218.1414s: [ RUN ] system_health.memory_desktop/multitab:misc:typical24@{'case': 'multitab', 'group': 'misc'} Traceback (most recent call last): File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 123, in _RunStoryAndProcessErrorIfNeeded state.RunStory(results) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 311, in RunStory self._current_page.Run(self) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/page/__init__.py", line 91, in Run shared_state.page_test.RunNavigateSteps(self, current_tab) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/page/legacy_page_test.py", line 152, in RunNavigateSteps page.RunNavigateSteps(action_runner) File "/b/s/w/ir/tools/perf/page_sets/system_health/multi_tab_stories.py", line 30, in RunNavigateSteps new_tab.action_runner.Navigate(url) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 186, in Navigate timeout_in_seconds=timeout_in_seconds)) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 62, in _RunAction action.RunAction(self._tab) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/navigate.py", line 27, in RunAction tab.WaitForDocumentReadyStateToBeInteractiveOrBetter(time_left_in_seconds) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 103, in WaitForDocumentReadyStateToBeInteractiveOrBetter 'document.readyState == "complete"', timeout=timeout) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 239, in WaitForJavaScriptCondition return self._inspector_backend.WaitForJavaScriptCondition(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 302, in WaitForJavaScriptCondition e.message + '\n' + debug_message) TimeoutException: Timed out while waiting 59s for IsJavaScriptExpressionTrue. Console output: [ FAILED ] system_health.memory_desktop/multitab:misc:typical24@{'case': 'multitab', 'group': 'misc'} (205206 ms) [ PASSED ] 0 tests. [ FAILED ] 1 test, listed below: [ FAILED ] multitab:misc:typical24@{'case': 'multitab', 'group': 'misc'}