New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 791830 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Dec 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug



Sign in to add a comment

Remove suite:perf_v2

Project Member Reported by djkurtz@chromium.org, Dec 5 2017

Issue description

CL [0] renamed the telemetry_benchmarks suite to perf_v2 for  issue 197092 .

[0] https://gerrit.chromium.org/gerrit/45875

platform_MemoryMonitor was added to the suite by [1] for  issue 371418 .

[1] https://chromium-review.googlesource.com/200262

Eventually most other tests were removed from this suite until just two remained: 
 (1) platform_MemoryMonitor
 (2) telemetry_GpuTests

For a while this suite was still being scheduled in the Chrome OS HW Test lab.
At one point, in fact, there was a special lab pool (chromeperf) created for running these tests, and three builders that used the results: 
  daisy-chrome-perf
  lumpy-chrome-perf
  parrot-chrome-perf

In Nov 2014, the pool was removed and these builders were turned down as discussed in [2] and issue 434765.
[2] https://groups.google.com/a/google.com/d/msg/chromeos-performance/nrjoLjuOnsw/HbwzK0nW-9kJ

However, suite:perf_v2 was still being scheduled nightly and weekly.

In June 2016, it was reported in issue 619041 and [3] that platform_MemoryMonitor was generating a huge number of timeseries.
This led to the decision to de-scheduled suite:perf_v2 [4] after moving its only other test (telemetry_GpuTests) out of suite:perf_v2 [5].
[3] https://groups.google.com/a/google.com/d/msg/chromeos-performance/KmJQsrCdrsA/zUNo23jFAQAJ
[4] https://chromium-review.googlesource.com/#/c/chromiumos/third_party/autotest/+/351460/
[5] https://chromium-review.googlesource.com/243654

However, suite:perf_v2 still exists to this day.
Since the current recommendation is to run all performance tests in the bvt-perbuild suite anyway, let's just remove the suite and any references to it.

Its only remaining test, platform_MemoryMonitor, is currently broken ( issue 791829 ), however, previously it was decided to keep the test in the tree so if we want to run it or change it and/or consider putting it back in the suite (and then monitor), we can do that [6].  Therefore, there is an attempt to fix the test for now [7], but it has not yet been added back to any suite.
[6] https://groups.google.com/a/google.com/d/msg/chromeos-performance/KmJQsrCdrsA/kW6hGJ_mAQAJ
[7] https://chromium-review.googlesource.com/#/c/chromiumos/third_party/autotest/+/798818/
 
Project Member

Comment 1 by bugdroid1@chromium.org, Dec 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/1a1a87f34416416cc77e4f7c28c8d3be041d3635

commit 1a1a87f34416416cc77e4f7c28c8d3be041d3635
Author: Daniel Kurtz <djkurtz@chromium.org>
Date: Wed Dec 06 04:22:32 2017

[autotest] Remove suite:perf_v2

This suite only has one test remaining in it (platform_MemoryMonitor).
That test creates very many timeseries which weren't particularly useful
on a performance dashboard, and, hence the entire suite was de-scheduled
from running in the lab.

In any case, the recommend way to track performance is now by tests
running in the suite:bvt-perbuild.

Therefore, let's completely kill the vestiges of this suite.

Signed-off-by: Daniel Kurtz <djkurtz@chromium.org>

BUG= chromium:791830 
TEST=none

Change-Id: Ie073bcf3694111c7bad69d627ae6826485c2ee28
Reviewed-on: https://chromium-review.googlesource.com/807638
Commit-Ready: Daniel Kurtz <djkurtz@chromium.org>
Tested-by: Daniel Kurtz <djkurtz@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/1a1a87f34416416cc77e4f7c28c8d3be041d3635/server/site_tests/telemetry_AFDOGenerate/telemetry_AFDOGenerate.py
[modify] https://crrev.com/1a1a87f34416416cc77e4f7c28c8d3be041d3635/site_utils/run_suite.py
[modify] https://crrev.com/1a1a87f34416416cc77e4f7c28c8d3be041d3635/site_utils/attribute_whitelist.txt
[modify] https://crrev.com/1a1a87f34416416cc77e4f7c28c8d3be041d3635/client/site_tests/platform_MemoryMonitor/control
[delete] https://crrev.com/21b670931c491e4d578efff00c0593ed2b2161f9/test_suites/control.perf_v2

Status: Verified (was: Started)

Sign in to add a comment