New issue
Advanced search Search tips

Issue 676188 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug



Sign in to add a comment

enable media_perftests to run continuously and be graphed

Project Member Reported by crouleau@chromium.org, Dec 21 2016

Issue description

I think these may have been turned off for almost 2 years. Since the switch to recipes. We need to modify src/testing/buildbot/chromium.perf.json (or the scripts that automatically write that file) to start the tests to run again.

relevant background CLs: https://codereview.chromium.org/873403002 https://codereview.chromium.org/890653002
 
Cc: jrumm...@chromium.org
They were turned off over two years ago: https://bugs.chromium.org/p/chromium/issues/detail?id=392620
The bug to re-enable load_library_perftests and cc_perftests and media_perftests was marked fixed even though media_perftests were never re-enabled.

Prabhu tried to get them re-enabled, but he was confused. the tests are not telemetry tests, so he was talking to the wrong people: https://bugs.chromium.org/p/chromium/issues/detail?id=518964

I should be able to trivially re-enable them so long as they are in good working condition.

Thanks John for the help with the investigation!
Status: Assigned (was: Untriaged)
I'm going to see if I can get the owner of crbug/392620 to re-enable them since he forgot to do that (I think). If I can't get him to do it then I will.

After we re-enable them, we may have to talk with the chromeperf dashboard folks (sullivan@'s team) to get "monitoring" turned back on for them. Then they will be on the dashboard again.
Summary: enable media_perftests to run continuously and be graphed (was: enable media_perftests to run continuously)
Thanks for digging in!
I re-enabled them, but I got an error, so I am going to revert my change.

Additional test environment:
    CHROME_DEVEL_SANDBOX=/opt/chromium/chrome_sandbox
    LANG=en_US.UTF-8
Command: c:\b\depot_tools\python276_bin\python.exe ..\..\testing\scripts\run_gtest_perf_test.py media_perftests --isolated-script-test-output=c:\b\s\w\io8zdfrb\output.json --isolated-script-test-chartjson-output=c:\b\s\w\io8zdfrb\chartjson-output.json

c:\b\depot_tools\python276_bin\python.exe: can't open file '..\..\testing\scripts\run_gtest_perf_test.py': [Errno 2] No such file or directory
Project Member

Comment 8 by bugdroid1@chromium.org, Jan 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/98e3953123e42e1fe2d82ab301eb09f4a24bfca5

commit 98e3953123e42e1fe2d82ab301eb09f4a24bfca5
Author: crouleau <crouleau@chromium.org>
Date: Thu Jan 12 01:29:48 2017

Enable media_perftests on linux perfbot.

This required
1. Adding media_perftests to gn_isolate_map so that isolate can understand how
   to run them
2. Adding data dependencies to media_perftests target that are needed by the
   isolate script.
3. Added media_perftests to generate_perf_json.py configuration file.

A previous version of this CL (https://codereview.chromium.org/2595123002) got
reverted because I didn't do step 2. For that CL, I only tested media_perftests
in a local run. This time, I also ran using isolates as well. (Thanks eyaich@!)
See test run: https://chromium-swarm.appspot.com/task?id=3390ffce2596c810&refresh=10&request_detail=true&show_raw=1

Follow up CL(s) will also add support on Mac, Windows, and Android.

BUG= 676188 , 392620 

Review-Url: https://codereview.chromium.org/2617133004
Cr-Commit-Position: refs/heads/master@{#443110}

[modify] https://crrev.com/98e3953123e42e1fe2d82ab301eb09f4a24bfca5/media/BUILD.gn
[modify] https://crrev.com/98e3953123e42e1fe2d82ab301eb09f4a24bfca5/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/98e3953123e42e1fe2d82ab301eb09f4a24bfca5/testing/buildbot/gn_isolate_map.pyl
[modify] https://crrev.com/98e3953123e42e1fe2d82ab301eb09f4a24bfca5/tools/perf/generate_perf_json.py

Okay, they are enabled for linux. Next I will enable them for Windows and Mac and Android.

Dale, do we need them enabled for CrOS? It is unclear how to do that, so I may need to reach out to Speed Infra team for help if we need CrOS.
It'd be nice, but Android && Linux are a good proxy if that's difficult.
Cc: prabhur@chromium.org
 Issue 518964  has been merged into this issue.
Project Member

Comment 13 by bugdroid1@chromium.org, Jan 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ff5b9223b5841e297444ddfd05b7c4380773d764

commit ff5b9223b5841e297444ddfd05b7c4380773d764
Author: crouleau <crouleau@chromium.org>
Date: Sat Jan 21 00:48:47 2017

Enable media_perftests continuous runs on windows and mac.

It is already running continuously on linux since
https://codereview.chromium.org/2617133004

BUG= 676188 , 392620 

Review-Url: https://codereview.chromium.org/2627343002
Cr-Commit-Position: refs/heads/master@{#445230}

[modify] https://crrev.com/ff5b9223b5841e297444ddfd05b7c4380773d764/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/ff5b9223b5841e297444ddfd05b7c4380773d764/tools/perf/generate_perf_json.py

Still haven't re-enabled for CrOS or Android. Will likely not do that work until next quarter still other things have priority.
*Still haven't _enabled_. I don't think these tests ever ran on Android or CrOS previously.
Blockedon: 709161
issue 709161 has progress for CrOS. Currently in research stage.

issue 714812 is related as well.
Labels: Performance-Responsiveness
Blockedon: -709161
Status: Fixed (was: Assigned)
I'm marking this fixed. Remaining work is in separate bugs: 

Android:  issue 772179 
CrOS: issue 772181

Sign in to add a comment