enable media_perftests to run continuously and be graphed |
|||||||
Issue descriptionI think these may have been turned off for almost 2 years. Since the switch to recipes. We need to modify src/testing/buildbot/chromium.perf.json (or the scripts that automatically write that file) to start the tests to run again. relevant background CLs: https://codereview.chromium.org/873403002 https://codereview.chromium.org/890653002
,
Dec 21 2016
I'm going to see if I can get the owner of crbug/392620 to re-enable them since he forgot to do that (I think). If I can't get him to do it then I will. After we re-enable them, we may have to talk with the chromeperf dashboard folks (sullivan@'s team) to get "monitoring" turned back on for them. Then they will be on the dashboard again.
,
Dec 21 2016
,
Dec 21 2016
Thanks for digging in!
,
Dec 22 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/feb10cb7e15bb5378e6664722e7ae4b082bf9191 commit feb10cb7e15bb5378e6664722e7ae4b082bf9191 Author: crouleau <crouleau@chromium.org> Date: Thu Dec 22 19:36:17 2016 Enable media_perftests to run on Windows, Mac, and Linux. BUG= 676188 , 392620 Review-Url: https://codereview.chromium.org/2595123002 Cr-Commit-Position: refs/heads/master@{#440473} [modify] https://crrev.com/feb10cb7e15bb5378e6664722e7ae4b082bf9191/testing/buildbot/chromium.perf.json [modify] https://crrev.com/feb10cb7e15bb5378e6664722e7ae4b082bf9191/testing/buildbot/gn_isolate_map.pyl [modify] https://crrev.com/feb10cb7e15bb5378e6664722e7ae4b082bf9191/tools/perf/generate_perf_json.py
,
Dec 23 2016
I re-enabled them, but I got an error, so I am going to revert my change.
Additional test environment:
CHROME_DEVEL_SANDBOX=/opt/chromium/chrome_sandbox
LANG=en_US.UTF-8
Command: c:\b\depot_tools\python276_bin\python.exe ..\..\testing\scripts\run_gtest_perf_test.py media_perftests --isolated-script-test-output=c:\b\s\w\io8zdfrb\output.json --isolated-script-test-chartjson-output=c:\b\s\w\io8zdfrb\chartjson-output.json
c:\b\depot_tools\python276_bin\python.exe: can't open file '..\..\testing\scripts\run_gtest_perf_test.py': [Errno 2] No such file or directory
,
Dec 27 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c05ba2d167e60d43fe59b60bac34eacb220446a5 commit c05ba2d167e60d43fe59b60bac34eacb220446a5 Author: crouleau <crouleau@chromium.org> Date: Tue Dec 27 20:06:25 2016 Revert "Enable media_perftests to run on Windows, Mac, and Linux." This reverts commit feb10cb7e15bb5378e6664722e7ae4b082bf9191. BUG= 676188 , 392620 TBR=dpranke@chromium.org Review-Url: https://codereview.chromium.org/2602543002 Cr-Commit-Position: refs/heads/master@{#440776} [modify] https://crrev.com/c05ba2d167e60d43fe59b60bac34eacb220446a5/testing/buildbot/chromium.perf.json [modify] https://crrev.com/c05ba2d167e60d43fe59b60bac34eacb220446a5/testing/buildbot/gn_isolate_map.pyl [modify] https://crrev.com/c05ba2d167e60d43fe59b60bac34eacb220446a5/tools/perf/generate_perf_json.py
,
Jan 12 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/98e3953123e42e1fe2d82ab301eb09f4a24bfca5 commit 98e3953123e42e1fe2d82ab301eb09f4a24bfca5 Author: crouleau <crouleau@chromium.org> Date: Thu Jan 12 01:29:48 2017 Enable media_perftests on linux perfbot. This required 1. Adding media_perftests to gn_isolate_map so that isolate can understand how to run them 2. Adding data dependencies to media_perftests target that are needed by the isolate script. 3. Added media_perftests to generate_perf_json.py configuration file. A previous version of this CL (https://codereview.chromium.org/2595123002) got reverted because I didn't do step 2. For that CL, I only tested media_perftests in a local run. This time, I also ran using isolates as well. (Thanks eyaich@!) See test run: https://chromium-swarm.appspot.com/task?id=3390ffce2596c810&refresh=10&request_detail=true&show_raw=1 Follow up CL(s) will also add support on Mac, Windows, and Android. BUG= 676188 , 392620 Review-Url: https://codereview.chromium.org/2617133004 Cr-Commit-Position: refs/heads/master@{#443110} [modify] https://crrev.com/98e3953123e42e1fe2d82ab301eb09f4a24bfca5/media/BUILD.gn [modify] https://crrev.com/98e3953123e42e1fe2d82ab301eb09f4a24bfca5/testing/buildbot/chromium.perf.json [modify] https://crrev.com/98e3953123e42e1fe2d82ab301eb09f4a24bfca5/testing/buildbot/gn_isolate_map.pyl [modify] https://crrev.com/98e3953123e42e1fe2d82ab301eb09f4a24bfca5/tools/perf/generate_perf_json.py
,
Jan 12 2017
Okay, they are enabled for linux. Next I will enable them for Windows and Mac and Android. Dale, do we need them enabled for CrOS? It is unclear how to do that, so I may need to reach out to Speed Infra team for help if we need CrOS.
,
Jan 12 2017
It'd be nice, but Android && Linux are a good proxy if that's difficult.
,
Jan 17 2017
,
Jan 18 2017
Graphs work for linux: https://chromeperf.appspot.com/report?sid=6be379dd118ce33f527b87d069f31daa7e1acdb87972388e084d1b758337195c
,
Jan 21 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ff5b9223b5841e297444ddfd05b7c4380773d764 commit ff5b9223b5841e297444ddfd05b7c4380773d764 Author: crouleau <crouleau@chromium.org> Date: Sat Jan 21 00:48:47 2017 Enable media_perftests continuous runs on windows and mac. It is already running continuously on linux since https://codereview.chromium.org/2617133004 BUG= 676188 , 392620 Review-Url: https://codereview.chromium.org/2627343002 Cr-Commit-Position: refs/heads/master@{#445230} [modify] https://crrev.com/ff5b9223b5841e297444ddfd05b7c4380773d764/testing/buildbot/chromium.perf.json [modify] https://crrev.com/ff5b9223b5841e297444ddfd05b7c4380773d764/tools/perf/generate_perf_json.py
,
Mar 1 2017
Still haven't re-enabled for CrOS or Android. Will likely not do that work until next quarter still other things have priority.
,
Mar 1 2017
*Still haven't _enabled_. I don't think these tests ever ran on Android or CrOS previously.
,
May 5 2017
issue 709161 has progress for CrOS. Currently in research stage. issue 714812 is related as well.
,
May 5 2017
,
Oct 5 2017
I'm marking this fixed. Remaining work is in separate bugs: Android: issue 772179 CrOS: issue 772181 |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by crouleau@chromium.org
, Dec 21 2016