Create an tools/perf/contrib/ directory for experimental benchmarks |
||||
Issue descriptionCreate an tools/perf/experimental_benchmarks directory for experimental benchmarks. Requirements: 1) All the experimental_benchmarks will not be scheduled to run on our perf bots 2) We will move the all benchmarks that are meant to be run locally or on cluster telemetry to this directory. 3) For the one that is runnable locally, users can trigger with "tools/perf/run_benchmark <...>" as other benchmarks. 4) Benchmarking team will take no responsibility for actively fixing breakages in this directory. Users who need the tool can make the fix themselves, as long as they don't use deprecated /legacy APIs. 5) Every benchmark in this directory must have clear ownership (individuals or team). Occasionally, we will remove old benchmarks whose owner no longer works on the project.
,
Apr 14 2017
+jbroman: FYI once this directory is in you can add the extensions benchmark you were working on if you still need it.
,
Apr 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/09cd8622560496d63dd16d0b10a7c515dbc5c3ff commit 09cd8622560496d63dd16d0b10a7c515dbc5c3ff Author: nednguyen <nednguyen@google.com> Date: Mon Apr 17 21:12:24 2017 [tools/perf] Remove default values of benchmark_dirs and top_level_dir in chromium_config module Part of this CL also moves GetChromiumSrcDir & GetTelemetryDir methods to tools.perf.core.path_util module Motivation: to move experiemental benchmarks to a separate folder & still make run_benchmarks be able to detect them, I will need to modify run_benchmarks to also include experiemental benchmark folder in |benchmark_dirs| parameter. Upon doing that, I find that there is no need to set the default the value of benchmark_dirs & top_level_dir. BUG= 709063 TBR=perezju@chromium.org Review-Url: https://codereview.chromium.org/2821663002 Cr-Commit-Position: refs/heads/master@{#465016} [modify] https://crrev.com/09cd8622560496d63dd16d0b10a7c515dbc5c3ff/tools/perf/chrome_telemetry_build/chromium_config.py [modify] https://crrev.com/09cd8622560496d63dd16d0b10a7c515dbc5c3ff/tools/perf/core/path_util.py [modify] https://crrev.com/09cd8622560496d63dd16d0b10a7c515dbc5c3ff/tools/perf/core/perf_data_generator.py [modify] https://crrev.com/09cd8622560496d63dd16d0b10a7c515dbc5c3ff/tools/perf/run_benchmark
,
Apr 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1e0bcd315fc5917a83cc3bdd2e1abf0d644e5119 commit 1e0bcd315fc5917a83cc3bdd2e1abf0d644e5119 Author: alexmos <alexmos@chromium.org> Date: Mon Apr 17 22:06:52 2017 Revert of [tools/perf] Remove default values of benchmark_dirs and top_level_dir in chromium_config module (patchset #2 id:20001 of https://codereview.chromium.org/2821663002/ ) Reason for revert: Suspected to be breaking various tests, e.g.: https://build.chromium.org/p/chromium.gpu/builders/Mac%20Release%20%28Intel%29/builds/62430 https://build.chromium.org/p/chromium.gpu/builders/Linux%20Release%20%28NVIDIA%29/builds/100927 https://build.chromium.org/p/chromium.mac/builders/Mac10.9%20Tests/builds/39799 The logs show a python failure similar to Traceback (most recent call last): File "../../content/test/gpu/run_unittests.py", line 15, in <module> path_util.SetupTelemetryPaths() File "/b/swarm_slave/w/ir/content/test/gpu/gpu_tests/path_util.py", line 31, in SetupTelemetryPaths from chrome_telemetry_build import chromium_config File "/b/swarm_slave/w/ir/tools/perf/chrome_telemetry_build/chromium_config.py", line 7, in <module> from core import path_util ImportError: No module named core Original issue's description: > [tools/perf] Remove default values of benchmark_dirs and top_level_dir in chromium_config module > > Part of this CL also moves GetChromiumSrcDir & GetTelemetryDir methods to > tools.perf.core.path_util module > > Motivation: to move experiemental benchmarks to a separate folder & still > make run_benchmarks be able to detect them, I will need to modify > run_benchmarks to also include experiemental benchmark folder in > |benchmark_dirs| parameter. Upon doing that, I find that there is no need > to set the default the value of benchmark_dirs & top_level_dir. > > BUG= 709063 > TBR=perezju@chromium.org > > Review-Url: https://codereview.chromium.org/2821663002 > Cr-Commit-Position: refs/heads/master@{#465016} > Committed: https://chromium.googlesource.com/chromium/src/+/09cd8622560496d63dd16d0b10a7c515dbc5c3ff TBR=martiniss@chromium.org,ashleymarie@chromium.org,perezju@chromium.org,nednguyen@google.com # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 709063 Review-Url: https://codereview.chromium.org/2823063003 Cr-Commit-Position: refs/heads/master@{#465038} [modify] https://crrev.com/1e0bcd315fc5917a83cc3bdd2e1abf0d644e5119/tools/perf/chrome_telemetry_build/chromium_config.py [modify] https://crrev.com/1e0bcd315fc5917a83cc3bdd2e1abf0d644e5119/tools/perf/core/path_util.py [modify] https://crrev.com/1e0bcd315fc5917a83cc3bdd2e1abf0d644e5119/tools/perf/core/perf_data_generator.py [modify] https://crrev.com/1e0bcd315fc5917a83cc3bdd2e1abf0d644e5119/tools/perf/run_benchmark
,
Apr 18 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f171cc50af563f712121e80872643cd733cc9b76 commit f171cc50af563f712121e80872643cd733cc9b76 Author: nednguyen <nednguyen@google.com> Date: Tue Apr 18 22:22:46 2017 [tools/perf] Remove default values of benchmark_dirs and top_level_dir in chromium_config module (Reland) Part of this CL also moves GetChromiumSrcDir & GetTelemetryDir methods to tools.perf.core.path_util module Motivation: to move experiemental benchmarks to a separate folder & still make run_benchmarks be able to detect them, I will need to modify run_benchmarks to also include experiemental benchmark folder in |benchmark_dirs| parameter. Upon doing that, I find that there is no need to set the default the value of benchmark_dirs & top_level_dir. BUG= 709063 R=perezju@chromium.org, kbr@chromium.org CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Review-Url: https://codereview.chromium.org/2824053002 Cr-Commit-Position: refs/heads/master@{#465395} [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/chrome/test/BUILD.gn [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/content/test/gpu/gpu_tests/path_util.py [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/tools/perf/chrome_telemetry_build/BUILD.gn [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/tools/perf/chrome_telemetry_build/chromium_config.py [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/tools/perf/core/path_util.py [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/tools/perf/core/perf_data_generator.py [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/tools/perf/record_android_profile.py [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/tools/perf/record_wpr [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/tools/perf/run_benchmark [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/tools/perf/run_telemetry_tests [modify] https://crrev.com/f171cc50af563f712121e80872643cd733cc9b76/tools/perf/run_tests
,
Apr 20 2017
We decide to rename this to tools/perf/contrib/ to make the support contract clearer.
,
Apr 20 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/866b6748f60027557ed0db184d85a755252e1c15 commit 866b6748f60027557ed0db184d85a755252e1c15 Author: nednguyen <nednguyen@google.com> Date: Thu Apr 20 21:27:35 2017 Create tools/perf/contrib directory BUG= 709063 Review-Url: https://codereview.chromium.org/2827393002 Cr-Commit-Position: refs/heads/master@{#466141} [add] https://crrev.com/866b6748f60027557ed0db184d85a755252e1c15/tools/perf/contrib/README.md
,
Apr 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4 commit bfe47df90b3d1d770f9a2dcef1bd83b966327ce4 Author: nednguyen <nednguyen@google.com> Date: Wed Apr 26 16:49:14 2017 Move blink_perf.xml_http_request benchmark to tools/perf/contrib/ directory * I verified that people still be able to run "./run_benchmark --browser=system blink_perf.xml_http_request" locally. * Try bot run are in: https://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_12_perf_bisect/builds/149 Other effects of this: _ People can run blink_perf.xml_http_request locally without needing to pass "--also-run-disabled-benchmark" _ The benchmarks' no-op run is no longer scheduled in perf waterfall, hence the size reduction to the testing/buildbot/chrome.perf.* files BUG= 709063 Review-Url: https://codereview.chromium.org/2840213003 Cr-Commit-Position: refs/heads/master@{#467347} [modify] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/testing/buildbot/chromium.perf.json [modify] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/tools/perf/benchmark.csv [modify] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/tools/perf/benchmarks/blink_perf.py [add] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/tools/perf/contrib/__init__.py [add] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/tools/perf/contrib/blink_perf_xml_http_request/OWNERS [add] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/tools/perf/contrib/blink_perf_xml_http_request/__init__.py [add] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/tools/perf/contrib/blink_perf_xml_http_request/blink_perf_xml_http_request.py [modify] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/tools/perf/core/benchmark_finders.py [modify] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/tools/perf/core/path_util.py [modify] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/tools/perf/core/trybot_command.py [modify] https://crrev.com/bfe47df90b3d1d770f9a2dcef1bd83b966327ce4/tools/perf/run_benchmark
,
Apr 26 2017
This is fixed. The next step is to move local benchmarks & cluster telemetry benchmarks to contrib/ directory.
,
May 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/aa3196de10db663af40f600d476dfb77587994c3 commit aa3196de10db663af40f600d476dfb77587994c3 Author: nednguyen <nednguyen@google.com> Date: Thu May 11 10:21:38 2017 Add PRESUBMIT check to enforce the rule of tools/perf/contrib/ directory This PRESUBMIT check makes sure: 1) All new files to tools/perf/contrib/ must belong to some subdirectory. 2) All the subdirectory of tools/perf/contrib/ must have at least 2 owners. BUG= 709063 Review-Url: https://codereview.chromium.org/2869273003 Cr-Commit-Position: refs/heads/master@{#470900} [add] https://crrev.com/aa3196de10db663af40f600d476dfb77587994c3/tools/perf/contrib/PRESUBMIT.py |
||||
►
Sign in to add a comment |
||||
Comment 1 by nedngu...@google.com
, Apr 14 2017Status: Started (was: Untriaged)