New Swarming custom trigger logic (OBBS) |
||||||||
Issue descriptionRough steps: 1) Enable custom trigger script in swarming recipe module 2) Add device_aware_swarming.py custom trigger script 3) Plumb custom trigger script into chromium_tests
,
Aug 24 2017
,
Sep 1 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/d9569f390d231faececbadfbc29b4adc5b5959ab commit d9569f390d231faececbadfbc29b4adc5b5959ab Author: Stephen Martinis <martiniss@chromium.org> Date: Fri Sep 01 20:18:12 2017 Add trigger_script to chromium_tests spec generation Bug: 758630 Change-Id: I6cf0a94a0784359a75749b8a254e385f1caf0370 Reviewed-on: https://chromium-review.googlesource.com/629936 Commit-Queue: Stephen Martinis <martiniss@chromium.org> Reviewed-by: John Budorick <jbudorick@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> [add] https://crrev.com/d9569f390d231faececbadfbc29b4adc5b5959ab/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_isolated_script.expected/swarming_trigger_script_invalid.json [add] https://crrev.com/d9569f390d231faececbadfbc29b4adc5b5959ab/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_gtest.expected/trigger_script_invalid.json [modify] https://crrev.com/d9569f390d231faececbadfbc29b4adc5b5959ab/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_gtest.py [modify] https://crrev.com/d9569f390d231faececbadfbc29b4adc5b5959ab/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_isolated_script.py [modify] https://crrev.com/d9569f390d231faececbadfbc29b4adc5b5959ab/scripts/slave/recipe_modules/chromium_tests/steps.py [add] https://crrev.com/d9569f390d231faececbadfbc29b4adc5b5959ab/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_gtest.expected/trigger_script.json [add] https://crrev.com/d9569f390d231faececbadfbc29b4adc5b5959ab/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_isolated_script.expected/swarming_trigger_script.json
,
Sep 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/b7eeeb36cc5d7dfc9cb3e659531dff9fd7784d6d commit b7eeeb36cc5d7dfc9cb3e659531dff9fd7784d6d Author: Stephen Martinis <martiniss@chromium.org> Date: Tue Sep 05 21:06:26 2017 chromium_tests: Prepend arguments Followup comment from https://chromium-review.googlesource.com/c/chromium/tools/build/+/629437, it makes more sense to prepend the arguments. Makes it easier for the script to parse the arguments usually as well. Bug: 758630 Change-Id: If8975fb13c7c41994e90824e43ea9d25ad6001f8 Reviewed-on: https://chromium-review.googlesource.com/647377 Commit-Queue: Stephen Martinis <martiniss@chromium.org> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> [modify] https://crrev.com/b7eeeb36cc5d7dfc9cb3e659531dff9fd7784d6d/scripts/slave/recipe_modules/swarming/examples/full.expected/isolated_script_with_custom_trigger_script.json [modify] https://crrev.com/b7eeeb36cc5d7dfc9cb3e659531dff9fd7784d6d/scripts/slave/recipe_modules/swarming/api.py
,
Sep 8 2017
,
Sep 13 2017
,
Sep 19 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/aa3f69a7f55bdfd18b3c4d69d2bcf631c093e615 commit aa3f69a7f55bdfd18b3c4d69d2bcf631c093e615 Author: Stephen Martinis <martiniss@chromium.org> Date: Tue Sep 19 02:30:34 2017 Add new perf test bot This builder will be used to test the new one buildbot step logic. Bug:758630 Change-Id: Iadc1ee42ee7a5995ef13feb1d5dc7cc7cd579ab8 Reviewed-on: https://chromium-review.googlesource.com/671680 Commit-Queue: Stephen Martinis <martiniss@chromium.org> Reviewed-by: David Tu <dtu@chromium.org> [modify] https://crrev.com/aa3f69a7f55bdfd18b3c4d69d2bcf631c093e615/scripts/slave/recipe_modules/chromium_tests/chromium_perf_fyi.py [modify] https://crrev.com/aa3f69a7f55bdfd18b3c4d69d2bcf631c093e615/scripts/slave/recipe_modules/chromium_tests/tests/api/archive_build.expected/linux_builder.json [modify] https://crrev.com/aa3f69a7f55bdfd18b3c4d69d2bcf631c093e615/masters/master.chromium.perf.fyi/master.cfg [modify] https://crrev.com/aa3f69a7f55bdfd18b3c4d69d2bcf631c093e615/masters/master.chromium.perf.fyi/slaves.cfg
,
Sep 19 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/834a9a99a878ae39d9cfe061f14ee6c640e8e5a6 commit 834a9a99a878ae39d9cfe061f14ee6c640e8e5a6 Author: Stephen Martinis <martiniss@chromium.org> Date: Tue Sep 19 10:41:59 2017 Add custom perf device trigger script This will be used by the one buildbot step effort for perf builders. This CL shouldn't have any effect when it lands. Bug: 758630 Change-Id: I2e8434a975be2908427e5c6fd0091a5f1490f5bf Reviewed-on: https://chromium-review.googlesource.com/629962 Commit-Queue: Ned Nguyen <nednguyen@google.com> Reviewed-by: Emily Hanley <eyaich@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Cr-Commit-Position: refs/heads/master@{#502820} [add] https://crrev.com/834a9a99a878ae39d9cfe061f14ee6c640e8e5a6/tools/perf/perf_device_trigger.py [add] https://crrev.com/834a9a99a878ae39d9cfe061f14ee6c640e8e5a6/tools/perf/perf_device_trigger_unittest.py
,
Sep 19 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/2cf9b0a6b5cecd909d08667fc8f9fa971867e4b9 commit 2cf9b0a6b5cecd909d08667fc8f9fa971867e4b9 Author: Stephen Martinis <martiniss@chromium.org> Date: Tue Sep 19 22:56:22 2017 Fix perf bot id I committed the build side CL before the bot config had landed, turns out the final bot id I'll be using is different. Bug: 758630 Change-Id: Id947a8cb8b925448f6835e71d2b5fcc01454a67b Reviewed-on: https://chromium-review.googlesource.com/673749 Reviewed-by: David Tu <dtu@chromium.org> Commit-Queue: Stephen Martinis <martiniss@chromium.org> [modify] https://crrev.com/2cf9b0a6b5cecd909d08667fc8f9fa971867e4b9/masters/master.chromium.perf.fyi/slaves.cfg
,
Sep 19 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infra/infra_internal/+/06220d30c3cd5dd329201828602f8ae0a5ca96b3 commit 06220d30c3cd5dd329201828602f8ae0a5ca96b3 Author: Stephen Martinis <martiniss@google.com> Date: Tue Sep 19 23:02:50 2017
,
Sep 19 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/master-manager/+/69dfe2a202ae326cec7534f68f93980d16147657 commit 69dfe2a202ae326cec7534f68f93980d16147657 Author: Stephen Martinis <martiniss@google.com> Date: Tue Sep 19 23:47:49 2017
,
Sep 20 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/cc36dd1f28869d0ce1ddca5265ec89e12dec3529 commit cc36dd1f28869d0ce1ddca5265ec89e12dec3529 Author: Stephen Martinis <martiniss@google.com> Date: Wed Sep 20 01:12:05 2017
,
Sep 25 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0b18efa102845275f904b40c4bb3d5c454b961d9 commit 0b18efa102845275f904b40c4bb3d5c454b961d9 Author: Stephen Martinis <martiniss@chromium.org> Date: Mon Sep 25 22:55:12 2017 Add new telemetry_perf_tests isolate and script This CL adds a new isolate called telemetry_perf_tests_new. This is a temporary name, so that we can test it out while keeping the old isolate name. Eventually, this isolate will probably replace the existing isolate. The main difference between this and the current telemetry_perf_tests is the script it's running, which is named run_multiple_telemetry_benchmarks_as_googletest.py. As the name suggests, the script runs multiple telemetry benchmarks, which is the plan for a single swarming task in the future. Bug: 758630 Change-Id: If3a8afbb22ae2dbf0ce9e69f9036449d47914d03 Reviewed-on: https://chromium-review.googlesource.com/668111 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Reviewed-by: Emily Hanley <eyaich@chromium.org> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#504196} [modify] https://crrev.com/0b18efa102845275f904b40c4bb3d5c454b961d9/chrome/test/BUILD.gn [modify] https://crrev.com/0b18efa102845275f904b40c4bb3d5c454b961d9/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/0b18efa102845275f904b40c4bb3d5c454b961d9/testing/buildbot/gn_isolate_map.pyl [modify] https://crrev.com/0b18efa102845275f904b40c4bb3d5c454b961d9/testing/scripts/OWNERS [add] https://crrev.com/0b18efa102845275f904b40c4bb3d5c454b961d9/testing/scripts/run_multiple_telemetry_benchmarks_as_googletest.py [modify] https://crrev.com/0b18efa102845275f904b40c4bb3d5c454b961d9/testing/scripts/run_telemetry_benchmark_as_googletest.py [modify] https://crrev.com/0b18efa102845275f904b40c4bb3d5c454b961d9/tools/perf/chromium.perf.fyi.extras.json
,
Sep 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/aa3c707d0789cb6c49063ebd88f03765876b0869 commit aa3c707d0789cb6c49063ebd88f03765876b0869 Author: Stephen Martinis <martiniss@chromium.org> Date: Tue Sep 26 22:21:50 2017 Fix run_multiple_telemetry_benchmarks_as_googletest.py compile targets https://build.chromium.org/p/chromium.perf.fyi/builders/One%20Buildbot%20Step%20Test%20Builder is failing because the isolate target isn't being built on Linux Builder. Bug: 758630 Change-Id: I793fed5a265b21762ba0a0cb6a669a3ba79f7ef6 Reviewed-on: https://chromium-review.googlesource.com/683462 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#504500} [modify] https://crrev.com/aa3c707d0789cb6c49063ebd88f03765876b0869/testing/scripts/run_multiple_telemetry_benchmarks_as_googletest.py
,
Sep 28 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b08d23835ed110d4f8e2e65e8eed60ab72ef77a6 commit b08d23835ed110d4f8e2e65e8eed60ab72ef77a6 Author: Stephen Martinis <martiniss@chromium.org> Date: Thu Sep 28 01:57:32 2017 //tools/perf: Fix chromium.perf.fyi.json Adds in the dimension set, to make sure we run on our bot. Also fixes the compile target, which might fix Linux Builder not generating the proper isolate. NOTRY=true Bug: 758630 Change-Id: I1f837b1c8854bce0de1bee9c4abd2103aefb661a Reviewed-on: https://chromium-review.googlesource.com/688217 Commit-Queue: Stephen Martinis <martiniss@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#504865} [modify] https://crrev.com/b08d23835ed110d4f8e2e65e8eed60ab72ef77a6/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/b08d23835ed110d4f8e2e65e8eed60ab72ef77a6/tools/perf/chromium.perf.fyi.extras.json
,
Sep 29 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ff9d46ff7d4406fcb3544a2eb5b14f14e896c1be commit ff9d46ff7d4406fcb3544a2eb5b14f14e896c1be Author: Stephen Martinis <martiniss@chromium.org> Date: Fri Sep 29 01:56:57 2017 //tools/perf: Add compile target to linux builder Also creates cleaner way to add additional compile targets to builders. NOTRY=true # win chromium rel ng seems flaky Bug: 758630 Change-Id: I03f2b599d8f953f21d3a6d0e57d8db90d8d13999 Reviewed-on: https://chromium-review.googlesource.com/691065 Commit-Queue: Stephen Martinis <martiniss@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#505251} [modify] https://crrev.com/ff9d46ff7d4406fcb3544a2eb5b14f14e896c1be/testing/buildbot/chromium.perf.json [modify] https://crrev.com/ff9d46ff7d4406fcb3544a2eb5b14f14e896c1be/tools/perf/core/perf_data_generator.py
,
Sep 29 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/782c13fbfb92e2295b25d9c66c7dda19c15653e2 commit 782c13fbfb92e2295b25d9c66c7dda19c15653e2 Author: Stephen Martinis <martiniss@chromium.org> Date: Fri Sep 29 02:30:14 2017 Revert "Fix run_multiple_telemetry_benchmarks_as_googletest.py compile targets" This reverts commit aa3c707d0789cb6c49063ebd88f03765876b0869. Reason for revert: Didn't do anything Original change's description: > Fix run_multiple_telemetry_benchmarks_as_googletest.py compile targets > > https://build.chromium.org/p/chromium.perf.fyi/builders/One%20Buildbot%20Step%20Test%20Builder > is failing because the isolate target isn't being built on Linux Builder. > > Bug: 758630 > Change-Id: I793fed5a265b21762ba0a0cb6a669a3ba79f7ef6 > Reviewed-on: https://chromium-review.googlesource.com/683462 > Reviewed-by: Ned Nguyen <nednguyen@google.com> > Commit-Queue: Stephen Martinis <martiniss@chromium.org> > Cr-Commit-Position: refs/heads/master@{#504500} TBR=nednguyen@google.com,martiniss@chromium.org,eyaich@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 758630 Change-Id: Iab205ccbc2b1ce5042746bea28a2d2dcebb4faa3 Reviewed-on: https://chromium-review.googlesource.com/690637 Reviewed-by: Stephen Martinis <martiniss@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#505260} [modify] https://crrev.com/782c13fbfb92e2295b25d9c66c7dda19c15653e2/testing/scripts/run_multiple_telemetry_benchmarks_as_googletest.py
,
Sep 29 2017
Ok, I'm having issues getting the 'Linux Builder' builder on chromium.perf to compile and isolate the telemetry_perf_tests_new target. I submitted https://chromium-review.googlesource.com/c/chromium/src/+/691065, which I thought would do the trick, but it seems to have done nothing. As I've looked at it, I'm convinced that it should be doing something. This is because the 'Android arm64 Compile' builder has an entry in chromium.perf.json with an additional_compile_target, which isn't present in chromium_perf.py( https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/chromium_perf.py?q=chromium_perf.py&sq=package:chromium&l=159), which is being compiled on the bot. This target is, AFAIK, not explicitly required on any builder that is triggered by Android arm64 Compile, so if I'm reading the code correctly, the chromium.perf.json file should be causing the Android arm64 Compile builder to compile the microdump_stackwalk target. This doesn't seem to be working for the CL I committed though, and I'm not sure why.
,
Sep 29 2017
Ok, we think the problem is that no builder on chromium.perf runs any test which depends on the telemetry_perf_tests_new isolate, which means the builder doesn't try to isolate and upload that. I'm going to make a builder on the fyi waterfall, which should help with this, since the chromium_tests recipe module doesn't really understand cross master triggering well. The other options we thought about were teaching the chromium_tests recipe about cross master triggering, or adding a semi fake test on a builder on the chromium.perf waterfall which runs the isolate, but ignores the output. The latter seemed like a hack, and the former seems complicated and very easy to get wrong. I already know how to set up new builders, so the plan we have should be fairly straightforward.
,
Oct 2 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infra/infra_internal/+/df12de20c8a587e810e82884e32e7e290fab4fb7 commit df12de20c8a587e810e82884e32e7e290fab4fb7 Author: Stephen Martinis <martiniss@google.com> Date: Mon Oct 02 23:22:50 2017
,
Oct 3 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/950a43e18cbbd37d77d9c141ac40e7ce6667384d commit 950a43e18cbbd37d77d9c141ac40e7ce6667384d Author: Stephen Martinis <martiniss@chromium.org> Date: Tue Oct 03 21:20:33 2017 Add new perf fyi compile builder Bug:758630 Change-Id: Id3b752757efc99cacccc991c5881f4314448a49e Reviewed-on: https://chromium-review.googlesource.com/696325 Commit-Queue: Stephen Martinis <martiniss@chromium.org> Reviewed-by: David Tu <dtu@chromium.org> [modify] https://crrev.com/950a43e18cbbd37d77d9c141ac40e7ce6667384d/scripts/slave/recipe_modules/chromium_tests/chromium_perf_fyi.py [modify] https://crrev.com/950a43e18cbbd37d77d9c141ac40e7ce6667384d/scripts/slave/recipe_modules/chromium_tests/tests/api/archive_build.expected/linux_builder.json [modify] https://crrev.com/950a43e18cbbd37d77d9c141ac40e7ce6667384d/masters/master.chromium.perf.fyi/master.cfg [modify] https://crrev.com/950a43e18cbbd37d77d9c141ac40e7ce6667384d/masters/master.chromium.perf.fyi/slaves.cfg
,
Oct 3 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/134138157054ebf82ba3eee3938b474f84c57d42 commit 134138157054ebf82ba3eee3938b474f84c57d42 Author: Stephen Martinis <martiniss@chromium.org> Date: Tue Oct 03 22:15:53 2017 Schedule new perf fyi bot TBR=dtu Bug: 758630 Change-Id: I6da6c8e836f3f47542a1a18fa5599666a85107f4 Reviewed-on: https://chromium-review.googlesource.com/699052 Reviewed-by: Stephen Martinis <martiniss@chromium.org> Commit-Queue: Stephen Martinis <martiniss@chromium.org> [modify] https://crrev.com/134138157054ebf82ba3eee3938b474f84c57d42/masters/master.chromium.perf.fyi/master.cfg
,
Oct 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4dccf5d1c1bf257fdf600271404a8dd8d6d06fc6 commit 4dccf5d1c1bf257fdf600271404a8dd8d6d06fc6 Author: Stephen Martinis <martiniss@chromium.org> Date: Wed Oct 04 18:06:21 2017 Add new Linux Compile builder to mb Bug: 758630 Change-Id: I38955c789d3cc360ac73d5eb080e66965ca117e3 Reviewed-on: https://chromium-review.googlesource.com/699023 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#506443} [modify] https://crrev.com/4dccf5d1c1bf257fdf600271404a8dd8d6d06fc6/tools/mb/mb_config.pyl
,
Oct 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/35a3498891d50a19f4c5e1fa83f2a0fd76cf16c8 commit 35a3498891d50a19f4c5e1fa83f2a0fd76cf16c8 Author: Stephen Martinis <martiniss@chromium.org> Date: Thu Oct 05 00:28:10 2017 Fix mb config for linux compile builder I had it set to be an android compile, because I copy pasted from the other compile builders on fyi. Bug: 758630 Change-Id: I351cb22173dff6c24b249cda6044a4869c1edbd1 Reviewed-on: https://chromium-review.googlesource.com/701374 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#506581} [modify] https://crrev.com/35a3498891d50a19f4c5e1fa83f2a0fd76cf16c8/tools/mb/mb_config.pyl
,
Oct 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d4ed4cc579a6c4202129f6fdd50ac685b7771899 commit d4ed4cc579a6c4202129f6fdd50ac685b7771899 Author: Stephen Martinis <martiniss@chromium.org> Date: Thu Oct 05 13:02:45 2017 Revert "//tools/perf: Add compile target to linux builder" This is a partial revert of https://chromium-review.googlesource.com/c/chromium/src/+/691065. Some of that CL changed code structure which we want to keep. We don't want the compile target on the builder anymore though. Bug: 758630 Change-Id: I480018df2bbaa9ded3f2747c93849a5e59a6ddf7 Reviewed-on: https://chromium-review.googlesource.com/695834 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#506709} [modify] https://crrev.com/d4ed4cc579a6c4202129f6fdd50ac685b7771899/testing/buildbot/chromium.perf.json [modify] https://crrev.com/d4ed4cc579a6c4202129f6fdd50ac685b7771899/tools/perf/core/perf_data_generator.py
,
Oct 5 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infra/infra_internal/+/f1f92941cd70e94ee073ef546c18b7f0a34b2454 commit f1f92941cd70e94ee073ef546c18b7f0a34b2454 Author: Stephen Martinis <martiniss@google.com> Date: Thu Oct 05 18:43:18 2017
,
Oct 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/23c0a48720d68bb598757108cee00bba97a2ae1c commit 23c0a48720d68bb598757108cee00bba97a2ae1c Author: Stephen Martinis <martiniss@chromium.org> Date: Thu Oct 05 22:34:44 2017 Add necessary script for telemetry_perf_tests_new target # No try because trybot failures are unrelated to the change. NOTRY=true Bug: 758630 Change-Id: Ie3adb8ed81ecb5dab1578391f0fca0785c0ca23e Reviewed-on: https://chromium-review.googlesource.com/703501 Commit-Queue: Stephen Martinis <martiniss@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Commit-Position: refs/heads/master@{#506888} [modify] https://crrev.com/23c0a48720d68bb598757108cee00bba97a2ae1c/chrome/test/BUILD.gn
,
Oct 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f393d2432378fc23c53b3b9c57b5f5a1d4818d30 commit f393d2432378fc23c53b3b9c57b5f5a1d4818d30 Author: Stephen Martinis <martiniss@chromium.org> Date: Fri Oct 06 23:32:22 2017 Fix gn map for new telemetry test target It's still set for the old gn target. Bug: 758630 Change-Id: I365090a1843d0c3289b3ecf4a86c91fcd12aa871 Reviewed-on: https://chromium-review.googlesource.com/706066 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#507223} [modify] https://crrev.com/f393d2432378fc23c53b3b9c57b5f5a1d4818d30/testing/buildbot/gn_isolate_map.pyl
,
Oct 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6740ad0f89212c6c5174ab1db1f773cf8e236a63 commit 6740ad0f89212c6c5174ab1db1f773cf8e236a63 Author: Stephen Martinis <martiniss@chromium.org> Date: Tue Oct 10 20:28:46 2017 More fixes to run multiple telemetry benchmark There was a typo in the path to the benchmark sharding file. Also, the arguments for each benchmark were incorrect. Bug: 758630 Change-Id: Ib46e2a0898c3b3bea4388099b5d3a7cea156ed11 Reviewed-on: https://chromium-review.googlesource.com/707679 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#507758} [modify] https://crrev.com/6740ad0f89212c6c5174ab1db1f773cf8e236a63/testing/scripts/run_multiple_telemetry_benchmarks_as_googletest.py
,
Oct 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/82a843c927f26da7af48f18547af1e35bc504cb4 commit 82a843c927f26da7af48f18547af1e35bc504cb4 Author: Stephen Martinis <martiniss@chromium.org> Date: Fri Oct 13 10:39:32 2017 Use xvfb for the new perf test builder The machine doesn't have an X server running as is, so use xvfb to run the tests. Bug: 758630 Change-Id: I28e9734cee969f6eaee8e105a34297de7d20f31d Reviewed-on: https://chromium-review.googlesource.com/713718 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#508667} [modify] https://crrev.com/82a843c927f26da7af48f18547af1e35bc504cb4/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/82a843c927f26da7af48f18547af1e35bc504cb4/tools/perf/chromium.perf.fyi.extras.json
,
Oct 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b902d921a5dbdfb7fde34c5ddfb442fab19f8dfa commit b902d921a5dbdfb7fde34c5ddfb442fab19f8dfa Author: Stephen Martinis <martiniss@chromium.org> Date: Fri Oct 13 22:13:11 2017 //tools/perf: Switch buildbot arguments to correct builder name The one buildbot step builder used to be running the tests that a particular bot on an android builder would run. This was confusing for several people though, as they thought the bot itself was an android bot. This CL changes this to create a custom bot in the sharding map, which I can edit to control exactly what tests the builder runs. I picked a few benchmarks which take less than 10 minutes and should get decent coverage on different types of benchmarks, in case anything is different between them. Bug: 758630 Change-Id: I2559add1fe037891630ef775fb812a4a51d6962b Reviewed-on: https://chromium-review.googlesource.com/716871 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#508839} [modify] https://crrev.com/b902d921a5dbdfb7fde34c5ddfb442fab19f8dfa/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/b902d921a5dbdfb7fde34c5ddfb442fab19f8dfa/tools/perf/chromium.perf.fyi.extras.json [modify] https://crrev.com/b902d921a5dbdfb7fde34c5ddfb442fab19f8dfa/tools/perf/core/benchmark_sharding_map.json
,
Oct 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/83f82da6b55768b696d1629dfba26ad16c7978ec commit 83f82da6b55768b696d1629dfba26ad16c7978ec Author: Stephen Martinis <martiniss@chromium.org> Date: Sat Oct 14 02:02:08 2017 Fix file open function call in multiple telemetry benchmark script The script was trying to write out to files, but wasn't properly setting the file permission when calling open() in python. Bug: 758630 Change-Id: Ifcefab5b1e38d0edd4de6fc8956205281f481a74 Reviewed-on: https://chromium-review.googlesource.com/719344 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#508911} [modify] https://crrev.com/83f82da6b55768b696d1629dfba26ad16c7978ec/testing/scripts/run_multiple_telemetry_benchmarks_as_googletest.py
,
Oct 20 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/ee7330f4e1f7ed8a6d557f931445c18c87eb2692 commit ee7330f4e1f7ed8a6d557f931445c18c87eb2692 Author: Stephen Martinis <martiniss@google.com> Date: Fri Oct 20 17:01:16 2017
,
Nov 29 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9a27cc10d3314c75d08c63bfdf96c30022a11863 commit 9a27cc10d3314c75d08c63bfdf96c30022a11863 Author: Ashley Enstad <ashleymarie@chromium.org> Date: Wed Nov 29 19:20:21 2017 Changing OBBS bot to use run_telemetry_benchmark... instead of run_multiple_telemetry... There seem to be some issues with run_multiple_telemetry_benchmarks_as_googletest that we haven't quite worked out yet. In the meantime, let's switch this builder back to the singular version that all the other bots are running to be consistent. This will allow us to use the OBBS bot to test the new merge script BUG= chromium:758630 Change-Id: I325614d42a48d4ceb838a4f2d3e9d492b2584b37 Reviewed-on: https://chromium-review.googlesource.com/796510 Reviewed-by: Emily Hanley <eyaich@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Ashley Enstad <ashleymarie@chromium.org> Cr-Commit-Position: refs/heads/master@{#520190} [modify] https://crrev.com/9a27cc10d3314c75d08c63bfdf96c30022a11863/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/9a27cc10d3314c75d08c63bfdf96c30022a11863/testing/buildbot/gn_isolate_map.pyl [modify] https://crrev.com/9a27cc10d3314c75d08c63bfdf96c30022a11863/tools/perf/chromium.perf.fyi.extras.json
,
Dec 28 2017
I'm not owning this anymore.
,
Jan 11 2018
,
Jan 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/489bda77551dc941af3f94bcf4124fe3112d6524 commit 489bda77551dc941af3f94bcf4124fe3112d6524 Author: Emily Hanley <eyaich@google.com> Date: Mon Jan 29 15:47:00 2018 Fixing args error in trigger script Bug: 758630 Change-Id: If82674727722c0aa5ea8d7513d10fdc12ef96165 Reviewed-on: https://chromium-review.googlesource.com/889041 Commit-Queue: Emily Hanley <eyaich@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#532440} [modify] https://crrev.com/489bda77551dc941af3f94bcf4124fe3112d6524/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/489bda77551dc941af3f94bcf4124fe3112d6524/tools/perf/chromium.perf.fyi.extras.json [modify] https://crrev.com/489bda77551dc941af3f94bcf4124fe3112d6524/tools/perf/perf_device_trigger.py [modify] https://crrev.com/489bda77551dc941af3f94bcf4124fe3112d6524/tools/perf/perf_device_trigger_unittest.py
,
Jan 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8c61e7e5a5446c74990214e0526487c2526ed527 commit 8c61e7e5a5446c74990214e0526487c2526ed527 Author: Emily Hanley <eyaich@google.com> Date: Mon Jan 29 22:20:40 2018 Enabling sharding collect for the Perf custom trigger script. Adding a few environment flags to the swarming.py trigger call in an attempt to connect the dots between our custom trigger script and the built in swarming sharding mechanism that we are re-using in a custom way. This CL also ignores the shards flag from sharding since we define our own shards based on the bot-ids passed on. This *should* get rid of the failures we have been seeing on the obbs_fyi bot :) Bug: 758630 Change-Id: I28f67ae50d747e42fbe30cc95a1becead751ef20 Reviewed-on: https://chromium-review.googlesource.com/890824 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Emily Hanley <eyaich@chromium.org> Cr-Commit-Position: refs/heads/master@{#532620} [modify] https://crrev.com/8c61e7e5a5446c74990214e0526487c2526ed527/tools/perf/perf_device_trigger.py [modify] https://crrev.com/8c61e7e5a5446c74990214e0526487c2526ed527/tools/perf/perf_device_trigger_unittest.py
,
Feb 7 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/08a62aeadaaeed4197a03806393f58ec4cd8d616 commit 08a62aeadaaeed4197a03806393f58ec4cd8d616 Author: Emily Hanley <eyaich@google.com> Date: Wed Feb 07 14:41:01 2018 Factoring out base class for trigger_multiple_dimensions.py. This will be used by perf team for intercepting trigger call to trigger on specific bot ids. Bug: 758630 Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Change-Id: Ib5152748169ccd50d01dccc1f61979bcb967a481 Reviewed-on: https://chromium-review.googlesource.com/896365 Reviewed-by: Emily Hanley <eyaich@chromium.org> Reviewed-by: Ashley Enstad <ashleymarie@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Commit-Queue: Emily Hanley <eyaich@chromium.org> Cr-Commit-Position: refs/heads/master@{#535001} [modify] https://crrev.com/08a62aeadaaeed4197a03806393f58ec4cd8d616/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/08a62aeadaaeed4197a03806393f58ec4cd8d616/testing/scripts/run_performance_tests.py [add] https://crrev.com/08a62aeadaaeed4197a03806393f58ec4cd8d616/testing/trigger_scripts/base_test_triggerer.py [add] https://crrev.com/08a62aeadaaeed4197a03806393f58ec4cd8d616/testing/trigger_scripts/perf_device_trigger.py [add] https://crrev.com/08a62aeadaaeed4197a03806393f58ec4cd8d616/testing/trigger_scripts/perf_device_trigger_unittest.py [modify] https://crrev.com/08a62aeadaaeed4197a03806393f58ec4cd8d616/testing/trigger_scripts/trigger_multiple_dimensions.py [modify] https://crrev.com/08a62aeadaaeed4197a03806393f58ec4cd8d616/testing/trigger_scripts/trigger_multiple_dimensions_unittest.py [modify] https://crrev.com/08a62aeadaaeed4197a03806393f58ec4cd8d616/tools/perf/chromium.perf.fyi.extras.json [delete] https://crrev.com/b9a814ec2034a8f3dcf1a85edb36326b5b91848f/tools/perf/perf_device_trigger.py [delete] https://crrev.com/b9a814ec2034a8f3dcf1a85edb36326b5b91848f/tools/perf/perf_device_trigger_unittest.py
,
Feb 9 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/69ccba2a3c6487f4040cfabf253c3b63ad06664e commit 69ccba2a3c6487f4040cfabf253c3b63ad06664e Author: Emily Hanley <eyaich@google.com> Date: Fri Feb 09 20:15:25 2018 Indexing into bot map by shad instead of bot id. Also triggers two jobs per benchmark in the map, one for the browser passed in and one for the reference build. This needs to be moved in to the script executing on the bot now so we only have one entry per isolate. Bug: 758630 Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Change-Id: Ia299659c9f1e6d2160830e1fdebb6360e4e8d595 Reviewed-on: https://chromium-review.googlesource.com/909342 Reviewed-by: Ashley Enstad <ashleymarie@chromium.org> Commit-Queue: Emily Hanley <eyaich@chromium.org> Cr-Commit-Position: refs/heads/master@{#535803} [modify] https://crrev.com/69ccba2a3c6487f4040cfabf253c3b63ad06664e/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/69ccba2a3c6487f4040cfabf253c3b63ad06664e/testing/scripts/run_performance_tests.py [modify] https://crrev.com/69ccba2a3c6487f4040cfabf253c3b63ad06664e/testing/scripts/run_telemetry_benchmark_as_googletest.py [modify] https://crrev.com/69ccba2a3c6487f4040cfabf253c3b63ad06664e/testing/trigger_scripts/perf_device_trigger.py [add] https://crrev.com/69ccba2a3c6487f4040cfabf253c3b63ad06664e/tools/perf/core/benchmark_android_bot_map.json [modify] https://crrev.com/69ccba2a3c6487f4040cfabf253c3b63ad06664e/tools/perf/core/benchmark_bot_map.json [add] https://crrev.com/69ccba2a3c6487f4040cfabf253c3b63ad06664e/tools/perf/core/benchmark_desktop_bot_map.json [modify] https://crrev.com/69ccba2a3c6487f4040cfabf253c3b63ad06664e/tools/perf/core/benchmark_sharding_map.json
,
Feb 15 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/943ef215852dd0c86005861d6c4c44c1d07eefb8 commit 943ef215852dd0c86005861d6c4c44c1d07eefb8 Author: Ashley Enstad <ashleymarie@chromium.org> Date: Thu Feb 15 03:25:51 2018 Fixing exception in generate_perf_data Comment added to benchmark sharding map file causes a crash while loading the benchmark sharding map. This change allows the code to ignore the comment at the start of the map and continue running the script. Bug: 758630 Change-Id: Ia0617a53fb6f5b43c9f22288ca916218e8472d37 Reviewed-on: https://chromium-review.googlesource.com/916891 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#536940} [modify] https://crrev.com/943ef215852dd0c86005861d6c4c44c1d07eefb8/tools/perf/core/sharding_map_generator.py
,
Mar 21 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ef2afc041459c67d785eba38e008a5447f932ecc commit ef2afc041459c67d785eba38e008a5447f932ecc Author: Emily Hanley <eyaich@google.com> Date: Wed Mar 21 15:48:58 2018 Adding more targeted trigger dimensions. Bug: 758630 Change-Id: I6cfe03cb7f2fd2d210ee370554af4044a10cfb59 Reviewed-on: https://chromium-review.googlesource.com/973183 Reviewed-by: Ashley Enstad <ashleymarie@chromium.org> Commit-Queue: Emily Hanley <eyaich@chromium.org> Cr-Commit-Position: refs/heads/master@{#544715} [modify] https://crrev.com/ef2afc041459c67d785eba38e008a5447f932ecc/testing/buildbot/chromium.perf.fyi.json [modify] https://crrev.com/ef2afc041459c67d785eba38e008a5447f932ecc/tools/perf/core/perf_data_generator.py
,
Mar 22 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/92aed2943501767dd614ab0e1b57774141b16f88 commit 92aed2943501767dd614ab0e1b57774141b16f88 Author: Emily Hanley <eyaich@google.com> Date: Thu Mar 22 15:46:00 2018 Ignore the return code of the reference build. Now that the recipe doesn't see each benchmark, we don't want a swarming shard to fail if just the reference build fails. Bug: 758630 Change-Id: I67a11a8ef18e9a3a5405896367e67faa542b584f Reviewed-on: https://chromium-review.googlesource.com/975626 Reviewed-by: Ashley Enstad <ashleymarie@chromium.org> Commit-Queue: Emily Hanley <eyaich@chromium.org> Cr-Commit-Position: refs/heads/master@{#545085} [modify] https://crrev.com/92aed2943501767dd614ab0e1b57774141b16f88/testing/scripts/run_performance_tests.py
,
Jan 7
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by eyaich@chromium.org
, Aug 24 2017