Look into increasing sharding for slow tests [webkit_layout_tests, browser_tests, etc.] on the CQ. |
|||
Issue descriptionQuestion: Can we increase the size of the fleet and/or increase the sharding parameter for slow tests to speed up the CQ? It's possible we're limited by non-parallelizable tasks [e.g. downloading isolate, startup/shutdown, etc.].
,
Jul 26
We do so on an as needed basis. It's a simple tuning parameter. Here's two recent examples: https://chromium-review.googlesource.com/1143435 https://chromium-review.googlesource.com/1133278 The sharding level is definitely bound by the amount of overhead, which varies from test to test and across platforms.
,
Jul 27
I thought I adjusted almost all tests not to run more than around 15 mins in https://chromium-review.googlesource.com/c/chromium/src/+/1104277 But we can/want to decrease that to 10 mins or less? http://shortn/_nZfp7PmMy8 I think there is some improvable space yet by simply increasing the number of shards. If we can use more shards, let me increase the shard num for some slowest tests.
,
Jul 27
It's a pure function of tolerable total fleet overhead. For example increasing from 10 to 20 shards is significantly more costly in term of total fleet overhead than 1 to 2. I'm working (sigh) on getting better data on this. John has the last word here.
,
Aug 7
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/eb6d997be97d8267af540deb57ce2763f77f5460 commit eb6d997be97d8267af540deb57ce2763f77f5460 Author: Takuto Ikuta <tikuta@chromium.org> Date: Tue Aug 07 00:46:43 2018 Increase shards for surface_sync_content_browsertests This is for slow surface_sync_content_browsertests step like belows. https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/54921 https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/54740 https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/54965 Overhead of the test is small, 6 seconds overhead vs 35 mins runtime. https://chromium-swarm.appspot.com/task?id=3f2773404bdad410&refresh=10&show_raw=1 So overhead increase is negligible. Bug: 868114 Change-Id: Iab6b396fcdf5a25ba50f299a0fa5e294ccea32ae Reviewed-on: https://chromium-review.googlesource.com/1163642 Commit-Queue: Takuto Ikuta <tikuta@chromium.org> Reviewed-by: Stephen Martinis <martiniss@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Cr-Commit-Position: refs/heads/master@{#581069} [modify] https://crrev.com/eb6d997be97d8267af540deb57ce2763f77f5460/testing/buildbot/chromium.android.fyi.json [modify] https://crrev.com/eb6d997be97d8267af540deb57ce2763f77f5460/testing/buildbot/chromium.android.json [modify] https://crrev.com/eb6d997be97d8267af540deb57ce2763f77f5460/testing/buildbot/chromium.clang.json [modify] https://crrev.com/eb6d997be97d8267af540deb57ce2763f77f5460/testing/buildbot/chromium.fyi.json [modify] https://crrev.com/eb6d997be97d8267af540deb57ce2763f77f5460/testing/buildbot/chromium.memory.json [modify] https://crrev.com/eb6d997be97d8267af540deb57ce2763f77f5460/testing/buildbot/test_suites.pyl
,
Aug 10
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/091a32d416d74d48bcaec9693c0ae2589bc2b986 commit 091a32d416d74d48bcaec9693c0ae2589bc2b986 Author: Takuto Ikuta <tikuta@chromium.org> Date: Fri Aug 10 23:26:22 2018 Increase shards for android buildbot I increased shards for some tests took time on https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/59189 Bug: 868114 Change-Id: I9ae5c80e0fe641f8946e0d00d8a322721d2de079 Reviewed-on: https://chromium-review.googlesource.com/1170922 Reviewed-by: Nico Weber <thakis@chromium.org> Commit-Queue: Takuto Ikuta <tikuta@chromium.org> Cr-Commit-Position: refs/heads/master@{#582381} [modify] https://crrev.com/091a32d416d74d48bcaec9693c0ae2589bc2b986/testing/buildbot/chromium.android.fyi.json [modify] https://crrev.com/091a32d416d74d48bcaec9693c0ae2589bc2b986/testing/buildbot/chromium.android.json [modify] https://crrev.com/091a32d416d74d48bcaec9693c0ae2589bc2b986/testing/buildbot/chromium.clang.json [modify] https://crrev.com/091a32d416d74d48bcaec9693c0ae2589bc2b986/testing/buildbot/chromium.memory.json [modify] https://crrev.com/091a32d416d74d48bcaec9693c0ae2589bc2b986/testing/buildbot/test_suites.pyl
,
Sep 5
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ea000e94b95e786748b5b6b860b2fc320ecca182 commit ea000e94b95e786748b5b6b860b2fc320ecca182 Author: Takuto Ikuta <tikuta@chromium.org> Date: Wed Sep 05 04:31:54 2018 Increase shard for browser_tests on win Currently browser_tests on win7_chromium_rel_ng sometimes took more than 10 mins. Let me increase the number of shards for such tests. e.g 753s https://ci.chromium.org/b/8936390613165886176 732s https://ci.chromium.org/b/8936406831776876544 706s https://ci.chromium.org/b/8936396862731996176 704s https://ci.chromium.org/b/8936363432077079248 Bug: 868114 Change-Id: I9a7eb07ce2c58f5b03e5f39152dba7ac6101f11b Reviewed-on: https://chromium-review.googlesource.com/1203519 Commit-Queue: Takuto Ikuta <tikuta@chromium.org> Reviewed-by: John Budorick <jbudorick@chromium.org> Cr-Commit-Position: refs/heads/master@{#588764} [modify] https://crrev.com/ea000e94b95e786748b5b6b860b2fc320ecca182/testing/buildbot/chromium.win.json [modify] https://crrev.com/ea000e94b95e786748b5b6b860b2fc320ecca182/testing/buildbot/test_suite_exceptions.pyl
,
Jan 9
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/f060055de7b1b7e537b5e490fd757ed98412c6a7 commit f060055de7b1b7e537b5e490fd757ed98412c6a7 Author: Takuto Ikuta <tikuta@google.com> Date: Wed Jan 09 23:05:11 2019
,
Jan 10
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c9c269a29ff911f91920ff108010a41f0b64e72b commit c9c269a29ff911f91920ff108010a41f0b64e72b Author: Takuto Ikuta <tikuta@chromium.org> Date: Thu Jan 10 22:40:25 2019 Increase shards for slow tests of win10_chromium_x64_rel_ng builder I'd like to increase shards that has more than 20 mins execution time in win10_chromium_x64_rel_ng builder. Slow tests are detected from below query http://shortn/_39HQP6rTpb AFAIK, win10_chromium_x64_rel_ng test does not have large per shard overhead. And current win10 capacity seems to able to handle this. http://shortn/_caTg5Sxv0F Bug: 868114 Change-Id: Ib0dbeefac8e7fe12ea3ef7fd7ca49c1de4f34598 Reviewed-on: https://chromium-review.googlesource.com/c/1391038 Reviewed-by: John Budorick <jbudorick@chromium.org> Reviewed-by: Stephen Martinis <martiniss@chromium.org> Commit-Queue: Takuto Ikuta <tikuta@chromium.org> Cr-Commit-Position: refs/heads/master@{#621783} [modify] https://crrev.com/c9c269a29ff911f91920ff108010a41f0b64e72b/testing/buildbot/chromium.win.json [modify] https://crrev.com/c9c269a29ff911f91920ff108010a41f0b64e72b/testing/buildbot/test_suite_exceptions.pyl |
|||
►
Sign in to add a comment |
|||
Comment 1 by dpranke@chromium.org
, Jul 26