adding machenbach for posterity, he is on vacation.

Swarming log: https://chromium-swarm.appspot.com/user/task/30acfd1f6729ec10 I don't see any obvious infra failures in there at first glance.

Says it ran for 45:00.011 before timing out. How long do you expect it to take under normal circumstances?

hablich@:

The other two shards were pretty close to the timeout already:
End of shard 1  Pending: 1.1s  Duration: 2241.3s  
End of shard 2  Pending: 0.9s  Duration: 2546.6s 

Maybe 0 just needs more time to run?
End of shard 0  Pending: 0.9s  Duration: 2700.0s

Swarming logs say you repro this task locally, maybe try that to see if it's actually hanging vs just slow?

  python swarming.py reproduce -S https://chromium-swarm.appspot.com 30acfd1f6729ec10

maruel: advice on how to deal with shard timeouts on swarming? Sounds like they may want to use more than 3 shards if they can.

I'd recommend increasing the number of shards if possible. I looked around but didn't find what code triggers this:
https://uberchromegw.corp.google.com/i/client.v8.ports/builders/V8%20Linux%20-%20arm64%20-%20sim%20-%20gc%20stress/builds/2029/steps/%5Btrigger%5D%20Mjsunit%20on%20Ubuntu-12.04/logs/stdio

From which directory do I run python swarming.py reproduce -S https://chromium-swarm.appspot.com 30acfd1f6729ec10?

I also can't do any tweakings to our bot setup, tandrii@ can you help?

all you need is swarming client, which should be DEPS in into v8. Easiest is to just:
mkdir repro && cd repro
git clone https://github.com/luci/luci-py.git
cd luci-py/client
python swarming.py reproduce -S https://chromium-swarm.appspot.com 30acfd1f6729ec10


Now I don't remember how exactly to tell test runner which tests to run in which shard but I'll find the place where # of shards is defined.

And test has finished on my workstation. So, yeah, it's rather slow and uses just 2 core. Looking for the place to increase shards.

https://codereview.chromium.org/2248363003

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/a415f195afcd9db6c89a5da50df939e76d69b849

commit a415f195afcd9db6c89a5da50df939e76d69b849
Author: tandrii <tandrii@chromium.org>
Date: Wed Aug 17 10:58:01 2016

V8: increase number of shards for arm64 gc stress.

Reason: shard#0 is timing out on swarming.
Expected follow up: if this helps, shard in other
builders as well.

R=hablich@chromium.org
CC=machenbach@chromium.org
BUG= 638295 

Review-Url: https://codereview.chromium.org/2248363003

[modify] https://crrev.com/a415f195afcd9db6c89a5da50df939e76d69b849/scripts/slave/recipe_modules/v8/builders.py
[modify] https://crrev.com/a415f195afcd9db6c89a5da50df939e76d69b849/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___arm64___sim___gc_stress.json

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chrome/tools/build_limited/scripts/slave/+/df848eb3cc6159123e99ca3ef3dc4cd019cfff37

commit df848eb3cc6159123e99ca3ef3dc4cd019cfff37
Author: recipe-roller <recipe-roller@chromium.org>
Date: Wed Aug 17 11:01:51 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/29b4489b6a2923fdae55e85775c2902958a1fd85

commit 29b4489b6a2923fdae55e85775c2902958a1fd85
Author: recipe-roller <recipe-roller@chromium.org>
Date: Wed Aug 17 11:16:37 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/a415f195afcd9db6c89a5da50df939e76d69b849 V8: increase number of shards for arm64 gc stress. (tandrii@chromium.org)

TBR=martiniss@chromium.org,phajdan.jr@chromium.org
BUG= 638295 

Recipe-Tryjob-Bypass-Reason: Autoroller
Review-Url: https://codereview.chromium.org/2251973002

[modify] https://crrev.com/29b4489b6a2923fdae55e85775c2902958a1fd85/infra/config/recipes.cfg

And latest build with 4 shards is green: https://uberchromegw.corp.google.com/i/client.v8.ports/builders/V8%20Linux%20-%20arm64%20-%20sim%20-%20gc%20stress/builds/2069

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bd3ca23d9881426164f1dbef90b8775ecdc34276

commit bd3ca23d9881426164f1dbef90b8775ecdc34276
Author: recipe-roller <recipe-roller@chromium.org>
Date: Fri Aug 19 21:11:13 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/cde40737c6fdc82d5923314b858aee7af9413aed Make test-results upload URL configurable. (jbudorick@chromium.org)
  https://crrev.com/61cbaf11512b007694b8a7a3bcb5e78540b508b3 Rename Dart's hot-reload builders. (whesse@google.com)
  https://crrev.com/cb3be09237faaf011f12e22b53910f98146733dc [Android] Stop using --flakiness-dashboard-server for gtests. (jbudorick@chromium.org)
  https://crrev.com/fb7d4a217be54db25a24736b3684c4cecdea8fc1 Added different colors for success/failure and enabled code search. (hzl@google.com)
  https://crrev.com/ccc7e42c27474fa3671625527ceb9ff49676886c Revert of Add logcats link in builbot page for android swarming tasks (patchset #10 id:180001 of https://codereview.chromium.org/2207263002/ ) (nicholaslin@google.com)
  https://crrev.com/fd87b7ef932d03a3e462f1034e68502340146f5a Archive Linux perf builds for manual bisect (miimnk@google.com)
  https://crrev.com/3a14129dc294c2f37a7e3041c646b888e0c0d50e Build the Flutter engine for android_debug and then the dist target (jsimmons@google.com)
  https://crrev.com/e434a4e3f39392fff4f5c908588a70ff81d26626 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/6e362607870ab3eae46cef835c492e1da2eb77c5 [Android] Surface --write-buildbot-json option of test_runner.py (perezju@chromium.org)
  https://crrev.com/394a606778b7ed3e29caa3122091ac0fb587798c Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/0b6afbd9546e9d43f51808e9a5fc2561a7995718 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/8c8efaa44980cfca495c2ab649fabd7ed2c22992 Deploy Win AMD R7 240 bot to chromium.gpu.fyi. (kbr@chromium.org)
  https://crrev.com/6936dc6c84035323fbb53e034ad3f217650fc9b4 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/d3dba1d2122e52ccb3de81c84181169c997529fe WebRTC: Don't specify All compile target (kjellander@chromium.org)
  https://crrev.com/59a9306a5c30de070c0c8a3daf68236a692c7f28 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/2b83bcc03bd67c1b3e14d5aee9115f05d64f8beb Add license header for test_results/config.py. (jbudorick@chromium.org)
  https://crrev.com/6571d071dcf67ea435cd1247c047a0f8c9063e41 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/0dabeec6241534115208cb966f9595928632fdbb added a comma in manual bisect file list (miimnk@google.com)
  https://crrev.com/8598c1dbdbc1c7cbb3936850256d419e9c04b942 Include //url/mojo generated Mojo bindings in the archive. (reillyg@chromium.org)
  https://crrev.com/9cab846f2ead44cef0e43d5f09eba5deb5871181 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/cabecc2d147dceff94f12daf9424cc3056f87a69 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/6da5a843e44836e7b7b084a6eca6c65eb4b4f16b Inherit appropiate environment variables in goma module for goma_utils.py (tikuta@chromium.org)
  https://crrev.com/651d39886fe48124507af6a74459abf3751d9768 Revert of Inherit appropiate environment variables in goma module for goma_utils.py (patchset #7 id:120001 of https://codereview.chromium.org/2245553002/ ) (phajdan.jr@chromium.org)
  https://crrev.com/6cbdfae7361d3ee8b1833ecab5a4bb824e4337d6 Roll recipe dependencies (nontrivial). (recipe-roller@chromium.org)
  https://crrev.com/dd5fb9292efabbbd1622131134d298620d1e967d Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/a415f195afcd9db6c89a5da50df939e76d69b849 V8: increase number of shards for arm64 gc stress. (tandrii@chromium.org)
  https://crrev.com/66b2cdbe64142ab8dc882619afbfe097bfe3df45 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/ca5861ac12b65fed94e4a1b884922f80162d0501 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/4f3df4173735261afe8eb594aa452023e476affe Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/ec204ef3434d4d752bd14fd71092fe376ab2cd50 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/d66f6826afe7193fbcdbbd053adeb6a7f8cb56ae Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/31a33ab94ae00abc12d57fafbbbdc06704febf70 Interpret collected iOS Swarming task results (smut@google.com)
  https://crrev.com/1b67c2b39afc5b0ac99d914dbe0bb684f52b2395 Deploy Linux slave with AMD R7 240 to chromium.gpu.fyi. (kbr@chromium.org)
  https://crrev.com/946f59b66de8e92ac1951af68b02f2dd9ddd9a52 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/3b623c022ebd9ecd876f8f5cb137bc21df19d40f Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/4e323b91fca31f987f70a57b5ac3e3d96fdeb26c Add --cs-base-url and --master-name for android inst tests. (hzl@google.com)
  https://crrev.com/808b48118eb904bf749b3e2cad6f893223d67489 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/ee3262441c2d143144fdf1f0796bdf5a3de969bd Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/4473ae7826ce3ca593d8a93c01bed3cdc1e87482 Use json.loads instead of json.load (smut@google.com)
  https://crrev.com/40fc2419139af2de77082e7349f4d8c701251bd0 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/cacd577ed71abeeb7f953fa877e9bd33c9d90a3f Roll recipe dependencies (nontrivial). (recipe-roller@chromium.org)
  https://crrev.com/814cb05c7b410646c0e58e4c3e2dfd697a03d96c Change the host_debug sky_shell used for testing to build with assertions enabled (jsimmons@google.com)
depot_tools:
  https://crrev.com/e38699bd9583c2ee9eb1436f3fc640cf97462722 Fix broken git tests on mac. (iannucci@chromium.org)
  https://crrev.com/e7f68952a4409cac43c75e0913fa4c661b359d08 Change default codereview instance to one we maintain. (iannucci@chromium.org)
  https://crrev.com/d090526a271d850cd3613865c02454ad45c9f138 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/2836bcf8102969bbbe56708d3dd6690bed271d5d Support additional user presubmit scripts named PRESUBMIT*.py. (tobiasjs@chromium.org)
  https://crrev.com/289d03e1c8a859328990899eb8941344192f9ee1 git-cl status: allow querying just the status of current CL (phajdan.jr@chromium.org)
  https://crrev.com/40ca0f912aefabf049588954fa6eba523fbc4dfa upload recipe bootstrap script (6a4b2e4) (phajdan.jr@chromium.org)
  https://crrev.com/ed7d0e38040ca5d335e2d66e28df3256dcd95b34 Make git version numbers external to batch script. (iannucci@chromium.org)
  https://crrev.com/31c2115e6a5a75f2ad2da8afc68c17385a73b0cc Whitespace change to trigger cipd package for git installer. (iannucci@chromium.org)
  https://crrev.com/bd604a16cbefb6a31531db8aa766954a2ddaa08c Revert of Whitespace change to trigger cipd package for git installer. (patchset #1 id:1 of https://chromiumcodereview.appspot.com/2251883002/ ) (iannucci@chromium.org)
  https://crrev.com/ff061c0d1d2d30909e201487cc72c106335ac3c6 Do not assume that all presubmit directories still exist. (tobiasjs@chromium.org)
  https://crrev.com/54d295468bac5fb259228f0b3ccb6b27f448e1d2 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
  https://crrev.com/eefe83265b93d55c024f23d58b3f8f45affab8dc Gerrit git cl upload: auto-filter patchset titles. (tandrii@chromium.org)
  https://crrev.com/ea9514aa207035624cf9909c15e9267989668d89 Fix bug in git cl status and add logging. (tandrii@chromium.org)
  https://crrev.com/3c972b9039766f02dd274a3620082b329d1f3ec3 Add --issue flag to git cl status. (iannucci@chromium.org)
  https://crrev.com/99a72f25140e574a27d9c7d52d840939e6c1c181 Gerrit git cl upload: self-lgtm on TBR. (tandrii@chromium.org)
  https://crrev.com/e53c935fe76b0514f49b9100e8a1e0c78b9f9c26 Add --issue to set-commit and set-close too. (iannucci@chromium.org)
  https://crrev.com/196aa81d8ae25804332ffca00ee33d4c0eab61f4 Remove old unused SVN related scripts from depot_tools (hinoka@google.com)
  https://crrev.com/4d8955052fd5d68f54292f2622130622b6e9eb6b git cl: avoid infinite recursion during debug/development. (tandrii@chromium.org)
  https://crrev.com/439170a2e7db7bd7062fd81aa44fb578d663f8e1 infra/config/recipes.cfg: Add recipe roller as owner (martiniss@chromium.org)
  https://crrev.com/6d0a5acdc43f959770d844a3ccb8bc3a5ad9afb6 Add git_bootstrap.py, which bootstraps git on Windows using CIPD (phajdan.jr@chromium.org)
  https://crrev.com/5d48c32f3d75691589c371266152b00aab355602 git cl: cleanup branch config get/set/unset. (tandrii@chromium.org)
  https://crrev.com/df09a46d7cec6d4a523a0bb7d740d70b3ac7c663 git cl patch: bail out quickly if there is no branch. (tandrii@chromium.org)
  https://crrev.com/2cbf79f3c976c392242cb2e8710d397ad1ee1762 git: use new python script to bootstrap git on windows (phajdan.jr@chromium.org)
recipe_engine:
  https://crrev.com/f50a2af3c5190b5b9780855085bd5a8df0d4ec58 Add the ability to dump initial recipe properties. (dnj@chromium.org)
  https://crrev.com/6c19ddf62e5c5cdcfda2a2ed55bdb4518f2e0c7a raw_io: fix passing retcode in test_api.stream_output (phajdan.jr@chromium.org)

TBR=martiniss@chromium.org,phajdan.jr@chromium.org
BUG= webrtc:5949 ,622062, 544330 ,638343,629679, 620494 , 607344 ,637561,https://github.com/dart-lang/sdk/issues/27005,637037,611020,604452,chromium:628770,635421,605572,None,chromium:448050,626364,598405,none,629863,635877,623207,638295

Recipe-Tryjob-Bypass-Reason: Autoroller
Review-Url: https://codereview.chromium.org/2259343002
Cr-Commit-Position: refs/heads/master@{#413236}

[modify] https://crrev.com/bd3ca23d9881426164f1dbef90b8775ecdc34276/infra/config/recipes.cfg

See comment on:
https://codereview.chromium.org/2248363003

This was caused by an extremely slow test that got added - more shards don't solve this problem. Assigning to CL author of: https://codereview.chromium.org/2224253003

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/864cdc124c55f48740100a4dcaa3b5582040d815

commit 864cdc124c55f48740100a4dcaa3b5582040d815
Author: bmeurer <bmeurer@chromium.org>
Date: Tue Aug 30 04:04:01 2016

[test] Speed-up regression test for growing stores.

TBR=machenbach@chromium.org
BUG= chromium:635798 , chromium:638295 

Review-Url: https://codereview.chromium.org/2288813003
Cr-Commit-Position: refs/heads/master@{#38991}

[modify] https://crrev.com/864cdc124c55f48740100a4dcaa3b5582040d815/test/mjsunit/regress/regress-crbug-635798.js

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/860d8d2ab8b6ad3a5ca09d23356252b4944624cc

commit 860d8d2ab8b6ad3a5ca09d23356252b4944624cc
Author: machenbach <machenbach@chromium.org>
Date: Tue Aug 30 07:01:10 2016

Revert of V8: increase number of shards for arm64 gc stress. (patchset #1 id:1 of https://codereview.chromium.org/2248363003/ )

Reason for revert:
The root cause of the slow tests has been fixed. A shard increase is not needed.

Original issue's description:
> V8: increase number of shards for arm64 gc stress.
>
> Reason: shard#0 is timing out on swarming.
> Expected follow up: if this helps, shard in other
> builders as well.
>
> R=hablich@chromium.org
> CC=machenbach@chromium.org
> BUG= 638295 
>
> Committed: https://chromium.googlesource.com/chromium/tools/build/+/a415f195afcd9db6c89a5da50df939e76d69b849

TBR=hablich@chromium.org,tandrii@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG= 638295 

Review-Url: https://codereview.chromium.org/2291703003

[modify] https://crrev.com/860d8d2ab8b6ad3a5ca09d23356252b4944624cc/scripts/slave/recipe_modules/v8/builders.py
[modify] https://crrev.com/860d8d2ab8b6ad3a5ca09d23356252b4944624cc/scripts/slave/recipes/v8.expected/full_client_v8_ports_V8_Linux___arm64___sim___gc_stress.json

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/c46ddcfc40b60bb992664b5465cabd3359cbd280

commit c46ddcfc40b60bb992664b5465cabd3359cbd280
Author: recipe-roller <recipe-roller@chromium.org>
Date: Tue Aug 30 10:04:33 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/8ceac5bce3e789666da2daa8607d254055788114 Use 'goma_recipe_module' ref for specifying a version of cloudtail to use. (vadimsh@chromium.org)
  https://crrev.com/860d8d2ab8b6ad3a5ca09d23356252b4944624cc Revert of V8: increase number of shards for arm64 gc stress. (patchset #1 id:1 of https://codereview.chromium.org/2248363003/ ) (machenbach@chromium.org)
  https://crrev.com/1f2161a43f7874cb6ea371dffef3b96cf435bc7f Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
depot_tools:
  https://crrev.com/26a201f2a9da493772dfc4bc1a498b200e6c801d Revert of Delete roll-dep-svn (patchset #1 id:1 of https://codereview.chromium.org/2282223002/ ) (machenbach@chromium.org)

TBR=martiniss@chromium.org,phajdan.jr@chromium.org
BUG= 638295 ,475320

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2292903002

[modify] https://crrev.com/c46ddcfc40b60bb992664b5465cabd3359cbd280/infra/config/recipes.cfg

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bc2bc512cef2f509e2032e954de763d621664322

commit bc2bc512cef2f509e2032e954de763d621664322
Author: recipe-roller <recipe-roller@chromium.org>
Date: Tue Aug 30 11:39:23 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/860d8d2ab8b6ad3a5ca09d23356252b4944624cc Revert of V8: increase number of shards for arm64 gc stress. (patchset #1 id:1 of https://codereview.chromium.org/2248363003/ ) (machenbach@chromium.org)
  https://crrev.com/1f2161a43f7874cb6ea371dffef3b96cf435bc7f Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
depot_tools:
  https://crrev.com/26a201f2a9da493772dfc4bc1a498b200e6c801d Revert of Delete roll-dep-svn (patchset #1 id:1 of https://codereview.chromium.org/2282223002/ ) (machenbach@chromium.org)

TBR=martiniss@chromium.org,phajdan.jr@chromium.org
BUG= 638295 ,475320

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2294743002
Cr-Commit-Position: refs/heads/master@{#415272}

[modify] https://crrev.com/bc2bc512cef2f509e2032e954de763d621664322/infra/config/recipes.cfg

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chrome/tools/build_limited/scripts/slave/+/94435b7257996b76edbbaeee73e2eb5165b13d32

commit 94435b7257996b76edbbaeee73e2eb5165b13d32
Author: recipe-roller <recipe-roller@chromium.org>
Date: Tue Aug 30 13:03:41 2016

 Issue v8:5285  has been merged into this issue.