Shard webgl_conformance_tests on Nexus 5X |
||||
Issue descriptionNow that single-device Swarming has been deployed for the Nexus 5X devices in the Swarming pool, the webgl_conformance_tests should be shareded. Runs take about 30 minutes currently: https://luci-milo.appspot.com/buildbot/chromium.gpu.fyi/Android%20Release%20%28Nexus%205X%29/9820 https://luci-milo.appspot.com/buildbot/chromium.gpu.fyi/Android%20Release%20%28Nexus%205X%29/9819 https://luci-milo.appspot.com/buildbot/chromium.gpu.fyi/Android%20Release%20%28Nexus%205X%29/9818 We should try sharding them 6 ways on this configuration to get the expected run time down to ~5 minutes.
,
Aug 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f656862394e0006712f3d3427b5d58f4191c3469 commit f656862394e0006712f3d3427b5d58f4191c3469 Author: Kenneth Russell <kbr@chromium.org> Date: Mon Aug 14 22:45:16 2017 Shard webgl_conformance_tests on Android. This will only have an effect on the Nexus 5X bots, which are the only ones Swarmed right now. They're sharded 6 ways to try to get the per-shard runtime down to 5 minutes. It may be necessary to follow this up with a CL that adds per-test runtimes, in the same way done for the WebGL 2.0 conformance tests. BUG= 754467 TBR=ynovikov@chromium.org Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Change-Id: Iaa8c63bebe6f517c02a62de93e52e218ef93f893 Reviewed-on: https://chromium-review.googlesource.com/612697 Commit-Queue: Kenneth Russell <kbr@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Cr-Commit-Position: refs/heads/master@{#494221} [modify] https://crrev.com/f656862394e0006712f3d3427b5d58f4191c3469/content/test/gpu/generate_buildbot_json.py [modify] https://crrev.com/f656862394e0006712f3d3427b5d58f4191c3469/testing/buildbot/chromium.gpu.fyi.json [modify] https://crrev.com/f656862394e0006712f3d3427b5d58f4191c3469/testing/buildbot/client.v8.fyi.json
,
Aug 15 2017
We definitely need to use the per-test times to get a better shard distribution. Right now one of the shards takes 19 minutes and the others take 4 minutes or less. https://luci-milo.appspot.com/buildbot/chromium.gpu.fyi/Android%20Release%20%28Nexus%205X%29/9900
,
Aug 17 2017
The test times in the forthcoming CL were gathered via: python gather_swarming_json_results.py --waterfall chromium.gpu.fyi --bot "Android Release (Nexus 5X)" --build 9948 --step webgl_conformance_tests --output webgl_conformance_tests_output.json
,
Aug 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/acf059f108c127f38f0234df97c4268869dfb507 commit acf059f108c127f38f0234df97c4268869dfb507 Author: Kenneth Russell <kbr@chromium.org> Date: Thu Aug 17 19:18:53 2017 Use test runtimes to distribute shards more evenly on Android. Updated the gather_swarming_json_results.py script to work with the new luci-milo prpc endpoints rather than the BuildBot JSON endpoints, which have been decommissioned. BUG= 754467 TBR=ynovikov@chromium.org Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Change-Id: I9397efbc337acee217b75a5cc94128aa0a581bfc Reviewed-on: https://chromium-review.googlesource.com/618299 Reviewed-by: Kenneth Russell <kbr@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Commit-Queue: Kenneth Russell <kbr@chromium.org> Cr-Commit-Position: refs/heads/master@{#495271} [add] https://crrev.com/acf059f108c127f38f0234df97c4268869dfb507/content/test/data/gpu/webgl_conformance_tests_output.json [modify] https://crrev.com/acf059f108c127f38f0234df97c4268869dfb507/content/test/gpu/gather_swarming_json_results.py [modify] https://crrev.com/acf059f108c127f38f0234df97c4268869dfb507/content/test/gpu/generate_buildbot_json.py [modify] https://crrev.com/acf059f108c127f38f0234df97c4268869dfb507/testing/buildbot/chromium.gpu.fyi.json [modify] https://crrev.com/acf059f108c127f38f0234df97c4268869dfb507/testing/buildbot/client.v8.fyi.json
,
Aug 17 2017
From https://luci-milo.appspot.com/buildbot/chromium.gpu.fyi/Android%20Release%20%28Nexus%205X%29/9970 and examining the run time of the shards, they were evenly distributed, around 6-7 minutes each. |
||||
►
Sign in to add a comment |
||||
Comment 1 by kbr@chromium.org
, Aug 10 2017