Per email thread, forked from Issue 722474 .
This will only happen once there is general agreement on the plan. Once there is agreement we are planning to keep one test and move it to bvt-inline.
https://chromium-review.googlesource.com/#/c/510112/
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58 commit 292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58 Author: Ilja H. Friedel <ihf@chromium.org> Date: Tue Jun 13 07:19:42 2017 Run fewer graphics tests in blocking part of CQ/BVT. TLDR: This reduces the number of graphics tests in BVT from 20 to 8 As autotest currently has a server overhead per client test of 90+ DUT holding seconds (crbug.com/726481), this saves more than 90s * (20-8) = 18 minutes in the locking suite (usually distributed over 4-7 DUTs; or still 3-4 minutes CQ cycle time). -- To make the Chrome OS commit queue work better we want to a) reduce builder cycle time b) reduce the number of possible flakes causing extra cycles We also want to make the Chrome gardener's life easier and impose less graphics tests to watch on the gardening rotation. Notice all tests in this change are very mature and well behaved, but the law of large numbers is against CQ runs. So the less valuable tests must go out of the critical loop and into a not-blocking suite. To not reduce the coverage frequency we move remaining tests to bvt-perbuild. For this to work the graphics team will need to watch regressions in a rotation and revert changes after failures are detected. We move several very important tests to bvt-inline, which right now finishes faster than bvt-cq and has greater hardware coverage. The currently important/blocking graphics tests are - graphics_GLBench.bvt - Chrome-like functionality, pixel accurate. - graphics_GLMark2.bvt - Exercises game like paths. - graphics_dEQP.bvt - gles2, gles3, gles31 and vulkan. - graphics_Drm.bvt - Functional/integration tests (not in this change will follow later). - graphics_Gbm - Verifies Mesa graphics buffer management. - graphics_Idle - Verifies graphics reaches low power state. - graphics_Idle.arc - Moved by Richard to bvt-arc (good place). - graphics_Sanity - Checks screenshot for corruption. Sum of remaining pure test times (without autotest overhead) on veyron_minnie R60-9574 are on the order of 3-4 minutes: bin/autotest tests/graphics_GLBench/control.bvt - real 0m45.225s bin/autotest tests/graphics_GLMark2/control.bvt - real 0m18.270s bin/autotest tests/graphics_dEQP/control.bvt - real 0m11.459s bin/autotest tests/graphics_Drm/control.bvt - real 0m55.893s bin/autotest tests/graphics_Gbm/control - real 0m12.223s While we are at it also get rid of JOB_RETRIES in bvt-perbuild. BUG= chromium:723898 , chromium:722474 TEST=CQ will test. CQ-DEPEND=CL:*390814 Change-Id: I41e33557e74757c579ff0e9c5b0d65c0e60d0e6d Reviewed-on: https://chromium-review.googlesource.com/510112 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Stéphane Marchesin <marcheu@chromium.org> [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_GLAPICheck/control [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.gles3.performance [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_Idle/control [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.gles3.accuracy [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.gles31.stress [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_GpuReset/control [rename] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_SanAngeles/control.bvt [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.vk.info [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.vk.binding_model.hasty [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.vk.spirv_assembly.hasty [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.gles2.performance [rename] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_GLBench/control.bvt [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty.5 [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.gles31.info [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.gles3.info [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.gles2.accuracy [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.gles2.info [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_PerfControl/control [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.gles2.capability [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty.8 [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty.9 [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/generate_controlfiles.py [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty.2 [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty.3 [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty.0 [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty.1 [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty.6 [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty.7 [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.vk-master.hasty.4 [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_LibDRM/control [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.bvt [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_Gbm/control [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.gles3.stress [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.vk.glsl.hasty [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.vk.api.hasty [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_KernelMemory/control [modify] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_dEQP/control.gles2.stress [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.vk.api.smoke [rename] https://crrev.com/292ee4862c1c2da93bf1a4ff8e87019d2ef9eb58/client/site_tests/graphics_GLMark2/control.bvt [delete] https://crrev.com/536f23869b1b94e59916e806bcbee15c55a1f190/client/site_tests/graphics_dEQP/control.vk.pipeline.hasty
The graphics team did its share.
Comment 1 by akes...@chromium.org
, May 17 2017