The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/drm-tests/+/f6fe9b731d3a805e9d3d6e6b144b560a9b164c99

commit f6fe9b731d3a805e9d3d6e6b144b560a9b164c99
Author: Stéphane Marchesin <marcheu@chromium.org>
Date: Tue Apr 03 03:16:47 2018

drm-tests: Speed up the tests

These tests take a long time, and we can make them faster without
losing coverage.

BUG=chromium:828163
TEST=compiles

Change-Id: I5d125f23deb71cf4e5694bbb27582a0319d2c868
Signed-off-by: Stéphane Marchesin <marcheu@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/991072
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>

[modify] https://crrev.com/f6fe9b731d3a805e9d3d6e6b144b560a9b164c99/atomictest.c
[modify] https://crrev.com/f6fe9b731d3a805e9d3d6e6b144b560a9b164c99/mmap_test.c

It looks like it still is timing out on eve R68-10595.0.0:
https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/193058899-chromeos-test/chromeos6-row4-rack17-host8/graphics_Drm/debug/

04/18 21:55:42.302 DEBUG|      graphics_Drm:0149| Running test atomictest.
04/18 21:55:42.302 DEBUG|             utils:0214| Running 'atomictest -t all'
[...]
04/18 22:00:37.777 DEBUG|             utils:0282| [stdout] ----- [CRTC: 45] -----
04/18 22:00:37.777 DEBUG|             utils:0282| [stdout] 	{Plane ID: 36, Plane format: NV12, Plane type: primary, CRTC_X: 450, CRTC_Y: 180, CRTC_W: 600, CRTC_H: 400}
04/18 22:00:42.784 WARNI|             utils:0925| run process timeout (300) fired on: atomictest -t all
04/18 22:00:43.792 DEBUG|             utils:0301| [stdout] 	{Plane ID: 39, Plane format: AR24, Plane type: over
04/18 22:00:43.793 ERROR|      graphics_Drm:0092| Failed: Timeout while running atomictest -t all (timeout=300.00s)
04/18 22:00:43.798 DEBUG|      graphics_Drm:0093| Command <atomictest -t all> failed, rc=-15, Command(s) did not complete within 300 seconds

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/drm-tests/+/08025e8d6770f6483839919b3788e1806454d22e

commit 08025e8d6770f6483839919b3788e1806454d22e
Author: Gurchetan Singh <gurchetansingh@chromium.org>
Date: Fri Apr 20 07:33:10 2018

drm-tests: Speed up the tests

Previously, test cases that moved across the screen had a usleep of
1e6 / 60, while tests that displayed a few frames (i.e, test_orientation)
had  sleep of 1e6.  The change to test_and_commit() made every case
have a usleep of 1e6. This changes the timeouts back, and adds an 'automatic'
option that runs with a usleep of 0.

BUG=chromium:828163
TEST=builds

Change-Id: Ife1889387fca9632cc37c9272cfc6115dbd1cdbe
Reviewed-on: https://chromium-review.googlesource.com/1020405
Commit-Ready: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Stéphane Marchesin <marcheu@chromium.org>

[modify] https://crrev.com/08025e8d6770f6483839919b3788e1806454d22e/atomictest.c

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/2bdd37fc3a0a4fb18b58ac748db99ee5a14497b0

commit 2bdd37fc3a0a4fb18b58ac748db99ee5a14497b0
Author: Ilja H. Friedel <ihf@chromium.org>
Date: Fri Apr 20 19:57:29 2018

graphics_Drm: use -a option to run atomic.

Use -a (automatic) option to run fast when not visually debugging
the output of the test. This speeds up atomic test.

BUG=chromium:828163
TEST=lab will test

Change-Id: I63b05c9fa29a510f618c90cc64c9534e2d39dc23
Reviewed-on: https://chromium-review.googlesource.com/1020880
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>

[modify] https://crrev.com/2bdd37fc3a0a4fb18b58ac748db99ee5a14497b0/client/site_tests/graphics_Drm/graphics_Drm.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/a215ae4e4eb04636a1cf5f0fbc030ef8ca33330a

commit a215ae4e4eb04636a1cf5f0fbc030ef8ca33330a
Author: Ilja H. Friedel <ihf@chromium.org>
Date: Tue Apr 24 00:48:13 2018

graphics_Drm: increase timeout back to 300s.

Still timing out on dashboard.

BUG=chromium:828163
TEST=None.

Change-Id: Ic7e7d0441c2f097ca5d70f5bbdb8b57fb43233a7
Reviewed-on: https://chromium-review.googlesource.com/1025257
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>

[modify] https://crrev.com/a215ae4e4eb04636a1cf5f0fbc030ef8ca33330a/client/site_tests/graphics_Drm/graphics_Drm.py

atomic runs in 144 seconds now eve and fails one test on

04/24 20:29:32.996 DEBUG|             utils:0282| [stdout] Committing the following configuration: 
04/24 20:29:32.996 DEBUG|             utils:0282| [stdout] ----- [CRTC: 55] -----
04/24 20:29:32.996 DEBUG|             utils:0282| [stdout] 	{Plane ID: 46, Plane format: XB24, Plane type: primary, CRTC_X: 0, CRTC_Y: 0, CRTC_W: 2400, CRTC_H: 1600}
04/24 20:29:45.655 DEBUG|             utils:0282| [stderr] WARN:run_atomictest():atomictest.c:1280:orientation failed test commit, testcase not run.
04/24 20:29:46.923 DEBUG|             utils:0282| [stderr] ERROR:test_video_underlay():atomictest.c:1043:failed with error: -22

Daniele: Stephane says you added this test, could you please take a look at the failure?

It seems like the crtc 55 doesn't support NV12. Let me upload a patch to skip crtcs that can't do NV12.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/drm-tests/+/dd271048f79bdf99ec74e56e6d9059560fc6656c

commit dd271048f79bdf99ec74e56e6d9059560fc6656c
Author: Daniele Castagna <dcastagna@chromium.org>
Date: Sat Apr 28 04:27:39 2018

drm-tests: underlay video, fall back to RGB when NV12 fails

Some crtc might not support NV12, fallback to a non YUV format in
underlay_test instead of making the test fail.

BUG=chromium:828163
TEST=Run atomictest on eve

Change-Id: Id0b124bf899961056f5d801d12ca6a2d267edafb
Reviewed-on: https://chromium-review.googlesource.com/1033894
Commit-Ready: Daniele Castagna <dcastagna@chromium.org>
Tested-by: Daniele Castagna <dcastagna@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>

[modify] https://crrev.com/dd271048f79bdf99ec74e56e6d9059560fc6656c/atomictest.c

The test failure is fixed. Thank you!

Stephane, do you want to land the performance optimization now?

I looked at it improving the performance significantly will not be trivial. There are probably small things we can do though, I'll take another look.

Most of the tests using move_plane will move planes until parts of them is out of the screen.
We really don't need to test that, we can end tests as soon as we can move_plane a few times when we are in 'automatic' mode. Does that make sense to you?

I already increased the size of the step for tests which move planes; we could increase it further; but at this point it isn't the bottlenecl. The test would also lose some coverage.

Note that at this point the bulk of the time is spend on turning *off* and *on* the crtc between each experiment. Since these things are stateful, this is an easy way to start from a clean slate; however if we did some state tracking we could transition from one state to the next without turning everything off then back on.

Instead of doing:
for each test:
  for each crtc:

Can we swap the for loops the other way? Would that help speeding up modesetting? Modesetting in Chrome seems to take a significant amount of time.

That's the kind of thing we could do, but we still need to track the state to restore it to "neutral" once we are done.

The original failures got fixed.
https://chromeperf.appspot.com/group_report?sid=54d05a41c0380228bc4db58910fc0f6116306e58467a75c542e8eff56498d5a1

If we want to run in CQ/pre-cq we need to get faster.