Owen, Kuang-che and I discussed. The conclusion is to add an autotest for thumbnail test only because it requires rendering. We can disable rendering for the original video_VideoDecodeAccelerator by passing --rendering_fps=0.

Kuang-che. Can you help with this?

Alex is kind to help. Thank you.

Steps:
(1) Add video_VDAThumbnail autotest.
Make a copy of video_VideoDecodeAccelerator and only run thumbnail test case.
(2) In video_VideoDecodeAccelerator, exclude thumbnail test case and add --rendering_fps=0 for the rest test cases.
(3) Fix video_VDAPerf.
Add a new VideoDecodeAccelerator::Client class that does cadence and doesn't actually render. This will require a refactor of video_decode_accelerator_unittest.cc. Another possibility is to modify GLRenderingVDAClient and add yet another parameter. This is not preferred because GLRenderingVDAClient is way too huge.

We can file a separate bug for (3) if we want.

After (1) and (2) are done, video_VideoDecodeAccelerator should be all green in wmatrix.

I filed  http://crbug.com/728504  for (3).

As the immediate issues with the test are fixed and the thumbnail test is working again, I'll close this.
We should have another issue created for further improvements for vdatest.

This issue is still valid. This issue is not really about fixing thumbnail test. It's to separate the thumbnail test to a standalone autotest.

Most test cases in VDA unittest do not need rendering except thumbnail test. VDA unittest is broken by graphics or media CLs around every two weeks. When that happens, we lost the test results of all test cases. Separating them to two autotests can prevent it. I think this is very important.

Todos:
(1) Add video_VDAThumbnail autotest.
Make a copy of video_VideoDecodeAccelerator and only run thumbnail test case.
(2) In video_VideoDecodeAccelerator, exclude thumbnail test case and add --rendering_fps=0 for the rest test cases.

Also in favor of keeping it opened as it reminds me I need to address this sometime this month. :)

Hi. Any update for this?

I have been thinking a bit about this. I am not sure whether we need to split the VDA test into two, especially since these two test programs would end up being closely related and would share a lot of code.

How about instead refactoring VDA test such that display is not enabled for tests that don't need it, and thus only enabled for thumbnail? That way we would only have one binary. If we want to test thumbnail separately out of concern that it is more likely to fail, we can then call that binary twice with two different sets of unit tests to perform and end up with two different reports.

This would make this change easier to implement, and probably easier to maintain in the long-term too.

wuchengli@, is that sounds good to you I will rather go in that direction.

We decided to remove display functionality from vda unittest. For thumbnail test,  we'll replace it with a javascript test or make it not depend on ozone.