chrome crashes on startup on ToT. I reported  Issue 646945 , and bisecting now. Not sure if it's the same cause of the failure.

video_ChromeHWDecodeUsed.vp9 still fails locally on ToT after the culprit CL in  issue 646945  has been reverted. Will try to find the bad CL.

According to #1 in  Issue 647103 , the video_ChromeHWDecodeUsed.h264 is the one that fails.
I tried both video_ChromeHWDecodeUsed.h264 and video_ChromeHWDecodeUsed.h264.mse locally on ToT on peach_pit and they both pass.

I'm not sure why they keep failing on the peach_pit and tricky informational builders.
Should I disable these tests on these boards for now?

+video folks

Just tested locally on a tricky device, both the tests video_ChromeHWDecodeUsed.h264 and video_ChromeHWDecodeUsed.h264.mse are passed.

This is the informational builder and chrome has not reved yet on goldeneye. Either use goldeneye image and build newest chrome binary using simple chrome, or install the same image from the links above to repro.

I am taking another look at the logs.

Re#7: Right. Both Ahmed and me built the Tot Chromeos image, installed it on the test device, and then flashed the Tot chrome binary using simple chrome. The tests were passed on both tricky and peach_pit.

Re#7: Yes, I built latest peach_pit Chrome OS image -- flashed it to the device -- then used simple-chrome to deploy the ToT chrome on the device.

Both tests passes on peach_pit.

On thing that is really odd is that we had a bunch of green peach_pit runs from #2886 to #2891 and the #2892 got bad again.

tricky on the other side is just completely randomly flaky.

Can you run again with
test_that --iterations 30 $DUT video_ChromeHWDecodeUsed

Running...

All 30 iterations for video_ChromeHWDecodeUsed.h264 have passed locally on peach_pit.

Ok, can you move the test from bvt-cq to bvt-perbuild and assign the issue to Pawel?

All are passed on tricky as well.

Re#15: Thanks! CL in review: https://chromium-review.googlesource.com/#/c/386018

video_ChromeHWDecodeUsed.vp9 should fail on peach and tricky, but it should also not be run on these boards, because it requires hw_video_acc_vp9 label on a DUT. DUTs that don't support VP9 should not have it.

So this could explain the apparent flakiness perhaps?

Actually, sorry, it should be succeeding since R53 on tricky (but not peach).

Hm, I'm not sure I understand the builder errors. The relevant subtests for unsupported codecs are correctly skipped, but then the whole test fails... 

Maybe unrelated: the -rc1 build here which I think comes from llvm-next-toolchain produces a lot of failures (for a long time now, but not on M54?)

https://wmatrix.googleplex.com/testrun/unfiltered?test_ids=349037153
https://wmatrix.googleplex.com/platform/unfiltered?hide_missing=True&tests=video_ChromeHWDecodeUsed&days_back=100&releases=tot&platforms=peppy
https://wmatrix.googleplex.com/platform/unfiltered?hide_missing=True&tests=video_ChromeHWDecodeUsed&days_back=100&releases=tot&platforms=daisy
https://wmatrix.googleplex.com/unfiltered?hide_missing=True&releases=tot&tests=video_ChromeHWDecodeUsed&days_back=100

Have we run this test on failing image from the PFQ builder? Looking at the test history, I don't see any flake on peach-* devices. 

As Pawel mentioned, please use vp8/h264 for test for verification as vp9 is not supported on all PFQ boards.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/e46fe984a9491f211ad68d006c5ac9de8e813439

commit e46fe984a9491f211ad68d006c5ac9de8e813439
Author: xdai <xdai@chromium.org>
Date: Thu Sep 15 22:12:52 2016

Move the test video_ChromeHWDecodeUsed from bvt-cq to bvt-perbuild.

The video_ChromeHWDecodeUsed tests are constantly failed on the information builders:
peach_pit-tot-chrome-pfq-informational and tricky-tot-chrome-pfq-informational.

Move them to bvt-perbuild until we figure out the reason of the failure.

BUG= chromium:646898 
TEST=none

Change-Id: I199bb3218ccf882a271605be8a42ceb39e98900b
Reviewed-on: https://chromium-review.googlesource.com/386018
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Trybot-Ready: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Xiaoqian Dai <xdai@chromium.org>

[modify] https://crrev.com/e46fe984a9491f211ad68d006c5ac9de8e813439/client/site_tests/video_ChromeHWDecodeUsed/control.h264
[modify] https://crrev.com/e46fe984a9491f211ad68d006c5ac9de8e813439/client/site_tests/video_ChromeHWDecodeUsed/control.h264.mse
[modify] https://crrev.com/e46fe984a9491f211ad68d006c5ac9de8e813439/client/site_tests/video_ChromeHWDecodeUsed/control.vp8
[modify] https://crrev.com/e46fe984a9491f211ad68d006c5ac9de8e813439/client/site_tests/video_ChromeHWDecodeUsed/control.vp9
[modify] https://crrev.com/e46fe984a9491f211ad68d006c5ac9de8e813439/client/site_tests/video_ChromeHWDecodeUsed/control.vp9.mse
[modify] https://crrev.com/e46fe984a9491f211ad68d006c5ac9de8e813439/client/site_tests/video_ChromeHWDecodeUsed/control.vp9.arc
[modify] https://crrev.com/e46fe984a9491f211ad68d006c5ac9de8e813439/client/site_tests/video_ChromeHWDecodeUsed/control.h264.arc
[modify] https://crrev.com/e46fe984a9491f211ad68d006c5ac9de8e813439/client/site_tests/video_ChromeHWDecodeUsed/control.vp8.arc
[modify] https://crrev.com/e46fe984a9491f211ad68d006c5ac9de8e813439/client/site_tests/video_ChromeHWDecodeUsed/control.vp8.mse

According to comment #4, the video_ChromeHWDecodeUsed.h264 is the one that fails on the two information builders. And we run this test locally in tricky and peach_pit, it passed locally (using ToT chrome os + ToT Chrome). We also tested vp8/h264_mse and they both passed locally.

I would try on a build downloaded from the pfq builders. We had instances where test passed on local builds but did fail on pfq builds. 

Anyway, the CL has been merged so if there is a real regression, we will see in the lab.

This seems to be failing also on kevin-release with this error:

09/17 11:41:03.636 DEBUG|     site_autotest:0194| AUTOTEST_STATUS::		ERROR	video_ChromeHWDecodeUsed	video_ChromeHWDecodeUsed	timestamp=1474137663	localtime=Sep 17 11:41:03	Media.GpuVideoDecoderInitializeStatus not loaded or histogram bucket not found or histogram bucket found at < 100%

see these auto-filed bugs:

 ☆	647971	----	----	1	----	----	Untriaged	----	[bvt-cq] video_ChromeHWDecodeUsed Failure on kevin-release/R54-8743.31.0 autofiled	Chrome	10 hours ago	 
 ☆	647954	----	----	1	----	----	Untriaged	----	[bvt-cq] video_ChromeHWDecodeUsed Failure on kevin-release/R54-8743.31.0 autofiled	Chrome	12 hours ago	 
 ☆	647952	----	----	1	----	----	Untriaged	----	[bvt-cq] video_ChromeHWDecodeUsed Failure on kevin-release/R54-8743.31.0 autofiled	Chrome	12 hours ago	 

It also failed on cyan-paladin just recently with this error:

09/17 18:59:18.263 DEBUG|        base_utils:0185| Running 'android-sh -c "getprop sys.boot_completed"'
09/17 18:59:20.443 DEBUG|        base_utils:0185| Running 'android-sh -c "getprop sys.boot_completed"'
09/17 18:59:22.563 DEBUG|        base_utils:0185| Running 'android-sh -c "getprop sys.boot_completed"'
09/17 18:59:24.753 DEBUG|        base_utils:0185| Running 'android-sh -c "getprop sys.boot_completed"'
09/17 18:59:26.879 DEBUG|        base_utils:0185| Running 'android-sh -c "getprop sys.boot_completed"'
09/17 18:59:27.058 ERROR|        site_utils:0932| Android did not boot!
09/17 18:59:27.101 DEBUG|        base_utils:0185| Running 'logger "autotest finished iteration /usr/local/autotest/results/default/video_ChromeHWDecodeUsed/sysinfo/iteration.1"

but I don't know if this is worth looking into because there may have been a bad CL in there, and also it sounds like "android did not boot" may happen before the test does anything specific.  (Just guessing.)

"Android did not boot!" is issue 648359

any update on this ?

Adding current gardeners.

From wmatrix, video_ChromeHWDecodeUsed started to fail since 8810.0.0. The failed tests are all mse (control.vp8.mse, control.vp9.mse, and control.h264.mse).

Owen. Please take a look.

Is #32 same as Issue 647526 ?

video_ChromeHWDecodeUsed started to pass since R55-8865.0.0.

https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=video_ChromeHWDecodeUsed&days_back=45&releases=55

verified as per wmatrix results.