actually first failure I can find is build 6212

https://luci-logdog.appspot.com/v/?s=chromeos/bb/chromeos/nyan_kitty-paladin/6212/+/recipes/steps/HWTest__bvt-cq_/0/stdout

Pawel, can you triage this further?

Does look like CL/chrome in R71-11140.0.0 is the culprit as it was passing prior:

https://stainless.corp.google.com/search?view=matrix&row=test&col=build&first_date=2018-10-03&last_date=2018-10-09&test=video_ChromeHWDecodeUsed.h264&board=nyan_kitty&exclude_cts=true&exclude_not_run=false&exclude_non_release=true&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=false

11140 did get new chrome, 71.0.3572.0

I checked chrome log.

[19382:20191:1009/164846.436749:VERBOSE1:v4l2_video_decode_accelerator.cc(2198)] CreateInputBuffers(): Setting error state:4
error is thrown from here. https://cs.chromium.org/chromium/src/media/gpu/v4l2/v4l2_video_decode_accelerator.cc?q=video_decode_accelerator.cc&sq=package:chromium&dr&l=2197

This is may be due to Alex's V4L2VDA refactoring CL.
Assign to acourbot@.

Taking a look. This device (as well as nyan-big which shows the same issue) uses a Tegra K1 which has its own v4l2_device implementation. I may have overlooked this during the V4L2VDA overhaul.

The issue comes from the Tegra libv4l2 library that accepts VIDIOC_QUERYBUF calls only if the number of planes exactly matches those of the format. And on top of that VIDIOC_G_FMT also returns incorrect plane number information. Currently working on fixes for both these issues.

Submitted https://chromium-review.googlesource.com/c/chromium/src/+/1275726 and https://chromium-review.googlesource.com/c/chromium/src/+/1275727 to fix this issue, PTAL!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/cec52397f2ea0fb8ac322378d229f1a454edf972

commit cec52397f2ea0fb8ac322378d229f1a454edf972
Author: Alexandre Courbot <acourbot@chromium.org>
Date: Fri Oct 12 06:30:03 2018

media/gpu/v4l2: use actual number of planes in ioctls

The Tegra V4L2 library has a bug in its handling of VIDIOC_QUERYBUF that
makes it fail if the number of planes of the passed v4l2_buffer does not
exactly match the number of planes of the currently set format. This CL
implements a workaround by first running a VIDIOC_G_FMT ioctl in
AllocateBuffers(), and then passing the returned number of planes to the
buffer constructor so the correct value can be passed to
VIDIOC_QUERYBUF.

And since we have this value, we also use it in VIDIOC_DQBUF.

BUG= 893661 
TEST=Checked that VDA unittest was passing on hana (Tegra devices have
another bug that is fixed in the next CL).

Change-Id: I830334de1cf322302fd476ac5535313ebc65900d
Reviewed-on: https://chromium-review.googlesource.com/c/1275726
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Pawel Osciak <posciak@chromium.org>
Cr-Commit-Position: refs/heads/master@{#599116}
[modify] https://crrev.com/cec52397f2ea0fb8ac322378d229f1a454edf972/media/gpu/v4l2/v4l2_device.cc
[modify] https://crrev.com/cec52397f2ea0fb8ac322378d229f1a454edf972/media/gpu/v4l2/v4l2_device.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/029544dab67d79059b7610a92890091520ef7603

commit 029544dab67d79059b7610a92890091520ef7603
Author: Alexandre Courbot <acourbot@chromium.org>
Date: Fri Oct 12 07:54:06 2018

media/gpu/v4l2: workaround VIDIOC_G_FMT bug in libtegrav4l2.so.

libtegrav4l2.so's implementation of V4L2_G_FMT has a bug which will make
it return 0 planes for single-plane multiplanar formats. We obviously
cannot have a multiplanar format with 0 planes, so add a workaround at
the fake-ioctl level. Implement it as a case-switch for future-proofing
against other workarounds we will likely have to add.

Also add a few recommendations in v4l2_device.cc against the temptation
to spare some memory by using an appropriately-sized array of
v4l2_planes: it turns out libv4l2 also likes to write above the number
of passed planes, and using anything shorter than VIDEO_MAX_PLANES will
result in memory corruption DCHECKS being raised.

BUG= 893661 
TEST=Checked that VDA unittest was passing on both hana and nyan_big.

Change-Id: I7ea638f19f5bf0bdfaa0a47885577eb9dec4c225
Reviewed-on: https://chromium-review.googlesource.com/c/1275727
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Kuang-che Wu <kcwu@chromium.org>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#599135}
[modify] https://crrev.com/029544dab67d79059b7610a92890091520ef7603/media/gpu/v4l2/tegra_v4l2_device.cc
[modify] https://crrev.com/029544dab67d79059b7610a92890091520ef7603/media/gpu/v4l2/v4l2_device.cc

All fixes merged, closing!

NOTE:
Branch was cut and the last Chrome is 71.0.3575.1.
The two CLs were landed on 72.0.3580.0.
Though PSA for back-porting is not announced yet, we may need to cherry-pick them to M71.

Let me reopen this issue then.

Hi, TPM, we need to cherry-pick #13 and #14 CLs to M71, in order to fix HW video decoder initialization failure on nyan devices.

Your change meets the bar and is auto-approved for M71. Please go ahead and merge the CL to branch 3578 manually. Please contact milestone owner if you have questions.
Owners: benmason@(Android), kariahda@(iOS), kbleicher@(ChromeOS), govind@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Created cherry-pick: https://chromium-review.googlesource.com/c/chromium/src/+/1282087

Waiting for owner approval.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8aed261ba7da81fc7fa980540ee077fef5b7dc95

commit 8aed261ba7da81fc7fa980540ee077fef5b7dc95
Author: Alexandre Courbot <acourbot@chromium.org>
Date: Tue Oct 16 06:45:22 2018

media/gpu/v4l2: workaround VIDIOC_G_FMT bug in libtegrav4l2.so.

libtegrav4l2.so's implementation of V4L2_G_FMT has a bug which will make
it return 0 planes for single-plane multiplanar formats. We obviously
cannot have a multiplanar format with 0 planes, so add a workaround at
the fake-ioctl level. Implement it as a case-switch for future-proofing
against other workarounds we will likely have to add.

Also add a few recommendations in v4l2_device.cc against the temptation
to spare some memory by using an appropriately-sized array of
v4l2_planes: it turns out libv4l2 also likes to write above the number
of passed planes, and using anything shorter than VIDEO_MAX_PLANES will
result in memory corruption DCHECKS being raised.

BUG= 893661 
TEST=Checked that VDA unittest was passing on both hana and nyan_big.

Change-Id: I7ea638f19f5bf0bdfaa0a47885577eb9dec4c225
Reviewed-on: https://chromium-review.googlesource.com/c/1275727
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Kuang-che Wu <kcwu@chromium.org>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#599135}(cherry picked from commit 029544dab67d79059b7610a92890091520ef7603)
Reviewed-on: https://chromium-review.googlesource.com/c/1282087
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Cr-Commit-Position: refs/branch-heads/3578@{#35}
Cr-Branched-From: 4226ddf99103e493d7afb23a4c7902ee496108b6-refs/heads/master@{#599034}
[modify] https://crrev.com/8aed261ba7da81fc7fa980540ee077fef5b7dc95/media/gpu/v4l2/tegra_v4l2_device.cc
[modify] https://crrev.com/8aed261ba7da81fc7fa980540ee077fef5b7dc95/media/gpu/v4l2/v4l2_device.cc

Cherry-picked to M71, closing.

This is landed in 71.0.3578.10 (diff https://chromium.googlesource.com/chromium/src/+log/71.0.3578.9..71.0.3578.10?n=10000). 

But test is still failing as per stainless results, https://stainless.corp.google.com/search?view=matrix&row=model&col=build&first_date=2018-10-16&last_date=2018-10-22&test=video_ChromeHWDecodeUsed.h264+&build=R71-*&board=nyan*&exclude_cts=false&exclude_not_run=false&exclude_non_release=true&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=false

The following revision refers to this bug: 
https://chromium.googlesource.com/chromium/src.git/+/8aed261ba7da81fc7fa980540ee077fef5b7dc95

Commit: 8aed261ba7da81fc7fa980540ee077fef5b7dc95
Author: acourbot@chromium.org
Commiter: acourbot@chromium.org
Date: 2018-10-16 06:45:22 +0000 UTC

media/gpu/v4l2: workaround VIDIOC_G_FMT bug in libtegrav4l2.so.

libtegrav4l2.so's implementation of V4L2_G_FMT has a bug which will make
it return 0 planes for single-plane multiplanar formats. We obviously
cannot have a multiplanar format with 0 planes, so add a workaround at
the fake-ioctl level. Implement it as a case-switch for future-proofing
against other workarounds we will likely have to add.

Also add a few recommendations in v4l2_device.cc against the temptation
to spare some memory by using an appropriately-sized array of
v4l2_planes: it turns out libv4l2 also likes to write above the number
of passed planes, and using anything shorter than VIDEO_MAX_PLANES will
result in memory corruption DCHECKS being raised.

BUG= 893661 
TEST=Checked that VDA unittest was passing on both hana and nyan_big.

Change-Id: I7ea638f19f5bf0bdfaa0a47885577eb9dec4c225
Reviewed-on: https://chromium-review.googlesource.com/c/1275727
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Kuang-che Wu <kcwu@chromium.org>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#599135}(cherry picked from commit 029544dab67d79059b7610a92890091520ef7603)
Reviewed-on: https://chromium-review.googlesource.com/c/1282087
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Cr-Commit-Position: refs/branch-heads/3578@{#35}
Cr-Branched-From: 4226ddf99103e493d7afb23a4c7902ee496108b6-refs/heads/master@{#599034}

The last CL#23 is landed on Chrome OS 11151.5.0.
However, this still fails on M71.

Alex, could you take a look again?

The following tests fail on nyan boards (nyan_big, nyan_blaze, nyan_kitty) on M71 Beta RC (11151.17.0, 71.0.3578.27) with same failure reason as mentioned above:

1. bvt-cq/video_ChromeHWDecodeUsed.h264/video_ChromeHWDecodeUsed.h264
2. bvt-cq/video_ChromeHWDecodeUsed.vp8.mse/video_ChromeHWDecodeUsed.vp8
3. bvt-cq/video_ChromeHWDecodeUsed.h264.mse/video_ChromeHWDecodeUsed.h264
4. bvt-cq/video_ChromeRTCHWDecodeUsed/video_ChromeRTCHWDecodeUsed.vp8
5. bvt-cq/video_ChromeHWDecodeUsed.vp8/video_ChromeHWDecodeUsed.vp8

Stainless results:
https://stainless.corp.google.com/search?view=list&first_date=2018-10-28&last_date=2018-10-30&test=video_ChromeHWDecodeUsed*&build=11151.17.0&board=nyan_*&status=FAIL&status=ERROR&exclude_cts=true&exclude_not_run=false&exclude_non_release=true&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=false

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a50ac2d43f9b3de01f4e6753b129f793a8760fa1

commit a50ac2d43f9b3de01f4e6753b129f793a8760fa1
Author: Alexandre Courbot <acourbot@chromium.org>
Date: Wed Oct 31 06:27:40 2018

media/gpu/v4l2: use actual number of planes in ioctls

The Tegra V4L2 library has a bug in its handling of VIDIOC_QUERYBUF that
makes it fail if the number of planes of the passed v4l2_buffer does not
exactly match the number of planes of the currently set format. This CL
implements a workaround by first running a VIDIOC_G_FMT ioctl in
AllocateBuffers(), and then passing the returned number of planes to the
buffer constructor so the correct value can be passed to
VIDIOC_QUERYBUF.

And since we have this value, we also use it in VIDIOC_DQBUF.

BUG= 893661 
TEST=Checked that VDA unittest was passing on hana (Tegra devices have
another bug that is fixed in the next CL).

Change-Id: I830334de1cf322302fd476ac5535313ebc65900d
Reviewed-on: https://chromium-review.googlesource.com/c/1275726
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Pawel Osciak <posciak@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#599116}(cherry picked from commit cec52397f2ea0fb8ac322378d229f1a454edf972)
Reviewed-on: https://chromium-review.googlesource.com/c/1308958
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Cr-Commit-Position: refs/branch-heads/3578@{#431}
Cr-Branched-From: 4226ddf99103e493d7afb23a4c7902ee496108b6-refs/heads/master@{#599034}
[modify] https://crrev.com/a50ac2d43f9b3de01f4e6753b129f793a8760fa1/media/gpu/v4l2/v4l2_device.cc
[modify] https://crrev.com/a50ac2d43f9b3de01f4e6753b129f793a8760fa1/media/gpu/v4l2/v4l2_device.h

That's strange, only one of the two patches has actually been merged into M71. I have just submitted the missing one, so this should now be fixed. Closing the issue.

The following revision refers to this bug: 
https://chromium.googlesource.com/chromium/src.git/+/a50ac2d43f9b3de01f4e6753b129f793a8760fa1

Commit: a50ac2d43f9b3de01f4e6753b129f793a8760fa1
Author: acourbot@chromium.org
Commiter: acourbot@chromium.org
Date: 2018-10-31 06:27:40 +0000 UTC

media/gpu/v4l2: use actual number of planes in ioctls

The Tegra V4L2 library has a bug in its handling of VIDIOC_QUERYBUF that
makes it fail if the number of planes of the passed v4l2_buffer does not
exactly match the number of planes of the currently set format. This CL
implements a workaround by first running a VIDIOC_G_FMT ioctl in
AllocateBuffers(), and then passing the returned number of planes to the
buffer constructor so the correct value can be passed to
VIDIOC_QUERYBUF.

And since we have this value, we also use it in VIDIOC_DQBUF.

BUG= 893661 
TEST=Checked that VDA unittest was passing on hana (Tegra devices have
another bug that is fixed in the next CL).

Change-Id: I830334de1cf322302fd476ac5535313ebc65900d
Reviewed-on: https://chromium-review.googlesource.com/c/1275726
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Pawel Osciak <posciak@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#599116}(cherry picked from commit cec52397f2ea0fb8ac322378d229f1a454edf972)
Reviewed-on: https://chromium-review.googlesource.com/c/1308958
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Cr-Commit-Position: refs/branch-heads/3578@{#431}
Cr-Branched-From: 4226ddf99103e493d7afb23a4c7902ee496108b6-refs/heads/master@{#599034}

Verified using https://stainless.corp.google.com/search?view=matrix&row=model&col=build&first_date=2018-10-23&last_date=2018-11-05&test=video_ChromeHWDecodeUsed.h264&board=nyan*&exclude_cts=false&exclude_not_run=false&exclude_non_release=true&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=false