video_VDAPerf still fails on daisy and daisy_spring.
I am going to investigate the reason and fix if possible, instead of owenlin@.

The permission of /usr/local on daisy_spring is drwx------.
It should be drwxr-xr-x.
I think this leads "Permission denied" in the above log.
If so, the wrong permission is set, flaky or depending on the ChromeOS version, since video_VDAPerf is failed and passed on the same lab device.

FYI, similar problem observed with video_VideoDecodeAccelerator test on daisy_skate on 10176.22.0. Logs: https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/163932154-chromeos-test/chromeos4-row9-rack5-host19

This error is shown in several tests, for example, video_VDAPerf, video_VEAPerf, and video_VideoDecodeAccelerator.
It is still flaky and isn't limited to daisy board. video_VEAPerf failed on peppy.

vsuley@, may I ask you to fix this?
I think this is an infra issue and it would be helpful to ask advice to infra developers.

Another failure of video_VideoDecodeAccelerator on peppy/R65-10329.0.0
https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/171454820-chromeos-test/chromeos4-row6-rack12-host5/debug/

Update so far. I'm trying to collect some useful information to pass on to the infra team to help investigate. First step was to try and construct a query that would return all related test failures. I found the test failure mentioned in C#8 on go/stainless and unfortunately the failure reason is not specific. 
https://stainless.corp.google.com/search?view=list&first_date=2018-01-10&last_date=2018-02-06&test=video_VideoDecodeAccelerator.*&build=%5ER66%5C-10329%5C.0%5C.0%24&model=%5Epeppy%24&hostname=%5Echromeos4-row6-rack12-host5%24&exclude_cts=false&exclude_not_run=false&exclude_non_release=false&exclude_au=false&exclude_acts=false&exclude_retried=false&exclude_non_production=false

I used part of the reason string to see how many tests were failing in a related manner. Surprisingly all such failures are coming from video tests. Does anyone have some clues as to why this might be? 
https://stainless.corp.google.com/search?view=list&first_date=2018-01-10&last_date=2018-02-06&reason=Unable+to+parse+fail+reason.*&exclude_cts=false&exclude_not_run=false&exclude_non_release=false&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=true

xixuan@, do you have any ideas on why this could be happening?

Issue 843043 has been merged into this issue.

 Issue 847131  has been merged into this issue.

 Issue 339655  has been merged into this issue.

Issue 371993 has been merged into this issue.

Issue 688068 has been merged into this issue.

there seems to be a long history of this.  the fact that it's affecting a lot of tests at random implies there's something in core code causing the breakage.

i've been getting periodic e-mails about logging_UserCrash failing with "Missing PID in crasher output." when in reality the failure is:
05/31 23:08:00.829 DEBUG|   user_crash_test:0240| Running crasher: ['su', 'chronos', '-c', '/usr/local/autotest/tests/logging_UserCrash/src/crasher/crasher_nobreakpad']
05/31 23:08:00.866 DEBUG|   user_crash_test:0254| Crasher output:
shell-init: error retrieving current directory: getcwd: cannot access parent directories: Permission denied
bash: /usr/local/autotest/tests/logging_UserCrash/src/crasher/crasher_nobreakpad: Permission denied

 Issue 856989  has been merged into this issue.

 Issue 856913  has been merged into this issue.

The permission denied failure pattern happens from time to time:
It first has:
  "shell-init: error retrieving current directory: getcwd: cannot access parent directories: Permission denied"

then:
  "bash: /path/to/binary: Permission denied"

However, probing permission seems ok. For example:
http://sponge/target?tab=Output+Files&sortBy=STATUS&show=ALL&id=90b9b7e8-4d06-4945-a61a-294c1aa02af4&target=daisy-release/R69-10857.0.0/bvt-perbuild/video_VideoDecodeAccelerator.h264

TestFail: Cannot execute command /usr/local/autotest/deps/chrome_test/test_src/out/Release/video_decode_accelerator_unittest. Permissions: /usr/local/autotest/deps/chrome_test/test_src/out/Release/video_decode_accelerator_unittest 0100755; /usr/local/autotest/deps/chrome_test/test_src/out/Release 040755; /usr/local/autotest/deps/chrome_test/test_src/out 040755; /usr/local/autotest/deps/chrome_test/test_src 040755; /usr/local/autotest/deps/chrome_test 040755; /usr/local/autotest/deps 040755; /usr/local/autotest 040755; /usr/local 040700; /usr 040755; / 040755

For example, video_VideoDecodeAccelerator.vp8 fails from time to time because of this.

http://stainless/search?exclude_cts=true&exclude_non_release=true&board=%5Edaisy%5C_skate%24&test=%5Evideo%5C_VideoDecodeAccelerator%5C.vp8%24&suite=%5Ebvt%5C-perbuild%24&view=matrix&col=build&row=board_model&first_date=2018-07-04&last_date=2018-07-10

And the DUT for failed cases were not the same. And the failed DUT can pass the test sometimes (for example http://sponge/invocations?searchFor=video_VideoDecodeAccelerator%20chromeos4-row9-rack5-host15) So we might rule out hardware/storage issue.

Deleted: Screen Shot 2018-07-10 at 1.58.13 PM.jpg 148 KB

	Screen Shot 2018-07-10 at 1.58.13 PM.jpg 148 KB View Download

However, for recent video_VideoDecodeAccelerator failed report, most of them are from daisy_skate or daisy. 
https://sponge.corp.google.com/invocations?searchFor=video_VideoDecodeAccelerator+status:failed