[bvt-cq] video_VideoSanity test fails pretty regularly on elm and oak boards. |
||||||
Issue descriptionIf you do an Advanced Search in the chromiumos bugs, using the terms "video_VideoSanity" and "autofiled", you will see that there are 44 autofiled bugs for video_VideoSanity failing, mostly on oak & elm boards, since Dec. 24. The most common failure reason seems to be: Expected current time >10s. Waited for 120s although sometimes the failure message is: Unhandled AttributeError: 'NetworkController' object has no attribute 'InitializeIfNeeded' Please either fix this test or disable it in bvt-cq for these boards.
,
Jan 11 2017
,
Jan 11 2017
I'll take a first look at it. Will ping for help if needed.
,
Jan 12 2017
What's the current status of this? Has there been any progress?
,
Jan 13 2017
Seems like something is wrong on Oak & Elm with H264 playback when Chrome is started in "ARC mode". Here is a link to ChromeOS changelog, but nothing stands out as super suspicious right off the bat. https://crosland.corp.google.com/log/9114.0.0..9115.0.0 Given that the failure is specific to ARC, I tried looking at the ARC diff on the given page, but can't tell if it's broken because it is showing me no changes (tried various version combination): https://android-build-uber.corp.google.com/repo.html?last_bid=3598214&bid=3598215&branch=git_mnc-dr-arc-dev
,
Jan 13 2017
Does anyone here know exactly what starting Chrome in "arc mode" does? I'm trying to find manual counterparts of the automated steps but blocked until Chrome's 'arc mode' makes sense to me. Looking through the source is leading me down a windier path that will take longer to figure out.
,
Jan 13 2017
I've reached out to Victor Hsieh and Rohit for help, will update here soon as I get some more info.
,
Jan 13 2017
So, "Arc mode" simply means that we test with Play store enabled to see if it interferes with the otherwise simple test case. In the case of Oak, it seems that Play store is making Chrome crash. Here is a feedback report from my Oak devce: 51580870464. I think this is a known issue though, will look up and update.
,
Jan 14 2017
Alright, the Chrome Crash I mentioned in #8 may be a separate issue - it only happens when the Play store UI is present and may have already been fixed. I ran the test locally a few times and while the playback is supposed to go for 10s, it gets stuck (appears as if paused) at a seemingly random point in time. I.e. say it'll go for 6.3 seconds and then get stuck on one frame. The test then times out after 2 mins and fails. Seems like an actual product glitch. Here's a link to phone camera footage of the test run: https://drive.google.com/open?id=0B6UuVwtpgIe4R3QzNkh5SXJFMWM
,
Jan 16 2017
,
Jan 16 2017
I'll disable this for elm and oak first. Then I'll investigate.
,
Jan 18 2017
I have a CL to disable the test. https://chromium-review.googlesource.com/#/c/429790. I discussed with Kuang-che. I'll try to debug again tomorrow. If I can't find out why, I'll disable h264 arc video_VideoSanity on elm and oak tomorrow.
,
Jan 19 2017
- On elm, the video playback seemed to drop frames in arc mode when running video_VideoSanity test. Playing the same video in chrome was smooth. The is a test-only issue. - Dropping frames happened on vp9, vp8, and h264 videos. video_VideoSanity only failed for h264 probably because the size of h264 test video is bigger than than the others. - I used python -m SimpleHttpServer to play the same video when the test was running. The video playback was smooth. The test failure started in 9115.0.0. I'll bisect between 9112.0.0-9115.0.0.
,
Jan 19 2017
I've sent https://chromium-review.googlesource.com/#/c/429790 to CQ. It will disable video_VideoSanity on elm and oak in arc mode.
,
Jan 19 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/62bece0ab55193f83f7ea4df15ade0671b3c1a21 commit 62bece0ab55193f83f7ea4df15ade0671b3c1a21 Author: Wu-Cheng Li <wuchengli@google.com> Date: Wed Jan 18 07:54:34 2017 video_VideoSanity: skip elm and oak in arc mode. The test is flaky on elm and oak in arc mode. Disable them until figuring out why. BUG= chromium:679864 TEST=Run the test on elm. Change-Id: I474bd0d8437b2a13749b4f6782062b05f9bda8d7 Reviewed-on: https://chromium-review.googlesource.com/429790 Commit-Ready: Wu-cheng Li <wuchengli@chromium.org> Tested-by: Wu-cheng Li <wuchengli@chromium.org> Reviewed-by: Kuang-che Wu <kcwu@chromium.org> Reviewed-by: Wu-cheng Li <wuchengli@chromium.org> [modify] https://crrev.com/62bece0ab55193f83f7ea4df15ade0671b3c1a21/client/site_tests/video_VideoSanity/video_VideoSanity.py [modify] https://crrev.com/62bece0ab55193f83f7ea4df15ade0671b3c1a21/client/site_tests/video_VideoSanity/control.h264.arc [modify] https://crrev.com/62bece0ab55193f83f7ea4df15ade0671b3c1a21/client/site_tests/video_VideoSanity/control.vp8.arc [modify] https://crrev.com/62bece0ab55193f83f7ea4df15ade0671b3c1a21/client/site_tests/video_VideoSanity/control.vp9.arc
,
Jan 20 2017
I used tot chrome os source to run test_that. The test failed on 9112.0.0-9115.0.0 images. So it's probably not related to chrome os image, chrome, or android. The issue may be in the tests or firmware (assuming firmware doesn't rollback when flashing an image). The test is disabled. I've changed the priority to p2.
,
Feb 3 2017
I saw a CQ failure earlier this week, that looks like the test timed out (the DUT seems to have passed provision). Related? https://uberchromegw.corp.google.com/i/chromeos/builders/veyron_minnie-paladin/builds/1505/steps/HWTest%20%5Barc-bvt-cq%5D/logs/stdio
,
Feb 16 2017
Re 17: I just saw this comment. I don't see any log related to video_VideoSanity there. The whole log of HWTest was missing.
,
Mar 31 2017
Obsolete. All video tests in ARC mode were removed. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by vsu...@chromium.org
, Jan 11 2017