Revisiting video_YouTubePage Autotests
Reported by
edward.b...@intel.com,
Oct 11
|
||
Issue description
UserAgent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0
Platform: eve
Steps to reproduce the problem:
1. test_that --board=${BOARD} ${DUT_IP} e:video_YouTubePage.*
2.
3.
What is the expected behavior?
Tests complete successfully.
What went wrong?
Currently video_YouTubePage does not run as web-release-qa.youtube.com is either not public or offline.
/tmp/test_that_results_42HNMe/results-1-video_YouTubePage.frame_drop [ FAILED ]
/tmp/test_that_results_42HNMe/results-1-video_YouTubePage.frame_drop FAIL: YouTube page failed to load.
/tmp/test_that_results_42HNMe/results-1-video_YouTubePage.frame_drop/video_YouTubePage.frame_drop [ FAILED ]
/tmp/test_that_results_42HNMe/results-1-video_YouTubePage.frame_drop/video_YouTubePage.frame_drop FAIL: YouTube page failed to load.
/tmp/test_that_results_42HNMe/results-2-video_YouTubePage.ending [ FAILED ]
/tmp/test_that_results_42HNMe/results-2-video_YouTubePage.ending FAIL: YouTube page failed to load.
/tmp/test_that_results_42HNMe/results-2-video_YouTubePage.ending/video_YouTubePage.ending [ FAILED ]
/tmp/test_that_results_42HNMe/results-2-video_YouTubePage.ending/video_YouTubePage.ending FAIL: YouTube page failed to load.
/tmp/test_that_results_42HNMe/results-3-video_YouTubePage.pausing [ FAILED ]
/tmp/test_that_results_42HNMe/results-3-video_YouTubePage.pausing FAIL: YouTube page failed to load.
<snip>
After swapping the the URL, all of the tests except ".ending" pass.
/tmp/test_that_results_FwuyH5/results-1-video_YouTubePage.frame_drop [ PASSED ]
/tmp/test_that_results_FwuyH5/results-1-video_YouTubePage.frame_drop/video_YouTubePage.frame_drop [ PASSED ]
/tmp/test_that_results_FwuyH5/results-2-video_YouTubePage.ending [ FAILED ]
/tmp/test_that_results_FwuyH5/results-2-video_YouTubePage.ending ERROR: Current player state "playing" is not the expected state "ended".
/tmp/test_that_results_FwuyH5/results-2-video_YouTubePage.ending/video_YouTubePage.ending [ FAILED ]
/tmp/test_that_results_FwuyH5/results-2-video_YouTubePage.ending/video_YouTubePage.ending ERROR: Current player state "playing" is not the expected state "ended".
/tmp/test_that_results_FwuyH5/results-3-video_YouTubePage.pausing [ PASSED ]
/tmp/test_that_results_FwuyH5/results-3-video_YouTubePage.pausing/video_YouTubePage.pausing [ PASSED ]
/tmp/test_that_results_FwuyH5/results-4-video_YouTubePage.playing [ PASSED ]
/tmp/test_that_results_FwuyH5/results-4-video_YouTubePage.playing/video_YouTubePage.playing [ PASSED ]
/tmp/test_that_results_FwuyH5/results-5-video_YouTubePage.resuming [ PASSED ]
/tmp/test_that_results_FwuyH5/results-5-video_YouTubePage.resuming/video_YouTubePage.resuming [ PASSED ]
/tmp/test_that_results_FwuyH5/results-6-video_YouTubePage.last_second [ PASSED ]
/tmp/test_that_results_FwuyH5/results-6-video_YouTubePage.last_second/video_YouTubePage.last_second [ PASSED ]
/tmp/test_that_results_FwuyH5/results-7-video_YouTubePage.seeking [ PASSED ]
/tmp/test_that_results_FwuyH5/results-7-video_YouTubePage.seeking/video_YouTubePage.seeking [ PASSED ]
Did this work before? N/A
Chrome version: Google Chrome 71.0.3567.0 Channel: dev
OS Version: 11138.0.0 (Official Build) dev-channel eve test
Flash Version:
This issue is to revisit the topic of direct YouTube page test cases. Tests such as desktopui_AudioFeedback and video_YouTubeHTML5 provide coverage for YouTube's embedded iframe API. This is fantastic, but of course prevents simple inspection of the <video> element properties. A bit of maintenance on video_YouTubePage should be very worthwhile. Studying my own browsing tendencies places YouTube easily among my top five use cases. Though I'm not sure how that matches broad Chrome OS usage patterns. In any case, having a few tests to quickly and directly assess such an important scenario should help maintain the overall experience, catch potential issues early, and provide a common point for future discussions.
,
Oct 12
,
Dec 1
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/a35bf3efb39eaa0120e4aac545bd191ed63d8454 commit a35bf3efb39eaa0120e4aac545bd191ed63d8454 Author: Ed Baker <edward.baker@intel.com> Date: Sat Dec 01 09:08:41 2018 Update video_YouTubePage tests For all subtests, the test URL was hoisted out of video_YouTubePage and passed as an argument from the control files. The URL was also updated since web-release-qa.youtube.com responds with a 404. Quickly checking The Wayback Machine seems to indicate it was last up around July 15th 2013. The seek time in perform_ending_test was increased since such a short amount of time was not actually cycling from playing to ended. YouTube seemed to transition from playing to paused, counted down for the next autoplay, and cycled back to playing on the next video clip. Calculating dropped frames percentages was moved into the main injected JavaScript file and simple wrapper functions added to the video_YouTubePage class. A fullscreen event listener was also added to the JavaScript. Finally, a new subtest was added to verify dropped frame percentages while cycling fullscreen modes. To fullscreen a YouTube video this test uses the normal 'f' hotkey. Note that this new subtest uses a Google Earth video published by Google. BUG=chromium:894606 TEST=All video_YouTubePage tests pass. `test_that --board=${BOARD} ${DUT_IP} e:video_YouTubePage.*` Change-Id: I141e06d6a9ad28c76e03f5164bccb678a0b0f356 Signed-off-by: Edward Baker <edward.baker@intel.com> Reviewed-on: https://chromium-review.googlesource.com/1255362 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Tested-by: Casey G Bowman <casey.g.bowman@intel.com> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [add] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.fullscreen [modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/files/video_YouTubePageCommon.js [modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/cros/input_playback/stylus.py [modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.resuming [modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/video_YouTubePage.py [modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.last_second [modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.playing [modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.pausing [add] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/cros/input_playback/keyboard_f [modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.ending [modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.frame_drop [modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.seeking
,
Dec 17
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/5a0ec3144726f22695024865b42e209726067396 commit 5a0ec3144726f22695024865b42e209726067396 Author: Ed Baker <edward.baker@intel.com> Date: Mon Dec 17 00:26:18 2018 Introduce a YouTube 3D 360 test This commit introduces a test to help measure 3D 360 degree YouTube playback. This use case is a great combination of interesting technology such as WebGL, VP9 decode, and hardware overlays. Of course, at high resolutions and frame rates. BUG=chromium:894606 TEST=Test completes and records frame drop information. `test_that --board=${BOARD} ${DUT_IP} video_YouTubePage.360` `cat <snip long results path>/results-chart.json` Change-Id: Ib629005c4ea3263f20e165e131796866f6a8b956 Signed-off-by: Edward Baker <edward.baker@intel.com> Reviewed-on: https://chromium-review.googlesource.com/1255363 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Casey G Bowman <casey.g.bowman@intel.com> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/5a0ec3144726f22695024865b42e209726067396/client/site_tests/video_YouTubePage/video_YouTubePage.py [add] https://crrev.com/5a0ec3144726f22695024865b42e209726067396/client/cros/input_playback/keyboard_s [add] https://crrev.com/5a0ec3144726f22695024865b42e209726067396/client/cros/input_playback/keyboard_d [add] https://crrev.com/5a0ec3144726f22695024865b42e209726067396/client/site_tests/video_YouTubePage/control.360 [add] https://crrev.com/5a0ec3144726f22695024865b42e209726067396/client/cros/input_playback/keyboard_w |
||
►
Sign in to add a comment |
||
Comment 1 by edward.b...@intel.com
, Oct 11