New issue
Advanced search Search tips

Issue 894606 link

Starred by 1 user

Issue metadata

Status: Unconfirmed
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Feature



Sign in to add a comment

Revisiting video_YouTubePage Autotests

Reported by edward.b...@intel.com, Oct 11

Issue description

UserAgent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0
Platform: eve

Steps to reproduce the problem:
1. test_that --board=${BOARD} ${DUT_IP} e:video_YouTubePage.*
2. 
3. 

What is the expected behavior?
Tests complete successfully.

What went wrong?
Currently video_YouTubePage does not run as web-release-qa.youtube.com is either not public or offline.
/tmp/test_that_results_42HNMe/results-1-video_YouTubePage.frame_drop                                [  FAILED  ]
/tmp/test_that_results_42HNMe/results-1-video_YouTubePage.frame_drop                                  FAIL: YouTube page failed to load.
/tmp/test_that_results_42HNMe/results-1-video_YouTubePage.frame_drop/video_YouTubePage.frame_drop   [  FAILED  ]
/tmp/test_that_results_42HNMe/results-1-video_YouTubePage.frame_drop/video_YouTubePage.frame_drop     FAIL: YouTube page failed to load.
/tmp/test_that_results_42HNMe/results-2-video_YouTubePage.ending                                    [  FAILED  ]
/tmp/test_that_results_42HNMe/results-2-video_YouTubePage.ending                                      FAIL: YouTube page failed to load.
/tmp/test_that_results_42HNMe/results-2-video_YouTubePage.ending/video_YouTubePage.ending           [  FAILED  ]
/tmp/test_that_results_42HNMe/results-2-video_YouTubePage.ending/video_YouTubePage.ending             FAIL: YouTube page failed to load.
/tmp/test_that_results_42HNMe/results-3-video_YouTubePage.pausing                                   [  FAILED  ]
/tmp/test_that_results_42HNMe/results-3-video_YouTubePage.pausing                                     FAIL: YouTube page failed to load.
<snip>

After swapping the the URL, all of the tests except ".ending" pass.
/tmp/test_that_results_FwuyH5/results-1-video_YouTubePage.frame_drop                                [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-1-video_YouTubePage.frame_drop/video_YouTubePage.frame_drop   [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-2-video_YouTubePage.ending                                    [  FAILED  ]
/tmp/test_that_results_FwuyH5/results-2-video_YouTubePage.ending                                      ERROR: Current player state "playing" is not the expected state "ended".
/tmp/test_that_results_FwuyH5/results-2-video_YouTubePage.ending/video_YouTubePage.ending           [  FAILED  ]
/tmp/test_that_results_FwuyH5/results-2-video_YouTubePage.ending/video_YouTubePage.ending             ERROR: Current player state "playing" is not the expected state "ended".
/tmp/test_that_results_FwuyH5/results-3-video_YouTubePage.pausing                                   [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-3-video_YouTubePage.pausing/video_YouTubePage.pausing         [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-4-video_YouTubePage.playing                                   [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-4-video_YouTubePage.playing/video_YouTubePage.playing         [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-5-video_YouTubePage.resuming                                  [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-5-video_YouTubePage.resuming/video_YouTubePage.resuming       [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-6-video_YouTubePage.last_second                               [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-6-video_YouTubePage.last_second/video_YouTubePage.last_second [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-7-video_YouTubePage.seeking                                   [  PASSED  ]
/tmp/test_that_results_FwuyH5/results-7-video_YouTubePage.seeking/video_YouTubePage.seeking         [  PASSED  ]

Did this work before? N/A 

Chrome version: Google Chrome 71.0.3567.0  Channel: dev
OS Version: 11138.0.0 (Official Build) dev-channel eve test
Flash Version: 

This issue is to revisit the topic of direct YouTube page test cases. Tests such as desktopui_AudioFeedback and video_YouTubeHTML5 provide coverage for YouTube's embedded iframe API. This is fantastic, but of course prevents simple inspection of the <video> element properties. A bit of maintenance on video_YouTubePage should be very worthwhile. Studying my own browsing tendencies places YouTube easily among my top five use cases. Though I'm not sure how that matches broad Chrome OS usage patterns. In any case, having a few tests to quickly and directly assess such an important scenario should help maintain the overall experience, catch potential issues early, and provide a common point for future discussions.
 
Accidentally clicked N/A for 'Did This work before?'.

Did this work before?
The tests mostly worked before, but needed maintenance. Issue https://bugs.chromium.org/p/chromium/issues/detail?id=599785 back in 2016 flagged video_YouTubePage.ending as needing additional work.
Components: Tests>Fails
Project Member

Comment 3 by bugdroid1@chromium.org, Dec 1

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/a35bf3efb39eaa0120e4aac545bd191ed63d8454

commit a35bf3efb39eaa0120e4aac545bd191ed63d8454
Author: Ed Baker <edward.baker@intel.com>
Date: Sat Dec 01 09:08:41 2018

Update video_YouTubePage tests

For all subtests, the test URL was hoisted out of video_YouTubePage
and passed as an argument from the control files. The URL was also
updated since web-release-qa.youtube.com responds with a 404. Quickly
checking The Wayback Machine seems to indicate it was last up around
July 15th 2013.

The seek time in perform_ending_test was increased since such a short
amount of time was not actually cycling from playing to ended. YouTube
seemed to transition from playing to paused, counted down for the next
autoplay, and cycled back to playing on the next video clip.

Calculating dropped frames percentages was moved into the main
injected JavaScript file and simple wrapper functions added to
the video_YouTubePage class. A fullscreen event listener was also
added to the JavaScript.

Finally, a new subtest was added to verify dropped frame percentages
while cycling fullscreen modes. To fullscreen a YouTube video this test
uses the normal 'f' hotkey. Note that this new subtest uses a Google
Earth video published by Google.

BUG=chromium:894606
TEST=All video_YouTubePage tests pass.
     `test_that --board=${BOARD} ${DUT_IP} e:video_YouTubePage.*`

Change-Id: I141e06d6a9ad28c76e03f5164bccb678a0b0f356
Signed-off-by: Edward Baker <edward.baker@intel.com>
Reviewed-on: https://chromium-review.googlesource.com/1255362
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Casey G Bowman <casey.g.bowman@intel.com>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>

[add] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.fullscreen
[modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/files/video_YouTubePageCommon.js
[modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/cros/input_playback/stylus.py
[modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.resuming
[modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/video_YouTubePage.py
[modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.last_second
[modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.playing
[modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.pausing
[add] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/cros/input_playback/keyboard_f
[modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.ending
[modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.frame_drop
[modify] https://crrev.com/a35bf3efb39eaa0120e4aac545bd191ed63d8454/client/site_tests/video_YouTubePage/control.seeking

Project Member

Comment 4 by bugdroid1@chromium.org, Dec 17

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/5a0ec3144726f22695024865b42e209726067396

commit 5a0ec3144726f22695024865b42e209726067396
Author: Ed Baker <edward.baker@intel.com>
Date: Mon Dec 17 00:26:18 2018

Introduce a YouTube 3D 360 test

This commit introduces a test to help measure 3D 360 degree YouTube
playback. This use case is a great combination of interesting technology
such as WebGL, VP9 decode, and hardware overlays. Of course, at high
resolutions and frame rates.

BUG=chromium:894606
TEST=Test completes and records frame drop information.
     `test_that --board=${BOARD} ${DUT_IP} video_YouTubePage.360`
     `cat <snip long results path>/results-chart.json`

Change-Id: Ib629005c4ea3263f20e165e131796866f6a8b956
Signed-off-by: Edward Baker <edward.baker@intel.com>
Reviewed-on: https://chromium-review.googlesource.com/1255363
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Casey G Bowman <casey.g.bowman@intel.com>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>

[modify] https://crrev.com/5a0ec3144726f22695024865b42e209726067396/client/site_tests/video_YouTubePage/video_YouTubePage.py
[add] https://crrev.com/5a0ec3144726f22695024865b42e209726067396/client/cros/input_playback/keyboard_s
[add] https://crrev.com/5a0ec3144726f22695024865b42e209726067396/client/cros/input_playback/keyboard_d
[add] https://crrev.com/5a0ec3144726f22695024865b42e209726067396/client/site_tests/video_YouTubePage/control.360
[add] https://crrev.com/5a0ec3144726f22695024865b42e209726067396/client/cros/input_playback/keyboard_w

Sign in to add a comment