New issue
Advanced search Search tips

Issue 900857 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Nov 21
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Feature

Blocked on:
issue 901763
issue 906493
issue 906514

Blocking:
issue 852302



Sign in to add a comment

Run tast.video.WebRTC* tests on Chrome/ChromeOS CQ

Project Member Reported by keiichiw@chromium.org, Nov 1

Issue description

We want to run the following tests to catch camera related bugs as early as we can:
- tast.video.WebRTCCamera
- tast.video.WebRTCPeerConnectionWithCamera(H264|VP8)

Note that this issues is a part of Issue 852302.

 
Technically, it can be done by just removing "informational" attributes from those tests. But, we should be careful not to block any good CLs.

In Chrome CQ, these tests will run with vivid on amd64-generic VM.
Though we don't have amd64-generic VM where these tests are running in the release test, we have betty-release, which is also a VM board.
TastVMTest looks stable there.
https://cros-goldeneye.corp.google.com/chromeos/legoland/builderHistory?buildConfig=betty-release&buildBranch=master

In Chrome OS CQ, they will run on many real devices.
I think these tests are also stable enough on boards that are used in CQ. (Please see the results after R72-11209.0.0, where  Issue 898745  was fixed.)
https://stainless.corp.google.com/search?view=matrix&row=model&col=build&first_date=2018-10-26&last_date=2018-11-01&test=tast.video.WebRTC&exclude_cts=false&exclude_not_run=false&exclude_non_release=true&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=false

So, I wonder if we can remove "informational" attribute from these tests.
If I missed some points, please let me know.
Description: Show this description
Created a CL: crrev.com/c/1312537
Ah, sorry for the duplication. I see that these failures are already being discussed at https://crrev.com/c/1312537.
Blockedon: 901763
Project Member

Comment 7 by bugdroid1@chromium.org, Nov 17

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast/+/7e0a96aceb089de8baffa068258215fd1c7a1ba1

commit 7e0a96aceb089de8baffa068258215fd1c7a1ba1
Author: Keiichi Watanabe <keiichiw@chromium.org>
Date: Sat Nov 17 03:50:40 2018

tast: Add "chrome_internal" software dependency

Add a dependency "chrome_internal" for features that requiring
official (internal) Chrome, rather than Chromium.
e.g. Chromium cannot encode and decode H.264 videos.

BUG= chromium:901763 ,  chromium:900857 
TEST=tast run for non-internal VM with CL:1319294
CQ-DEPEND=CL:1319354

Change-Id: I73affe2aa1e8729d8b914dcda8d72bdc5f6f6e26
Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1319292
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: Shuhei Takahashi <nya@chromium.org>

[modify] https://crrev.com/7e0a96aceb089de8baffa068258215fd1c7a1ba1/src/chromiumos/cmd/local_test_runner/main.go
[modify] https://crrev.com/7e0a96aceb089de8baffa068258215fd1c7a1ba1/docs/test_dependencies.md

Project Member

Comment 8 by bugdroid1@chromium.org, Nov 17

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/7c840e6cd4cc7559b6363a9f95384d2421356786

commit 7c840e6cd4cc7559b6363a9f95384d2421356786
Author: Keiichi Watanabe <keiichiw@chromium.org>
Date: Sat Nov 17 03:50:40 2018

tast-use-flags: Add chrome_internal flag to IUSE.

amd64-generic builder in Chromium/ChromeOS infra uses Chromium,
rather than Chrome.
Since Chromium's features are somehow limited, we want to skip
some tests on the builder.
For example, encoding and decoding H.264 video are unavailable in Chromium.

This patch makes tast-use-flags capture chrome_internal USE flags so that
we can skip test cases requiring Chrome (non-Chromium) features.

BUG= chromium:901763 ,  chromium:900857 
TEST=emerge tast-use-flags

Change-Id: Ie7abc3fe0c208bad640f767c5b5a6314aeca959a
Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1319354

[rename] https://crrev.com/7c840e6cd4cc7559b6363a9f95384d2421356786/chromeos-base/tast-use-flags/tast-use-flags-0.0.1-r14.ebuild
[modify] https://crrev.com/7c840e6cd4cc7559b6363a9f95384d2421356786/chromeos-base/tast-use-flags/tast-use-flags-0.0.1.ebuild

Project Member

Comment 9 by bugdroid1@chromium.org, Nov 17

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/d8ff4b33c655a8b9daa279f63d8fd8fe600db9fc

commit d8ff4b33c655a8b9daa279f63d8fd8fe600db9fc
Author: Keiichi Watanabe <keiichiw@chromium.org>
Date: Sat Nov 17 03:50:41 2018

video.WebRTC*: Remove "informational" attribute

Remove "informational" attributes from
- tast.video.WebRTCCamera,
- tast.video.WebRTCPeerConnCameraH264 and
- tast.video.WebRTCPeerConnCameraVP8
so that these tests will run in Chrome CQ and Chrome OS CQ.

BUG= chromium:900857 
TEST=tast run <amd64-generic VM> video.WebRTC*

Change-Id: I5f0220edab5ab6ae9b49a0bdb51f9fc206f937f8
Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1312537
Reviewed-by: Dan Erat <derat@chromium.org>

[modify] https://crrev.com/d8ff4b33c655a8b9daa279f63d8fd8fe600db9fc/src/chromiumos/tast/local/bundles/cros/video/webrtc_camera.go
[modify] https://crrev.com/d8ff4b33c655a8b9daa279f63d8fd8fe600db9fc/src/chromiumos/tast/local/bundles/cros/video/webrtc_peer_conn_camera_vp8.go
[modify] https://crrev.com/d8ff4b33c655a8b9daa279f63d8fd8fe600db9fc/src/chromiumos/tast/local/bundles/cros/video/webrtc_peer_conn_camera_h264.go

Project Member

Comment 10 by bugdroid1@chromium.org, Nov 17

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/7226517aa5881a2b982c065a0cc441d15d4ba31d

commit 7226517aa5881a2b982c065a0cc441d15d4ba31d
Author: Keiichi Watanabe <keiichiw@chromium.org>
Date: Sat Nov 17 03:50:39 2018

video: Add "chrome_internal" dependency in tests playing H264 videos

Add "chrome_internal" dependency in tests using H.264 encoding/decoding
features on browsers; H.264 encoding and decoding are available in
Chrome, but not in Chromium.

BUG= chromium:901763 ,  chromium:900857 
TEST=tast run -buildlocalrunner -checktestdeps always \
     <non-internal amd64-generic VM> video.* with CL:1319354 and CL:1319292
CQ-DEPEND=CL:1319354, CL:1319292

Change-Id: I8e88124a02dfd6036d1f6c74031450738933c5e8
Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1319294
Reviewed-by: Dan Erat <derat@chromium.org>

[modify] https://crrev.com/7226517aa5881a2b982c065a0cc441d15d4ba31d/src/chromiumos/tast/local/bundles/cros/video/webrtc_peer_conn_camera_h264_perf.go
[modify] https://crrev.com/7226517aa5881a2b982c065a0cc441d15d4ba31d/src/chromiumos/tast/local/bundles/cros/video/webrtc_peer_conn_camera_h264.go
[modify] https://crrev.com/7226517aa5881a2b982c065a0cc441d15d4ba31d/src/chromiumos/tast/local/bundles/cros/video/playback_perf_h264.go
[modify] https://crrev.com/7226517aa5881a2b982c065a0cc441d15d4ba31d/src/chromiumos/tast/local/bundles/cros/video/play_h264.go

Project Member

Comment 11 by bugdroid1@chromium.org, Nov 19

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/40cb081f6ae71be191546d3b866ddf2e6722a9e4

commit 40cb081f6ae71be191546d3b866ddf2e6722a9e4
Author: Keiichi Watanabe <keiichiw@chromium.org>
Date: Mon Nov 19 04:44:51 2018

Revert "video.WebRTC*: Remove "informational" attribute"

This reverts commit d8ff4b33c655a8b9daa279f63d8fd8fe600db9fc.

Reason for revert: Failing on bob-paladin  crbug.com/906493 

Original change's description:
> video.WebRTC*: Remove "informational" attribute
>
> Remove "informational" attributes from
> - tast.video.WebRTCCamera,
> - tast.video.WebRTCPeerConnCameraH264 and
> - tast.video.WebRTCPeerConnCameraVP8
> so that these tests will run in Chrome CQ and Chrome OS CQ.
>
> BUG= chromium:900857 
> TEST=tast run <amd64-generic VM> video.WebRTC*
>
> Change-Id: I5f0220edab5ab6ae9b49a0bdb51f9fc206f937f8
> Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
> Reviewed-on: https://chromium-review.googlesource.com/1312537
> Reviewed-by: Dan Erat <derat@chromium.org>

Bug:  chromium:900857 ,  chromium:906493 
Change-Id: I143af696352c8579938a04d265a588ad7be7978b
Reviewed-on: https://chromium-review.googlesource.com/c/1341290
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Trybot-Ready: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Shuhei Takahashi <nya@chromium.org>

[modify] https://crrev.com/40cb081f6ae71be191546d3b866ddf2e6722a9e4/src/chromiumos/tast/local/bundles/cros/video/webrtc_camera.go
[modify] https://crrev.com/40cb081f6ae71be191546d3b866ddf2e6722a9e4/src/chromiumos/tast/local/bundles/cros/video/webrtc_peer_conn_camera_vp8.go
[modify] https://crrev.com/40cb081f6ae71be191546d3b866ddf2e6722a9e4/src/chromiumos/tast/local/bundles/cros/video/webrtc_peer_conn_camera_h264.go

Blockedon: 906493
Blockedon: 906514
Project Member

Comment 14 by bugdroid1@chromium.org, Nov 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/b3406e4da8ba30d5b961b24912c412a3600c03ef

commit b3406e4da8ba30d5b961b24912c412a3600c03ef
Author: Keiichi Watanabe <keiichiw@chromium.org>
Date: Wed Nov 21 09:25:20 2018

Reland "video.WebRTC*: Remove "informational" attribute"

This is a reland of d8ff4b33c655a8b9daa279f63d8fd8fe600db9fc

Original change's description:
> video.WebRTC*: Remove "informational" attribute
>
> Remove "informational" attributes from
> - tast.video.WebRTCCamera,
> - tast.video.WebRTCPeerConnCameraH264 and
> - tast.video.WebRTCPeerConnCameraVP8
> so that these tests will run in Chrome CQ and Chrome OS CQ.
>
> BUG= chromium:900857 
> TEST=tast run <amd64-generic VM> video.WebRTC*
>
> Change-Id: I5f0220edab5ab6ae9b49a0bdb51f9fc206f937f8
> Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
> Reviewed-on: https://chromium-review.googlesource.com/1312537
> Reviewed-by: Dan Erat <derat@chromium.org>

Bug:  chromium:900857 
Change-Id: I70ca79a46264a14bd16d3881881cd2edea6ef6de
Reviewed-on: https://chromium-review.googlesource.com/1343831
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Shuhei Takahashi <nya@chromium.org>

[modify] https://crrev.com/b3406e4da8ba30d5b961b24912c412a3600c03ef/src/chromiumos/tast/local/bundles/cros/video/webrtc_camera.go
[modify] https://crrev.com/b3406e4da8ba30d5b961b24912c412a3600c03ef/src/chromiumos/tast/local/bundles/cros/video/webrtc_peer_conn_camera_vp8.go
[modify] https://crrev.com/b3406e4da8ba30d5b961b24912c412a3600c03ef/src/chromiumos/tast/local/bundles/cros/video/webrtc_peer_conn_camera_h264.go

Status: Fixed (was: Started)
tast.video.WebRTC* tests is running on Chrome OS CQ and they look stable for now.
They will start running once Chrome's LKGM is updated.
So, let me close this issue.

Sign in to add a comment