phoglund@ could you help here?

I'll defer to hbos@ here. Henrik, can you port your codec-choosing code from the browser tests to the CrOS site tests to https://chromium.googlesource.com/chromiumos/third_party/autotest/+/master/client/site_tests/video_WebRtcPeerConnectionWithCamera/?

This could be beneficial to check that h264 works on real CrOS hardware, including hw acceleration. If you don't have the cycles, maybe niklase@ can help with an owner?

That test acquires a real webcam on a CrOS device and runs them through a peer connection in loopback to a video tag. It should be enough to force the h264 codec in SDP. Do we need to set any flags to enable h264?

hbos@ ping? Please update the milestone if it's not correct.

mflodman: Can you delegate?

It didn't make it into M55 so moving to M56.

Bumping to M57. Please update if that's wrong.

@phoglund, would this be covered under your recent automation efforts? 

No, but maybe I can whip something up since I'm working in this area already. My recent automation work is more about preventing CrOS cam regressions and WebRTC perf regressions.

Assigning to phoglund@ according to #10.

I'm curious. How do we force webrtc to use H264 encode? Is there a javascript API or a flag?

You put h264 first in the codec list in the offer/answer SDP: https://cs.chromium.org/chromium/src/chrome/test/data/webrtc/munge_sdp.js?sq=package:chromium

Do you just want h264 for all existing tests here? It sounds to me video_ChromeRTCHWDecodeUsed/ / video_ChromeRTCHWEncode used would be valuable targets to add, but I don't know anything about those tests so I'd appreciate if someone else looks at those. I can add a h264 case to the WebRtcPeerConnectionWithCamera test, and you can re-use the approach I take there. SGTY?

Ok, I think I'll have to clean up the peer connection test first... I found it verifies cam resolutions, which should really be done by the Camera test. I don't think it makes sense to test peer connections with both VGA and 720p, so it's good to get rid of that that axis on the testing matrix before adding the h264/vp8 axis.

Re #14: sounds good.

video_ChromeRTCHWDecodeUsed and video_ChromeRTCHWEncodeUsed do a webrtc loopback. Then the tests check if the histogram exists to decide if hardware decode or encode is used.

https://chromium.googlesource.com/chromiumos/third_party/autotest/+/master/client/site_tests/video_ChromeRTCHWDecodeUsed/loopback.html

Patrik. How's cleaning up the peer connection test going?

https://chromium-review.googlesource.com/#/c/411842/ is in review.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/47bc347fed9dd1aa5dc892adad0ea683cf9ca91b

commit 47bc347fed9dd1aa5dc892adad0ea683cf9ca91b
Author: Patrik Höglund <phoglund@chromium.org>
Date: Wed Nov 16 14:06:19 2016

Greatly simplify webrtc peerconnection test.

I don't see any reason to exercise 720p if it is available,
since video_WebRtcCamera is already doing that. I'm able to
greatly simplify the test (which will be nice if I add h264
support later).

Also fixes broken error handling and removes unnecessary
resolution management (video tags take the appropriate size
automatically, gUM selects a suitable constraint automatically
among the available ones, etc).

Also moves resolution check code to camera test, where it
belongs.

BUG= chromium:625241 , chromium:628630 
TEST=locally on link laptop

Change-Id: I14ba78fcb137ee688be4e82ac7ca1ceedb1634cc
Reviewed-on: https://chromium-review.googlesource.com/411842
Commit-Ready: Patrik Höglund <phoglund@chromium.org>
Commit-Ready: Rohit Makasana <rohitbm@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
Reviewed-by: Rohit Makasana <rohitbm@chromium.org>

[modify] https://crrev.com/47bc347fed9dd1aa5dc892adad0ea683cf9ca91b/client/site_tests/video_WebRtcPeerConnectionWithCamera/control
[modify] https://crrev.com/47bc347fed9dd1aa5dc892adad0ea683cf9ca91b/client/site_tests/video_WebRtcPeerConnectionWithCamera/loopback.html
[modify] https://crrev.com/47bc347fed9dd1aa5dc892adad0ea683cf9ca91b/client/site_tests/video_WebRtcPeerConnectionWithCamera/video_WebRtcPeerConnectionWithCamera.py
[modify] https://crrev.com/47bc347fed9dd1aa5dc892adad0ea683cf9ca91b/client/site_tests/video_WebRtcCamera/getusermedia.html

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/47bc347fed9dd1aa5dc892adad0ea683cf9ca91b

commit 47bc347fed9dd1aa5dc892adad0ea683cf9ca91b
Author: Patrik Höglund <phoglund@chromium.org>
Date: Wed Nov 16 14:06:19 2016

Greatly simplify webrtc peerconnection test.

I don't see any reason to exercise 720p if it is available,
since video_WebRtcCamera is already doing that. I'm able to
greatly simplify the test (which will be nice if I add h264
support later).

Also fixes broken error handling and removes unnecessary
resolution management (video tags take the appropriate size
automatically, gUM selects a suitable constraint automatically
among the available ones, etc).

Also moves resolution check code to camera test, where it
belongs.

BUG= chromium:625241 , chromium:628630 
TEST=locally on link laptop

Change-Id: I14ba78fcb137ee688be4e82ac7ca1ceedb1634cc
Reviewed-on: https://chromium-review.googlesource.com/411842
Commit-Ready: Patrik Höglund <phoglund@chromium.org>
Commit-Ready: Rohit Makasana <rohitbm@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
Reviewed-by: Rohit Makasana <rohitbm@chromium.org>

[modify] https://crrev.com/47bc347fed9dd1aa5dc892adad0ea683cf9ca91b/client/site_tests/video_WebRtcPeerConnectionWithCamera/control
[modify] https://crrev.com/47bc347fed9dd1aa5dc892adad0ea683cf9ca91b/client/site_tests/video_WebRtcPeerConnectionWithCamera/loopback.html
[modify] https://crrev.com/47bc347fed9dd1aa5dc892adad0ea683cf9ca91b/client/site_tests/video_WebRtcPeerConnectionWithCamera/video_WebRtcPeerConnectionWithCamera.py
[modify] https://crrev.com/47bc347fed9dd1aa5dc892adad0ea683cf9ca91b/client/site_tests/video_WebRtcCamera/getusermedia.html

Sent out https://chromium-review.googlesource.com/c/414866/. Wu-cheng, please take a look.

Ping Wu-cheng

Re 21: Sorry for the delay. LGTM with two small comments. 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/85fc0cb73af39c43a438382dbd8b7e23887170c1

commit 85fc0cb73af39c43a438382dbd8b7e23887170c1
Author: Patrik Höglund <phoglund@chromium.org>
Date: Thu Nov 24 12:45:17 2016

Add H264 WebRTC test.

We now run for both H264 and VP8 and verify the desired codec
actually gets used.

The test will split into a .h264 and .vp8 variant, and both tests
will report separately on the perf dashboard.

This patch will split the perf targets from e.g. black_frames to
black_frames_VP8 and black_frames_H264. We could migrate black_frames
to black_frames_VP8 in the perf dashboard, but I think we don't
have a lot of data anyway so maybe that's overkill. We don't really
need the codec on the labels, but I think we should have them since
they make things clearer.

BUG= chromium:625241 
TEST=locally on link laptop

Change-Id: Ie316c9e2b1f29a80190c82e155c284efa8d90ae1
Reviewed-on: https://chromium-review.googlesource.com/414866
Commit-Ready: Patrik Höglund <phoglund@chromium.org>
Tested-by: Wu-cheng Li <wuchengli@chromium.org>
Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>

[rename] https://crrev.com/85fc0cb73af39c43a438382dbd8b7e23887170c1/client/site_tests/video_WebRtcPeerConnectionWithCamera/control.vp8
[add] https://crrev.com/85fc0cb73af39c43a438382dbd8b7e23887170c1/client/site_tests/video_WebRtcPeerConnectionWithCamera/munge_sdp.js
[modify] https://crrev.com/85fc0cb73af39c43a438382dbd8b7e23887170c1/tko/perf_upload/perf_dashboard_config.json
[modify] https://crrev.com/85fc0cb73af39c43a438382dbd8b7e23887170c1/client/site_tests/video_WebRtcPeerConnectionWithCamera/loopback.html
[add] https://crrev.com/85fc0cb73af39c43a438382dbd8b7e23887170c1/client/site_tests/video_WebRtcPeerConnectionWithCamera/control.h264
[modify] https://crrev.com/85fc0cb73af39c43a438382dbd8b7e23887170c1/client/site_tests/video_WebRtcPeerConnectionWithCamera/video_WebRtcPeerConnectionWithCamera.py

Rohit, can you ask whomever is current to push the CrOS dashboard config.

dshi@ is the lab deputy for this week. 

What "CrOS dashboard config" are we talking about? GoldenEye, or wmatrix?

That's for perf dashboard json change.

tko/perf_upload/perf_dashboard_config.json

There we go, it's running now: https://chromeperf.appspot.com/report?sid=ad00c925cb1457e2543dc1edefb567dd805fd646b6fc4ad58b196a2605ed463a. Thanks dshi@.

Wu-cheng, feel free to reuse my approach like we agreed in #14-16. The magic is that we call munge_sdp.js on the offer, to put H264 first in the codec list. We then verify it was actually chosen by looking at the answer (though you can skip that part in your test if you want).

https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/testDetails?testName=video_WebRtcPeerConnectionWithCamera.h264&suite&daysBack=7&version&milestone