New issue
Advanced search Search tips

Issue 670306 link

Starred by 3 users

Issue metadata

Status: Verified
Owner:
Closed: Dec 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 627816



Sign in to add a comment

Performance browser tests for promise-based RTCPeerConnection.getStats

Project Member Reported by hbos@chromium.org, Dec 1 2016

Issue description

Add performance browser tests, similar to WebRtcPerfBrowserTest, that gets stats from a 60 second audio and video call such that they are picked up by the Chrome Performance Dashboard[1]. Example from other browser test: [2].

Anything worth tracking that is available through the new getStats API should be tracked. (We do not have the same set or amount of stats as in the old getStats API.) Decide what to track on this bug.

[1] https://chromeperf.appspot.com/
[2] https://chromeperf.appspot.com/report?sid=4a337ff35047d3a12a0b6d05fa1bf95e001c824cd1821be95dc7799134ae9b67
 

Comment 1 by hbos@chromium.org, Dec 1 2016

Include bytes sent and received for audio and video for different codecs.

Comment 2 by hbos@chromium.org, Dec 1 2016

Cc: hbos@chromium.org
 Issue 668965  has been merged into this issue.

Comment 3 by hbos@chromium.org, Dec 1 2016

Include RTCDataChannelStats.bytes_sent/received for tests sending and receiving data using data channels (with or without sending audio and video at the same time?)
Project Member

Comment 4 by bugdroid1@chromium.org, Dec 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c6ab14b0aa877c3e8e885934608d95849ff9a1a8

commit c6ab14b0aa877c3e8e885934608d95849ff9a1a8
Author: hbos <hbos@chromium.org>
Date: Fri Dec 02 09:25:32 2016

Preparation CL for WebRTC performance test using promise-based getStats

In order to have a perf test looking at stats of interest we need to be
able to get the results of a JavaScript getStats call. These can be
returned as JSON-stringified JavaScript dictionaries.

In this CL:
- [g|G]etStatsReportDictionary added to webrtc_browsertest_base.cc and
  peerconnection_getstats.js, which produce/read a JSON-stringified
  version of the stats.
- RTCStatsReportDictionary and RTCStatsDictionary, helper classes for
  reading stats from the base::Dictionary representation of the stats.
- Unittests for the helper classes.

In a follow-up CL these methods/classes will be used to obtain the
stats in the to-be-added performance test. This will yield pretty
graphs.

BUG= 670306 ,  627816 

Review-Url: https://codereview.chromium.org/2534633002
Cr-Commit-Position: refs/heads/master@{#435894}

[add] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/browser/media/webrtc/test_stats_dictionary.cc
[add] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/browser/media/webrtc/test_stats_dictionary.h
[add] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/browser/media/webrtc/test_stats_dictionary_unittest.cc
[modify] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/browser/media/webrtc/webrtc_browsertest_base.cc
[modify] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/browser/media/webrtc/webrtc_browsertest_base.h
[modify] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/test/BUILD.gn
[modify] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/test/data/webrtc/peerconnection_getstats.js

Project Member

Comment 5 by bugdroid1@chromium.org, Dec 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f50152dfc457a09afb6dca4a6ea9942af9a3b07a

commit f50152dfc457a09afb6dca4a6ea9942af9a3b07a
Author: hbos <hbos@chromium.org>
Date: Fri Dec 02 09:39:09 2016

Revert of Preparation CL for WebRTC performance test using promise-based getStats (patchset #4 id:160001 of https://codereview.chromium.org/2534633002/ )

Reason for revert:
Compile error on http://build.chromium.org/p/chromium.webrtc/builders/Linux%20Builder/builds/88436 due to uninitialized variables (warning treated as an error).

False alarm warning but I'll have to init and reland

Original issue's description:
> Preparation CL for WebRTC performance test using promise-based getStats
>
> In order to have a perf test looking at stats of interest we need to be
> able to get the results of a JavaScript getStats call. These can be
> returned as JSON-stringified JavaScript dictionaries.
>
> In this CL:
> - [g|G]etStatsReportDictionary added to webrtc_browsertest_base.cc and
>   peerconnection_getstats.js, which produce/read a JSON-stringified
>   version of the stats.
> - RTCStatsReportDictionary and RTCStatsDictionary, helper classes for
>   reading stats from the base::Dictionary representation of the stats.
> - Unittests for the helper classes.
>
> In a follow-up CL these methods/classes will be used to obtain the
> stats in the to-be-added performance test. This will yield pretty
> graphs.
>
> BUG= 670306 ,  627816 
>
> Committed: https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8
> Cr-Commit-Position: refs/heads/master@{#435894}

TBR=phoglund@chromium.org,hta@chromium.org,tommi@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 670306 ,  627816 

Review-Url: https://codereview.chromium.org/2544033003
Cr-Commit-Position: refs/heads/master@{#435897}

[delete] https://crrev.com/a12310b23b600d971816c01bb5cd7f6034db50fa/chrome/browser/media/webrtc/test_stats_dictionary.cc
[delete] https://crrev.com/a12310b23b600d971816c01bb5cd7f6034db50fa/chrome/browser/media/webrtc/test_stats_dictionary.h
[delete] https://crrev.com/a12310b23b600d971816c01bb5cd7f6034db50fa/chrome/browser/media/webrtc/test_stats_dictionary_unittest.cc
[modify] https://crrev.com/f50152dfc457a09afb6dca4a6ea9942af9a3b07a/chrome/browser/media/webrtc/webrtc_browsertest_base.cc
[modify] https://crrev.com/f50152dfc457a09afb6dca4a6ea9942af9a3b07a/chrome/browser/media/webrtc/webrtc_browsertest_base.h
[modify] https://crrev.com/f50152dfc457a09afb6dca4a6ea9942af9a3b07a/chrome/test/BUILD.gn
[modify] https://crrev.com/f50152dfc457a09afb6dca4a6ea9942af9a3b07a/chrome/test/data/webrtc/peerconnection_getstats.js

Project Member

Comment 6 by bugdroid1@chromium.org, Dec 5 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4f0df039e3c3e854d5a527744e7855b09b4add67

commit 4f0df039e3c3e854d5a527744e7855b09b4add67
Author: hbos <hbos@chromium.org>
Date: Mon Dec 05 10:11:32 2016

Preparation CL for WebRTC performance test using promise-based getStats
Re-land of https://codereview.chromium.org/2534633002/

In order to have a perf test looking at stats of interest we need to be
able to get the results of a JavaScript getStats call. These can be
returned as JSON-stringified JavaScript dictionaries.

In this CL:
- [g|G]etStatsReportDictionary added to webrtc_browsertest_base.cc and
  peerconnection_getstats.js, which produce/read a JSON-stringified
  version of the stats.
- RTCStatsReportDictionary and RTCStatsDictionary, helper classes for
  reading stats from the base::Dictionary representation of the stats.
- Unittests for the helper classes.

In a follow-up CL these methods/classes will be used to obtain the
stats in the to-be-added performance test. This will yield pretty
graphs.

BUG= 670306 ,  627816 

Review-Url: https://codereview.chromium.org/2543173002
Cr-Commit-Position: refs/heads/master@{#436252}

[add] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/browser/media/webrtc/test_stats_dictionary.cc
[add] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/browser/media/webrtc/test_stats_dictionary.h
[add] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/browser/media/webrtc/test_stats_dictionary_unittest.cc
[modify] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/browser/media/webrtc/webrtc_browsertest_base.cc
[modify] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/browser/media/webrtc/webrtc_browsertest_base.h
[modify] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/test/BUILD.gn
[modify] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/test/data/webrtc/peerconnection_getstats.js

Project Member

Comment 7 by bugdroid1@chromium.org, Dec 5 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/32bafc424946864c744ecfc0cbd83f6d5c18852b

commit 32bafc424946864c744ecfc0cbd83f6d5c18852b
Author: hbos <hbos@chromium.org>
Date: Mon Dec 05 11:19:58 2016

WebRtcStatsPerfBrowserTest added, a perf test using the new getStats.

Using the promise-based getStats to collect stats, writes stats after
60+10* seconds of audio and video calling. These will be picked up by
the Chrome Performance Dashboard[1]. E.g. from other browser test: [2].
*(After 60s of ramp-up, performs measurments during a 10s window, to
avoid unstable stats due to variations in ramp-up time.)

This is similar to the WebRtcPerfBrowserTest - renamed
WebRtcInternalsBrowserTest to avoid confusion - which instead uses
chrome://webrtc-internals relying on the old callback-based version of
getStats. Available stats are different.

This performance test only picks up:
- browser_tests / audio_[codec] / send_rate
- browser_tests / audio_[codec] / receive_rate
- browser_tests / video_[codec] / send_rate
- browser_tests / video_[codec] / receive_rate
Based on "RTC[In/Out]boundRTPStreamStats.bytes_[sent/received]" values
[3][4].
For audio codecs: opus, ISAC, G722, PCMU, PCMA.
And video codecs: VP8, VP9, H264.

More stats can be picked up in follow-up CLs, but the new getStats API
has a limited number of stats available.

[1] https://chromeperf.appspot.com/
[2] https://chromeperf.appspot.com/report?sid=4a337ff35047d3a12a0b6d05fa1bf95e001c824cd1821be95dc7799134ae9b67
[3] https://w3c.github.io/webrtc-stats/#dom-rtcoutboundrtpstreamstats-bytessent
[4] https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-bytesreceived

BUG= 670306 ,  627816 

Review-Url: https://codereview.chromium.org/2545553003
Cr-Commit-Position: refs/heads/master@{#436260}

[modify] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/browser/media/webrtc/webrtc_browsertest_base.cc
[modify] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/browser/media/webrtc/webrtc_browsertest_base.h
[add] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/browser/media/webrtc/webrtc_internals_perf_browsertest.cc
[modify] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/browser/media/webrtc/webrtc_perf_browsertest.cc
[add] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/browser/media/webrtc/webrtc_stats_perf_browsertest.cc
[modify] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/test/BUILD.gn
[modify] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/test/data/webrtc/peerconnection.js

Comment 8 by hbos@chromium.org, Dec 12 2016

Status: Verified (was: Started)
Even though only send and receive rates are reported by the above CL, I'll close this issue since there aren't that many stats useful for plotting a graph implemented. Tracking more stats can be filed for separately or part of the bugs for those stats.

Sign in to add a comment