Performance browser tests for promise-based RTCPeerConnection.getStats |
||
Issue descriptionAdd performance browser tests, similar to WebRtcPerfBrowserTest, that gets stats from a 60 second audio and video call such that they are picked up by the Chrome Performance Dashboard[1]. Example from other browser test: [2]. Anything worth tracking that is available through the new getStats API should be tracked. (We do not have the same set or amount of stats as in the old getStats API.) Decide what to track on this bug. [1] https://chromeperf.appspot.com/ [2] https://chromeperf.appspot.com/report?sid=4a337ff35047d3a12a0b6d05fa1bf95e001c824cd1821be95dc7799134ae9b67
,
Dec 1 2016
,
Dec 1 2016
Include RTCDataChannelStats.bytes_sent/received for tests sending and receiving data using data channels (with or without sending audio and video at the same time?)
,
Dec 2 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c6ab14b0aa877c3e8e885934608d95849ff9a1a8 commit c6ab14b0aa877c3e8e885934608d95849ff9a1a8 Author: hbos <hbos@chromium.org> Date: Fri Dec 02 09:25:32 2016 Preparation CL for WebRTC performance test using promise-based getStats In order to have a perf test looking at stats of interest we need to be able to get the results of a JavaScript getStats call. These can be returned as JSON-stringified JavaScript dictionaries. In this CL: - [g|G]etStatsReportDictionary added to webrtc_browsertest_base.cc and peerconnection_getstats.js, which produce/read a JSON-stringified version of the stats. - RTCStatsReportDictionary and RTCStatsDictionary, helper classes for reading stats from the base::Dictionary representation of the stats. - Unittests for the helper classes. In a follow-up CL these methods/classes will be used to obtain the stats in the to-be-added performance test. This will yield pretty graphs. BUG= 670306 , 627816 Review-Url: https://codereview.chromium.org/2534633002 Cr-Commit-Position: refs/heads/master@{#435894} [add] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/browser/media/webrtc/test_stats_dictionary.cc [add] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/browser/media/webrtc/test_stats_dictionary.h [add] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/browser/media/webrtc/test_stats_dictionary_unittest.cc [modify] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/browser/media/webrtc/webrtc_browsertest_base.cc [modify] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/browser/media/webrtc/webrtc_browsertest_base.h [modify] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/test/BUILD.gn [modify] https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8/chrome/test/data/webrtc/peerconnection_getstats.js
,
Dec 2 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f50152dfc457a09afb6dca4a6ea9942af9a3b07a commit f50152dfc457a09afb6dca4a6ea9942af9a3b07a Author: hbos <hbos@chromium.org> Date: Fri Dec 02 09:39:09 2016 Revert of Preparation CL for WebRTC performance test using promise-based getStats (patchset #4 id:160001 of https://codereview.chromium.org/2534633002/ ) Reason for revert: Compile error on http://build.chromium.org/p/chromium.webrtc/builders/Linux%20Builder/builds/88436 due to uninitialized variables (warning treated as an error). False alarm warning but I'll have to init and reland Original issue's description: > Preparation CL for WebRTC performance test using promise-based getStats > > In order to have a perf test looking at stats of interest we need to be > able to get the results of a JavaScript getStats call. These can be > returned as JSON-stringified JavaScript dictionaries. > > In this CL: > - [g|G]etStatsReportDictionary added to webrtc_browsertest_base.cc and > peerconnection_getstats.js, which produce/read a JSON-stringified > version of the stats. > - RTCStatsReportDictionary and RTCStatsDictionary, helper classes for > reading stats from the base::Dictionary representation of the stats. > - Unittests for the helper classes. > > In a follow-up CL these methods/classes will be used to obtain the > stats in the to-be-added performance test. This will yield pretty > graphs. > > BUG= 670306 , 627816 > > Committed: https://crrev.com/c6ab14b0aa877c3e8e885934608d95849ff9a1a8 > Cr-Commit-Position: refs/heads/master@{#435894} TBR=phoglund@chromium.org,hta@chromium.org,tommi@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 670306 , 627816 Review-Url: https://codereview.chromium.org/2544033003 Cr-Commit-Position: refs/heads/master@{#435897} [delete] https://crrev.com/a12310b23b600d971816c01bb5cd7f6034db50fa/chrome/browser/media/webrtc/test_stats_dictionary.cc [delete] https://crrev.com/a12310b23b600d971816c01bb5cd7f6034db50fa/chrome/browser/media/webrtc/test_stats_dictionary.h [delete] https://crrev.com/a12310b23b600d971816c01bb5cd7f6034db50fa/chrome/browser/media/webrtc/test_stats_dictionary_unittest.cc [modify] https://crrev.com/f50152dfc457a09afb6dca4a6ea9942af9a3b07a/chrome/browser/media/webrtc/webrtc_browsertest_base.cc [modify] https://crrev.com/f50152dfc457a09afb6dca4a6ea9942af9a3b07a/chrome/browser/media/webrtc/webrtc_browsertest_base.h [modify] https://crrev.com/f50152dfc457a09afb6dca4a6ea9942af9a3b07a/chrome/test/BUILD.gn [modify] https://crrev.com/f50152dfc457a09afb6dca4a6ea9942af9a3b07a/chrome/test/data/webrtc/peerconnection_getstats.js
,
Dec 5 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4f0df039e3c3e854d5a527744e7855b09b4add67 commit 4f0df039e3c3e854d5a527744e7855b09b4add67 Author: hbos <hbos@chromium.org> Date: Mon Dec 05 10:11:32 2016 Preparation CL for WebRTC performance test using promise-based getStats Re-land of https://codereview.chromium.org/2534633002/ In order to have a perf test looking at stats of interest we need to be able to get the results of a JavaScript getStats call. These can be returned as JSON-stringified JavaScript dictionaries. In this CL: - [g|G]etStatsReportDictionary added to webrtc_browsertest_base.cc and peerconnection_getstats.js, which produce/read a JSON-stringified version of the stats. - RTCStatsReportDictionary and RTCStatsDictionary, helper classes for reading stats from the base::Dictionary representation of the stats. - Unittests for the helper classes. In a follow-up CL these methods/classes will be used to obtain the stats in the to-be-added performance test. This will yield pretty graphs. BUG= 670306 , 627816 Review-Url: https://codereview.chromium.org/2543173002 Cr-Commit-Position: refs/heads/master@{#436252} [add] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/browser/media/webrtc/test_stats_dictionary.cc [add] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/browser/media/webrtc/test_stats_dictionary.h [add] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/browser/media/webrtc/test_stats_dictionary_unittest.cc [modify] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/browser/media/webrtc/webrtc_browsertest_base.cc [modify] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/browser/media/webrtc/webrtc_browsertest_base.h [modify] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/test/BUILD.gn [modify] https://crrev.com/4f0df039e3c3e854d5a527744e7855b09b4add67/chrome/test/data/webrtc/peerconnection_getstats.js
,
Dec 5 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/32bafc424946864c744ecfc0cbd83f6d5c18852b commit 32bafc424946864c744ecfc0cbd83f6d5c18852b Author: hbos <hbos@chromium.org> Date: Mon Dec 05 11:19:58 2016 WebRtcStatsPerfBrowserTest added, a perf test using the new getStats. Using the promise-based getStats to collect stats, writes stats after 60+10* seconds of audio and video calling. These will be picked up by the Chrome Performance Dashboard[1]. E.g. from other browser test: [2]. *(After 60s of ramp-up, performs measurments during a 10s window, to avoid unstable stats due to variations in ramp-up time.) This is similar to the WebRtcPerfBrowserTest - renamed WebRtcInternalsBrowserTest to avoid confusion - which instead uses chrome://webrtc-internals relying on the old callback-based version of getStats. Available stats are different. This performance test only picks up: - browser_tests / audio_[codec] / send_rate - browser_tests / audio_[codec] / receive_rate - browser_tests / video_[codec] / send_rate - browser_tests / video_[codec] / receive_rate Based on "RTC[In/Out]boundRTPStreamStats.bytes_[sent/received]" values [3][4]. For audio codecs: opus, ISAC, G722, PCMU, PCMA. And video codecs: VP8, VP9, H264. More stats can be picked up in follow-up CLs, but the new getStats API has a limited number of stats available. [1] https://chromeperf.appspot.com/ [2] https://chromeperf.appspot.com/report?sid=4a337ff35047d3a12a0b6d05fa1bf95e001c824cd1821be95dc7799134ae9b67 [3] https://w3c.github.io/webrtc-stats/#dom-rtcoutboundrtpstreamstats-bytessent [4] https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-bytesreceived BUG= 670306 , 627816 Review-Url: https://codereview.chromium.org/2545553003 Cr-Commit-Position: refs/heads/master@{#436260} [modify] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/browser/media/webrtc/webrtc_browsertest_base.cc [modify] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/browser/media/webrtc/webrtc_browsertest_base.h [add] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/browser/media/webrtc/webrtc_internals_perf_browsertest.cc [modify] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/browser/media/webrtc/webrtc_perf_browsertest.cc [add] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/browser/media/webrtc/webrtc_stats_perf_browsertest.cc [modify] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/test/BUILD.gn [modify] https://crrev.com/32bafc424946864c744ecfc0cbd83f6d5c18852b/chrome/test/data/webrtc/peerconnection.js
,
Dec 12 2016
Even though only send and receive rates are reported by the above CL, I'll close this issue since there aren't that many stats useful for plotting a graph implemented. Tracking more stats can be filed for separately or part of the bugs for those stats. |
||
►
Sign in to add a comment |
||
Comment 1 by hbos@chromium.org
, Dec 1 2016