cc tdresser for some questions.

How are these events expected to be correlated? Should INPUT_EVENT_GPU_SWAP_BUFFER_COMPONENT always follow INPUT_EVENT_LATENCY_ORIGINAL_COMPONENT? Should it *only* follow INPUT_EVENT_GPU_SWAP_BUFFER_COMPONENT?

In VR we're always swapping buffers every frame regardless of whether we've received a new content frame. Do the benchmarking scripts do something like ensuring new frames are generated only in response to scrolling, or are the scripts doing something more clever?

If we have the guarantee that all new content frames are in response to scroll events, then we could watch for new content frames arriving and only fire the trace events on the swapbuffers call after a new content frame arrives. This would give us a 1:1 mapping between original components and swap buffer components. Is this what we want?

Ordering should be consistent, and is described in this diagram:
https://drive.google.com/file/d/0B6aINcG6kDLXcmR1NTBnZFVnM2M/view

+nednguyen to keep me honest, but I think the current smoothness tests to annotate the region during which the page is scrolling, and the metrics are only computed over that range.

The way LatencyInfo works, we'll only have LatencyInfo when there has been input, so you won't end up tracing anything for frames that didn't have input. Given this, I'm not quite following your concern about having swap buffers calls without original components.

#2: correct, the current smoothness benchmarks annotate the region of animation in each test.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/33648c554cafddb23cf3140a1b915ebd3f50b379

commit 33648c554cafddb23cf3140a1b915ebd3f50b379
Author: mthiesse <mthiesse@chromium.org>
Date: Fri Jun 16 19:39:37 2017

VR: Use more accurate timestamps for input events.

This change pulls timestamps from the controller data, asking it when the
values we poll were last updated.

BUG=732830

Review-Url: https://codereview.chromium.org/2941293002
Cr-Commit-Position: refs/heads/master@{#480125}

[modify] https://crrev.com/33648c554cafddb23cf3140a1b915ebd3f50b379/chrome/browser/android/vr_shell/vr_controller.cc
[modify] https://crrev.com/33648c554cafddb23cf3140a1b915ebd3f50b379/chrome/browser/android/vr_shell/vr_controller.h
[modify] https://crrev.com/33648c554cafddb23cf3140a1b915ebd3f50b379/chrome/browser/android/vr_shell/vr_shell_gl.cc
[modify] https://crrev.com/33648c554cafddb23cf3140a1b915ebd3f50b379/chrome/browser/android/vr_shell/vr_shell_gl.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3e964c0d15f8e1cb3c9b63d1d47f6a98582dbc51

commit 3e964c0d15f8e1cb3c9b63d1d47f6a98582dbc51
Author: Michael Thiessen <mthiesse@chromium.org>
Date: Mon Jul 17 20:41:21 2017

VR: Revert controller timestamp adjustment

Recently we replaced the controller event timestamps with timestamps
generated by the VR Controller. However, these timestamps aren't in the
same timebase, and so cannot be used.

Bug: 732830
Change-Id: I33cb642e0008eb1676a65596c83e3982fe5a59f6
Reviewed-on: https://chromium-review.googlesource.com/568687
Reviewed-by: Amirhossein Simjour <asimjour@chromium.org>
Commit-Queue: Michael Thiessen <mthiesse@chromium.org>
Cr-Commit-Position: refs/heads/master@{#487231}
[modify] https://crrev.com/3e964c0d15f8e1cb3c9b63d1d47f6a98582dbc51/chrome/browser/android/vr_shell/vr_controller.cc

I handed a prototype CL over to Tibor, but haven't worked on this for a while and don't plan to get to it any time soon.

This is quite difficult and the benefits are pretty minor, so we should re-prioritize.

Closing this since nobody is actively working on it. We can reopen once we pick it up.

After some internal discussions resuscitating this bug and giving it a lower prio since we will likely not work on it in 2018Q1 but plan on doing it at some point.

Per the feedback on issue 817173 there's some concerns about scroll perf. Tibor/Lei/Brian perhaps we should start measuring this sooner?

I'm available to help out with this - the largest amount of work would be adding the correct tracing events to our scroll pipeline so that we can re-use the existing scroll metrics. After that, adding a way to do scrolling from within a perf test and actually adding it to tests should be pretty simple.

I may be helpful to break this task into separate metrics for scroll throughput and latency. The latter is a bit trickier since we need to measure the extended pipeline, but I'm hoping the former can be more straightforward (though we may need to do some work to determine if we have a new frame of web content when we go to draw).

I really don't have enough context to add meaningful insight, but I like Ian's suggestion (i.e. if it's easy to get some visibility into this, that would be worthwhile now). Promoting to MVP-Next but I'm supportive of doing it for MVP if it's <1-2 days work

Currently, I have https://chromium-review.googlesource.com/c/chromium/src/+/963042, which reuses the existing smoothness benchmark in VR. So, while it won't be useful for latency, it should be good for throughput/jank/smoothness. However, there are a couple of issues:

1. It's currently not landable until two prerequisite CLs land. One is a refactor of the the smoothness page set that we're based off of so that we ran re-use the URLs, etc. The other is an infra CL that will make the presubmit stop failing.

2. The smoothness benchmark is a legacy benchmark as opposed to the newer TBMv2 (Timeline Based Measurements), the latter of which being what we use for all our other metrics. This means that we can't add our own VR metrics to the existing smoothness benchmark (e.g. web content FPS). Apparently there's a (somewhat low priority) effort to move the smoothness benchmark to TBMv2, so I'll be talking with the people working on that in the near future to discuss ETAs and such.

That's great and definitely ok as a good starting point. (1) just sounds like some process that needs to happen and that's fine. (2) is a larger concern for future but this at least gives a baseline to start monitoring jank/smoothness regressions

What would we be able to measure with the old metric?

Based on what's reported currently on the perf dashboard for smoothness.key_mobile_sites_smooth (what the VR one is based on), we would get:

avg_surface_fps
first_gesture_scroll_update_latency
frame_lengths
frame_time_discrepancy
frame_times
input_event_latency
input_event_latency_discrepancy
jank_count
main_thread_scroll_latency
main_thread_scroll_latency_discrepancy
max_frame_delay
mean_frame_time
mean_input_event_latency
mean_main_thread_scroll_latency
mean_pixels_approximated
mean_pixels_checkerboarded
percentage_smooth
queuing_durations

My guess would be that the only ones we would care about/would get accurate data from would be jank_count, percentage_smooth, and queuing_durations?

I'm not sure if those are valid in VR since we don't necessarily show every web contents frame in VR.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ae3ef6ba8bede188af3ecb033e4650bf3d380447

commit ae3ef6ba8bede188af3ecb033e4650bf3d380447
Author: bsheedy <bsheedy@chromium.org>
Date: Thu Mar 22 18:07:19 2018

Move key_mobile_sites_smooth URL lists to constants

Moves fastpath_urls and urls_list in
tools/perf/page_sets/key_mobile_sites_smooth.py to constants in the same
file so that the same list of URLs can be used in multiple files.

Bug: 732830
Change-Id: I8eba6c147f61f27f50999a540122ac50aba931c6
Reviewed-on: https://chromium-review.googlesource.com/965164
Reviewed-by: Sunny Sachanandani <sunnyps@chromium.org>
Reviewed-by: Charlie Andrews <charliea@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#545153}
[modify] https://crrev.com/ae3ef6ba8bede188af3ecb033e4650bf3d380447/tools/perf/page_sets/key_mobile_sites_smooth.py

See also issue 724221.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0a50f56357cb7161b6f5c6e11127eed2850b605d

commit 0a50f56357cb7161b6f5c6e11127eed2850b605d
Author: bsheedy <bsheedy@chromium.org>
Date: Mon Apr 30 19:11:20 2018

Add VR version of smoothness benchmark

Adds a version of KeyMobileSitesSmoothPageSet page set and
SmoothnessKeyMobileSites benchmark for use in the VR browser.

This re-uses all the same URLs/pages, but also adds a Vimeo page since
video is a large use case for VR.

Bug: 732830
Change-Id: I9146c1d2056527b8460b56a8b28f5da47ead1b23
Reviewed-on: https://chromium-review.googlesource.com/963042
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Tibor Goldschwendt <tiborg@chromium.org>
Cr-Commit-Position: refs/heads/master@{#554835}
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/README
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_000.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_001.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_002.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_003.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_004.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_005.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_006.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_007.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_008.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_009.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_010.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_011.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_012.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_013.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_014.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_015.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_016.wprgo.sha1
[add] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/data/key_mobile_sites/key_mobile_sites_smooth.json
[modify] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/vr_benchmarks.py
[modify] https://crrev.com/0a50f56357cb7161b6f5c6e11127eed2850b605d/tools/perf/contrib/vr_benchmarks/vr_browsing_mode_pages.py

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chrome/tools/build_limited/scripts/slave/+/df87d7152eaad7b7b2253cb2204f087530639f24

commit df87d7152eaad7b7b2253cb2204f087530639f24
Author: bsheedy <bsheedy@google.com>
Date: Mon Apr 30 19:53:09 2018

Issue 724221 has been merged into this issue.

I won't get to this anymore. Assigning to you, bsheedy@, to find a new owner.