1. Support resampling input events coordinates on main thread. 2. Support resampling scroll event delta on compositor thread.
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8 commit be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8 Author: Ella Ge <eirage@chromium.org> Date: Wed Apr 25 21:04:33 2018 Add resample event to main thread event queue This CL adds input event predictor in main thread event queue under a flag kResampleInputEvents. It is the basic structure for resample events, without only an empty predictor. The input predictor handled events before the event is dispatched. And apply current time to the event. design doc: https://docs.google.com/document/d/1DLfibi2NkV85p7AfEtNkvy24d283VRBSd3hz1Qh24Jw/edit?usp=sharing Bug: 836352 Change-Id: I7c17a33b2e0e5e9a51920e54a9b8ce795537f4ef Reviewed-on: https://chromium-review.googlesource.com/860429 Reviewed-by: Dave Tapuska <dtapuska@chromium.org> Reviewed-by: Alex Moshchuk <alexmos@chromium.org> Reviewed-by: Navid Zolghadr <nzolghadr@chromium.org> Commit-Queue: Ella Ge <eirage@chromium.org> Cr-Commit-Position: refs/heads/master@{#553744} [modify] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/content/public/common/content_features.cc [modify] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/content/public/common/content_features.h [modify] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/content/renderer/BUILD.gn [add] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/content/renderer/input/input_event_prediction.cc [add] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/content/renderer/input/input_event_prediction.h [add] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/content/renderer/input/input_event_prediction_unittest.cc [modify] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/content/renderer/input/main_thread_event_queue.cc [modify] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/content/renderer/input/main_thread_event_queue.h [modify] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/content/test/BUILD.gn [modify] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/ui/events/blink/BUILD.gn [add] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/ui/events/blink/prediction/empty_predictor.cc [add] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/ui/events/blink/prediction/empty_predictor.h [add] https://crrev.com/be3bfc9d9de4bd4f49b8cdb8056e6700e6d15af8/ui/events/blink/prediction/input_predictor.h
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a2089b42e91a143c74363885e1931fc447e74a42 commit a2089b42e91a143c74363885e1931fc447e74a42 Author: Ella Ge <eirage@chromium.org> Date: Wed Jun 06 21:08:15 2018 Add lsq predictor to input prediction This CL adds LSQ predictor to input prediction. LSQ predictor uses a quadratic least square regression model: y = b0 + b1 * x + b2 * x ^ 2. The resampling is still behind flag: kResampleInputEvents. design doc: https://docs.google.com/document/d/1DLfibi2NkV85p7AfEtNkvy24d283VRBSd3hz1Qh24Jw/edit# Bug: 836352 Change-Id: Ie1959c6b95152a80076debfb762c49d88a758af3 Reviewed-on: https://chromium-review.googlesource.com/1003220 Commit-Queue: Ella Ge <eirage@chromium.org> Reviewed-by: Dave Tapuska <dtapuska@chromium.org> Reviewed-by: danakj <danakj@chromium.org> Reviewed-by: Navid Zolghadr <nzolghadr@chromium.org> Cr-Commit-Position: refs/heads/master@{#565035} [modify] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/content/renderer/input/input_event_prediction.cc [modify] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/content/renderer/input/input_event_prediction.h [modify] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/content/renderer/input/input_event_prediction_unittest.cc [modify] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/ui/events/BUILD.gn [modify] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/ui/events/blink/BUILD.gn [add] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/ui/events/blink/prediction/least_squares_predictor.cc [add] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/ui/events/blink/prediction/least_squares_predictor.h [add] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/ui/events/blink/prediction/least_squares_predictor_unittest.cc [modify] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/ui/gfx/geometry/matrix3_f.cc [modify] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/ui/gfx/geometry/matrix3_f.h [modify] https://crrev.com/a2089b42e91a143c74363885e1931fc447e74a42/ui/gfx/geometry/matrix3_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5abdd4b46233b105db38273d42171fe6fe699851 commit 5abdd4b46233b105db38273d42171fe6fe699851 Author: Ella Ge <eirage@chromium.org> Date: Mon Jun 18 14:53:55 2018 Resample scroll event in compositor thread event queue This CL adds scroll predictor in compositor thread under flag kResamplingScrollEvents. design doc: https://docs.google.com/document/d/1g6eGYdLZKznF3sCiktc5BbAPeiudWk8xTzx3qdrxowI/edit# Bug: 836352 Change-Id: Iecfad7ad75dd4b871426e00a2f2d542996544b07 Reviewed-on: https://chromium-review.googlesource.com/919662 Commit-Queue: Ella Ge <eirage@chromium.org> Reviewed-by: Sahel Sharifymoghaddam <sahel@chromium.org> Reviewed-by: Dave Tapuska <dtapuska@chromium.org> Reviewed-by: Navid Zolghadr <nzolghadr@chromium.org> Cr-Commit-Position: refs/heads/master@{#568003} [modify] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/BUILD.gn [modify] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/blink/BUILD.gn [modify] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/blink/blink_features.cc [modify] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/blink/blink_features.h [modify] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/blink/event_with_callback.h [modify] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/blink/input_handler_proxy.cc [modify] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/blink/input_handler_proxy.h [modify] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/blink/input_handler_proxy_unittest.cc [add] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/blink/scroll_predictor.cc [add] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/blink/scroll_predictor.h [add] https://crrev.com/5abdd4b46233b105db38273d42171fe6fe699851/ui/events/blink/scroll_predictor_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f7a90b897500a7269758ba752fd9537c73cc8430 commit f7a90b897500a7269758ba752fd9537c73cc8430 Author: Ella Ge <eirage@chromium.org> Date: Mon Jun 18 16:34:32 2018 Input Prediction use PointF instead of pos_x and pos_y This Cl makes InputData in input prediction to use PointF instead of pos_x and pos_y. And also adds two extra unit tests to lsq predictor Bug: 836352 Change-Id: I3e551add7536e55be82f11ed2338b0bb9dfe8e07 Reviewed-on: https://chromium-review.googlesource.com/1093392 Commit-Queue: Ella Ge <eirage@chromium.org> Reviewed-by: Dave Tapuska <dtapuska@chromium.org> Reviewed-by: Navid Zolghadr <nzolghadr@chromium.org> Cr-Commit-Position: refs/heads/master@{#568031} [modify] https://crrev.com/f7a90b897500a7269758ba752fd9537c73cc8430/content/renderer/input/input_event_prediction.cc [modify] https://crrev.com/f7a90b897500a7269758ba752fd9537c73cc8430/content/renderer/input/input_event_prediction_unittest.cc [modify] https://crrev.com/f7a90b897500a7269758ba752fd9537c73cc8430/ui/events/blink/input_handler_proxy_unittest.cc [modify] https://crrev.com/f7a90b897500a7269758ba752fd9537c73cc8430/ui/events/blink/prediction/empty_predictor.cc [modify] https://crrev.com/f7a90b897500a7269758ba752fd9537c73cc8430/ui/events/blink/prediction/input_predictor.h [modify] https://crrev.com/f7a90b897500a7269758ba752fd9537c73cc8430/ui/events/blink/prediction/least_squares_predictor.cc [modify] https://crrev.com/f7a90b897500a7269758ba752fd9537c73cc8430/ui/events/blink/prediction/least_squares_predictor_unittest.cc [modify] https://crrev.com/f7a90b897500a7269758ba752fd9537c73cc8430/ui/events/blink/scroll_predictor.cc [modify] https://crrev.com/f7a90b897500a7269758ba752fd9537c73cc8430/ui/events/blink/scroll_predictor_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/db1af3539a8ba9cbe05256b37ef72d5098339775 commit db1af3539a8ba9cbe05256b37ef72d5098339775 Author: Ella Ge <eirage@chromium.org> Date: Wed Jul 04 18:26:04 2018 Add kalman_filter predictor input predicton This CL adds kalman filter predictor to input predicions The state transition model used in this model is: new_pos = pre_pos + v * dt + 1/2 * a * dt^2; new_v = v + a * dt; new_a = a; The resampling is still behind flag: kResampleInputEvents and kResampleScrollEvents, can be enabled by using field trial params 'predictor:kalman' with the flag. design doc: https://docs.google.com/document/d/1g6eGYdLZKznF3sCiktc5BbAPeiudWk8xTzx3qdrxowI/edit# Bug: 836352 Change-Id: Ib3a050203cbc463712e05faac0a25d5c98ca9c96 Reviewed-on: https://chromium-review.googlesource.com/1100000 Reviewed-by: danakj <danakj@chromium.org> Reviewed-by: Dave Tapuska <dtapuska@chromium.org> Reviewed-by: Navid Zolghadr <nzolghadr@chromium.org> Commit-Queue: Ella Ge <eirage@chromium.org> Cr-Commit-Position: refs/heads/master@{#572625} [modify] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/content/renderer/input/input_event_prediction.cc [modify] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/content/renderer/input/input_event_prediction.h [modify] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/BUILD.gn [modify] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/blink/BUILD.gn [add] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/blink/prediction/input_predictor_unittest.cc [add] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/blink/prediction/kalman_filter.cc [add] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/blink/prediction/kalman_filter.h [add] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/blink/prediction/kalman_predictor.cc [add] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/blink/prediction/kalman_predictor.h [add] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/blink/prediction/kalman_predictor_unittest.cc [modify] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/blink/prediction/least_squares_predictor.h [modify] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/blink/prediction/least_squares_predictor_unittest.cc [modify] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/events/blink/scroll_predictor.cc [modify] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/gfx/geometry/matrix3_f.cc [modify] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/gfx/geometry/matrix3_f.h [modify] https://crrev.com/db1af3539a8ba9cbe05256b37ef72d5098339775/ui/gfx/geometry/matrix3_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1c79ea98f51481434f87cf103edc16faffa14da5 commit 1c79ea98f51481434f87cf103edc16faffa14da5 Author: Ella Ge <eirage@chromium.org> Date: Tue Aug 14 00:25:03 2018 scroll predictor should not change scrolling direction This Cl made a few change to correct scroll resampling behavior: 1. Adds a rule to prevent scroll predictor change direction of scroll delta. Otherwise it might cause the scroll to jump back, which is a very bad behavior. When new_delta and original delta is in different direction, makes new_delta to 0. 2.This CL also moved scroll_predictor_.HandleEvents() from DispatchSingleInputEvent to DispatchQueuedInputEvents. This makes events not in the compositor event queue do not affect by scroll_predictor. 3. The scroll predictor record phase of the event. When predictor gets a GSU with inertial_phase is kMomentumPhase, it stops resampling. This is because when fling, GSU is sent per frame, we don't need to resampe the event. 4. Adds some limits to resampling time. This is because we sometimes get incorrect timestamp from OS (android, see the example logout: https://docs.google.com/spreadsheets/d/1yY1CNaL6ovQpg7LDm_Uyq5dIbVMQy684od0J6OX0Hn0/). Besides that, since we value the accuracy in resampling, we don't want to predict to far in the future. Bug: 836352 Change-Id: I7d84c80e76d48596cf2012611a25133276474d71 Reviewed-on: https://chromium-review.googlesource.com/1136899 Commit-Queue: Ella Ge <eirage@chromium.org> Reviewed-by: Dave Tapuska <dtapuska@chromium.org> Reviewed-by: Navid Zolghadr <nzolghadr@chromium.org> Reviewed-by: Sahel Sharify <sahel@chromium.org> Cr-Commit-Position: refs/heads/master@{#582767} [modify] https://crrev.com/1c79ea98f51481434f87cf103edc16faffa14da5/ui/events/blink/input_handler_proxy.cc [modify] https://crrev.com/1c79ea98f51481434f87cf103edc16faffa14da5/ui/events/blink/prediction/input_predictor.h [modify] https://crrev.com/1c79ea98f51481434f87cf103edc16faffa14da5/ui/events/blink/prediction/kalman_predictor.cc [modify] https://crrev.com/1c79ea98f51481434f87cf103edc16faffa14da5/ui/events/blink/prediction/least_squares_predictor.cc [modify] https://crrev.com/1c79ea98f51481434f87cf103edc16faffa14da5/ui/events/blink/scroll_predictor.cc [modify] https://crrev.com/1c79ea98f51481434f87cf103edc16faffa14da5/ui/events/blink/scroll_predictor.h [modify] https://crrev.com/1c79ea98f51481434f87cf103edc16faffa14da5/ui/events/blink/scroll_predictor_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/bee81c970aee9e9b9c8d0d863b613f7af8a1f438 commit bee81c970aee9e9b9c8d0d863b613f7af8a1f438 Author: Ella Ge <eirage@chromium.org> Date: Tue Aug 14 15:51:09 2018 use chrome flag to enable resamplings Adds resampling input events and scroll events to chrome://flags Removes an old flag enable-scroll-prediction that seems not in used not. Bug: 836352 Change-Id: I37e78acd6bde0f69f3555df998ec91fedc284360 Reviewed-on: https://chromium-review.googlesource.com/1155407 Commit-Queue: Ella Ge <eirage@chromium.org> Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org> Reviewed-by: Navid Zolghadr <nzolghadr@chromium.org> Cr-Commit-Position: refs/heads/master@{#582924} [modify] https://crrev.com/bee81c970aee9e9b9c8d0d863b613f7af8a1f438/chrome/browser/about_flags.cc [modify] https://crrev.com/bee81c970aee9e9b9c8d0d863b613f7af8a1f438/chrome/browser/flag_descriptions.cc [modify] https://crrev.com/bee81c970aee9e9b9c8d0d863b613f7af8a1f438/chrome/browser/flag_descriptions.h [modify] https://crrev.com/bee81c970aee9e9b9c8d0d863b613f7af8a1f438/tools/metrics/histograms/enums.xml [modify] https://crrev.com/bee81c970aee9e9b9c8d0d863b613f7af8a1f438/ui/events/event_switches.cc [modify] https://crrev.com/bee81c970aee9e9b9c8d0d863b613f7af8a1f438/ui/events/event_switches.h
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05 commit 2cf1bb0ffbf773b884b31df0b6ab0237acd45a05 Author: Ella Ge <eirage@chromium.org> Date: Wed Oct 17 01:06:10 2018 Add accuracy metrics input event prediction This CL adds histogram metrics to measure input prediction accuracy, There are 3 metrics for each of mouse/touch/scroll (9 total). They are measuring the distance between predicted position at event time to the event's position. We separate each of the result in 3 metrics base on the prediction time delta, because the accuracy is sensitive to how far we are predicting. This CL changes the InputEventPrediction and ScrollPredictor class to choose different predictor by the *PredictorTypeChoice flag, and do the prediction when the flag passing corresponding predictor type, so we will gather accuracy data for the predictor via finch. Whether apply the resampling to the event is still under the resampling feature flag. Bug: 836352 Change-Id: Ie66e4fe1c810120415f9ee976ffa4d8faa0003f1 Reviewed-on: https://chromium-review.googlesource.com/c/1237240 Commit-Queue: Ella Ge <eirage@chromium.org> Reviewed-by: Mark Pearson <mpearson@chromium.org> Reviewed-by: Avi Drissman <avi@chromium.org> Reviewed-by: Dave Tapuska <dtapuska@chromium.org> Reviewed-by: Navid Zolghadr <nzolghadr@chromium.org> Cr-Commit-Position: refs/heads/master@{#600219} [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/content/public/common/content_features.cc [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/content/public/common/content_features.h [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/content/renderer/input/input_event_prediction.cc [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/content/renderer/input/input_event_prediction.h [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/content/renderer/input/input_event_prediction_unittest.cc [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/tools/metrics/histograms/histograms.xml [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/blink_features.cc [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/blink_features.h [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/input_handler_proxy.cc [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/input_handler_proxy_unittest.cc [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/prediction/empty_predictor.cc [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/prediction/empty_predictor.h [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/prediction/input_predictor.h [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/prediction/kalman_predictor.cc [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/prediction/kalman_predictor.h [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/prediction/least_squares_predictor.cc [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/prediction/least_squares_predictor.h [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/scroll_predictor.cc [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/scroll_predictor.h [modify] https://crrev.com/2cf1bb0ffbf773b884b31df0b6ab0237acd45a05/ui/events/blink/scroll_predictor_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/891c835d60bd3c67f2fe151260900f7091ab81e3 commit 891c835d60bd3c67f2fe151260900f7091ab81e3 Author: Ella Ge <eirage@chromium.org> Date: Thu Nov 29 22:16:07 2018 Add scroll prediction over/under predict metric This CL adds two metrics for scroll event prediction to help analyse prediction behavior, Scroll.OverPredict and Scroll.UnderPredict. The new metrics only records the y direction difference of prediction result instead of euclidean distance, since y direction scroll is more popular and only record one direction is easier for determine over/under predict. Bug: 836352 Change-Id: I2f83fa5404b02c52f8856dfb64fa0151b7b686d6 Reviewed-on: https://chromium-review.googlesource.com/c/1354314 Reviewed-by: Dave Tapuska <dtapuska@chromium.org> Reviewed-by: Mark Pearson <mpearson@chromium.org> Reviewed-by: Navid Zolghadr <nzolghadr@chromium.org> Commit-Queue: Ella Ge <eirage@chromium.org> Cr-Commit-Position: refs/heads/master@{#612401} [modify] https://crrev.com/891c835d60bd3c67f2fe151260900f7091ab81e3/tools/metrics/histograms/histograms.xml [modify] https://crrev.com/891c835d60bd3c67f2fe151260900f7091ab81e3/ui/events/blink/scroll_predictor.cc
Comment 1 by eirage@chromium.org
, Apr 24 2018