New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 855252 link

Starred by 2 users
Status: Fixed
Owner:
benjhayden@chromium.org
Closed: Sep 5
Cc:
simonhatch@chromium.org
█ sbalana@google.com
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocking:
issue 875941



Sign in to add a comment

Cache timeseries data in browser

Project Member Reported by benjhayden@chromium.org, Jun 21 2018 
The chromeperf dashboard does not cache any data in the browser.
Many important users monitor the same timeseries over time.
Caching timeseries data in the browser will greatly speed up loading charts and enable offline use.
Timeseries data is generally keyed by timeseries identifier (composed of test suite name, measurement name, bot name, test case name, build type) and x-coordinate (generally commit position or timestamp), and contains summary statistics (avg, std, sample count, min, max, etc), annotations (alerts, sparse diagnostics), and full Histogram objects (description, bins, diagnostics, etc).
Lighthouse recommends using a service worker to cache complex JSON data using IndexedDB.
"idb" will be used to promisify the IndexedDB API.
Project Member

Comment 1 by bugdroid1@chromium.org, Jun 22 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/34451063bc2942c7f8f7a70cafb758c07462a637

commit 34451063bc2942c7f8f7a70cafb758c07462a637
Author: sam <sbalana@google.com>
Date: Fri Jun 22 17:53:05 2018

Add idb to third_party.

This library is useful for interfacing with IndexedDB using promises, rather
than the traditional way of using request objects and specifying event
handlers.

Bug:  chromium:855252 
Change-Id: I20f4760d2a07e755432950c140f5884019b20f9a
Reviewed-on: https://chromium-review.googlesource.com/1111005
Commit-Queue: Sam Balana <sbalana@google.com>
Reviewed-by: Ben Hayden <benjhayden@chromium.org>
Reviewed-by: Annie Sullivan <sullivan@chromium.org>

[add] https://crrev.com/34451063bc2942c7f8f7a70cafb758c07462a637/third_party/idb/README.md
[add] https://crrev.com/34451063bc2942c7f8f7a70cafb758c07462a637/third_party/idb/idb.js
[add] https://crrev.com/34451063bc2942c7f8f7a70cafb758c07462a637/third_party/idb/LICENSE
Project Member

Comment 2 by bugdroid1@chromium.org, Jun 23 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6496ff23de6421d900baab8badc7f279e8afb14d

commit 6496ff23de6421d900baab8badc7f279e8afb14d
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Sat Jun 23 01:09:46 2018

Roll src/third_party/catapult fa74d9840634..34451063bc29 (1 commits)

https://chromium.googlesource.com/catapult.git/+log/fa74d9840634..34451063bc29


git log fa74d9840634..34451063bc29 --date=short --no-merges --format='%ad %ae %s'
2018-06-22 sbalana@google.com Add idb to third_party.


Created with:
  gclient setdep -r src/third_party/catapult@34451063bc29

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:855252 
TBR=sullivan@chromium.org

Change-Id: I2f265280fdc9d9b62b7621d3b5be5276f9266ec2
Reviewed-on: https://chromium-review.googlesource.com/1112224
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#569866}
[modify] https://crrev.com/6496ff23de6421d900baab8badc7f279e8afb14d/DEPS
Project Member

Comment 3 by bugdroid1@chromium.org, Jul 6

Labels: merge-merged-v2spa
The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/7d5aeedd35fc9f81d21184f81f88fc4b14cf07b6

commit 7d5aeedd35fc9f81d21184f81f88fc4b14cf07b6
Author: Sam Balana <sbalana@google.com>
Date: Fri Jul 06 22:51:50 2018

Add gitignore and update documentation.

Before this change, I frequently committed dependency files on accident.
With a `.gitignore` file, I do not have to worry anymore! To follow the
convention with tracing, systrace, and telemetry, I added this file in
dashboard's root directory (`/dashboard`).

Since I was here, I updated the documentation for setting up v2spa on a
local development machine.

Bug:  chromium:855252 
Change-Id: Id205ce2c2e730b0f8e5a02657477b41b2d122310
Reviewed-on: https://chromium-review.googlesource.com/1121104
Commit-Queue: Ben Hayden <benjhayden@chromium.org>
Reviewed-by: Ben Hayden <benjhayden@chromium.org>

[modify] https://crrev.com/7d5aeedd35fc9f81d21184f81f88fc4b14cf07b6/dashboard/dashboard/spa/chart-section.js
[modify] https://crrev.com/7d5aeedd35fc9f81d21184f81f88fc4b14cf07b6/dashboard/docs/v2spa.md
[add] https://crrev.com/7d5aeedd35fc9f81d21184f81f88fc4b14cf07b6/common/node_runner/node_runner/package-lock.json
[modify] https://crrev.com/7d5aeedd35fc9f81d21184f81f88fc4b14cf07b6/dashboard/v2spa_dev.yaml
[add] https://crrev.com/7d5aeedd35fc9f81d21184f81f88fc4b14cf07b6/dashboard/.gitignore
Project Member

Comment 4 by bugdroid1@chromium.org, Jul 6 

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/561c9f103b1053577e2cb5765cac46dfb05bb9c9

commit 561c9f103b1053577e2cb5765cac46dfb05bb9c9
Author: Sam Balana <sbalana@google.com>
Date: Fri Jul 06 22:55:10 2018

Support Redux DevTools Extension in dashboard v2spa.

It is currently not possible to use the Redux DevTools Extension with v2spa.
Redux DevTools Extension allows us to view the Redux state historically before
and after each action dispatched. This is incredibly helpful for debugging.
This CL allows us to use Redux DevTools Extension in development and
production.

Performance:
The Redux DevTools Extension is supported through Redux middleware, which is
only injected when the user has Redux DevTools installed.  This will not cause
a performance hit for users who do not have the extension installed. For users
who do have the extension installed in production, the extension is turned off
by default. To turn the extension on, users can click the "Start recording"
button.

Demos:
 - https://redux-devtools-extension-dot-chromeperf.appspot.com/
 - https://redux-devtools-extension-prod-dot-chromeperf.appspot.com/

Change-Id: I07eef459052fa4b66248bf284d60a2bb0173d006
Bug:  chromium:855252 
Reviewed-on: https://chromium-review.googlesource.com/1108570
Commit-Queue: Ben Hayden <benjhayden@chromium.org>
Reviewed-by: Ben Hayden <benjhayden@chromium.org>

[modify] https://crrev.com/561c9f103b1053577e2cb5765cac46dfb05bb9c9/dashboard/dashboard/spa/report-section.js
[modify] https://crrev.com/561c9f103b1053577e2cb5765cac46dfb05bb9c9/dashboard/dashboard/spa/request-base.js
[modify] https://crrev.com/561c9f103b1053577e2cb5765cac46dfb05bb9c9/dashboard/dashboard/spa/alerts-section.js
[modify] https://crrev.com/561c9f103b1053577e2cb5765cac46dfb05bb9c9/dashboard/dashboard/spa/element-base.js
[modify] https://crrev.com/561c9f103b1053577e2cb5765cac46dfb05bb9c9/dashboard/dashboard/spa/chart-timeseries.js
[modify] https://crrev.com/561c9f103b1053577e2cb5765cac46dfb05bb9c9/dashboard/dashboard/spa/session-ids.js
[modify] https://crrev.com/561c9f103b1053577e2cb5765cac46dfb05bb9c9/dashboard/dashboard/spa/triage-new.js
[modify] https://crrev.com/561c9f103b1053577e2cb5765cac46dfb05bb9c9/dashboard/dashboard/spa/chromeperf-app.js
Project Member

Comment 5 by bugdroid1@chromium.org, Jul 12 

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/3e8353eb2960a838c942e694d3a5bc1c9b657585

commit 3e8353eb2960a838c942e694d3a5bc1c9b657585
Author: Sam Balana <sbalana@google.com>
Date: Thu Jul 12 22:32:55 2018

Call thunks directly within other thunks.

Taking from the Polymer Redux Tips document,
"Think carefully about when you dispatch an action. That causes polymer to
re-render, which can cost a lot of CPU time, which would be wasted if another
action is dispatched shortly after. Instead of dispatching multiple reducers in
rapid succession, you can either define a single reducer that calls other
reducers directly, or use a meta reducer like chain that takes several actions
and calls them directly through the reducer registry."

These changes only cover portions of the code that CL 1130105 did not completely
rewrite. There will be more of these changes in that CL, but this would be a
great opportunity to get some early feedback on this method.

It is also worthwhile to note that Redux DevTools no longer goes past the
default action limit. Everything action is now recorded from the very start of
the application. This seemed to speed up the application overall while using
Redux DevTools.

Bug:  chromium:855252 
Change-Id: I6df708fe11dbab5a6585463e41273992c98e9316
Reviewed-on: https://chromium-review.googlesource.com/1132605
Reviewed-by: Ben Hayden <benjhayden@chromium.org>
Commit-Queue: Sam Balana <sbalana@google.com>

[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/report-section.js
[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/chart-pair.js
[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/dropdown-input.js
[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/chart-timeseries.js
[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/request-base.js
[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/chart-base.js
[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/chart-section.js
[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/expand-button.js
[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/test-suites.js
[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/test-suite-descriptors.js
[modify] https://crrev.com/3e8353eb2960a838c942e694d3a5bc1c9b657585/dashboard/dashboard/spa/chromeperf-app.js
Project Member

Comment 6 by bugdroid1@chromium.org, Aug 13 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0374f46f94484f939b9a96fc75b4331ad72ca526

commit 0374f46f94484f939b9a96fc75b4331ad72ca526
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Mon Aug 13 23:33:44 2018

Roll src/third_party/catapult 21cd5a309631..501bed82a534 (1 commits)

https://chromium.googlesource.com/catapult.git/+log/21cd5a309631..501bed82a534


git log 21cd5a309631..501bed82a534 --date=short --no-merges --format='%ad %ae %s'
2018-08-13 sbalana@google.com Add support for ES modules in third_party/idb.


Created with:
  gclient setdep -r src/third_party/catapult@501bed82a534

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:855252 
TBR=sullivan@chromium.org

Change-Id: Ib96332d41a3b8187d1fa323a543a987136e113ba
Reviewed-on: https://chromium-review.googlesource.com/1173173
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#582743}
[modify] https://crrev.com/0374f46f94484f939b9a96fc75b4331ad72ca526/DEPS

Comment 7 by benjhayden@chromium.org, Aug 20

Blocking: 875941
Project Member

Comment 8 by bugdroid1@chromium.org, Sep 3 

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/450ae33192a96c3841358571bbbc5ea0dd2747b9

commit 450ae33192a96c3841358571bbbc5ea0dd2747b9
Author: Sam Balana <sbalana@google.com>
Date: Mon Sep 03 04:32:18 2018

Use IndexedDB to cache timeseries data.

The Chrome performance dashboard takes a considerable time fetching
timeseries data from the backend. It is fair to assume that users
will be viewing the same chart several times to monitor regressions
and improvements. By storing timeseries data in browser persistent
storage, users will not have to wait for the data they previously
viewed.

Demo: https://sbalana-dot-chromeperf.appspot.com

Bug:  chromium:855252 
Change-Id: I53fa5e008041d07571401b5ef3da0a0946ca0095
Reviewed-on: https://chromium-review.googlesource.com/1130105
Commit-Queue: Ben Hayden <benjhayden@chromium.org>
Reviewed-by: Ben Hayden <benjhayden@chromium.org>

[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/report-section.js
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/chart-pair.js
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/index.js
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/bin/deploy_v2spa
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/timeseries-cache.html
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/common/node_runner/node_runner/package.json
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/common/node_runner/node_runner/package-lock.json
[add] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/webpack.config.js
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/docs/v2spa.md
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/chart-timeseries.js
[add] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/sw-utils/timing.js
[add] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/sw-utils/timeseries-cache-request-test.html
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/v2spa_dev.yaml
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/timeseries-cache.js
[add] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/service-worker-listener-test.html
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/request-base.js
[add] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/sw-utils/cache-request-base.js
[add] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/sw-utils/timeseries-cache-request.js
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/v2spa.yaml
[add] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/service-worker-listener.js
[add] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/service-worker.js
[add] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/sw-utils/range.js
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/chart-timeseries.html
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/tracing/tracing/base/math/range_test.html
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/utils.js
[add] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/sw-utils/google-analytics.js
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/report-section.html
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/tracing/tracing/base/math/range.html
[modify] https://crrev.com/450ae33192a96c3841358571bbbc5ea0dd2747b9/dashboard/dashboard/spa/test-suite-descriptors.js

Comment 9 by benjhayden@chromium.org, Sep 5

Status: Fixed (was: Assigned)
Thanks, Sam!
Project Member

Comment 10 by bugdroid1@chromium.org, Sep 5 

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/506ab3270b2fc584e2f8504e09c6cde173eba2f5

commit 506ab3270b2fc584e2f8504e09c6cde173eba2f5
Author: Sam Balana <sbalana@google.com>
Date: Tue Sep 04 21:40:44 2018

Rename reportTemplateIds to reportTemplateInfos.

The variable `reportTemplateIds` caused some confusion for me when
working through adding Redux to reports. The variable holds an object of
information about a report template. The current name made me assume
that it was a unique identifying string or integer.

Renaming the variable to `reportTemplateInfos` helps ease this confusion
for me. Let me know what you think! :)

Bug:  chromium:855252 
Change-Id: Ia78a7df54f2017df43088e250f3f13a6df5cbb31
Reviewed-on: https://chromium-review.googlesource.com/1178633
Commit-Queue: Sam Balana <sbalana@google.com>
Reviewed-by: Ben Hayden <benjhayden@chromium.org>

[modify] https://crrev.com/506ab3270b2fc584e2f8504e09c6cde173eba2f5/dashboard/dashboard/spa/report-names.js
[modify] https://crrev.com/506ab3270b2fc584e2f8504e09c6cde173eba2f5/dashboard/dashboard/spa/alerts-section.js
[modify] https://crrev.com/506ab3270b2fc584e2f8504e09c6cde173eba2f5/dashboard/dashboard/spa/report-section.js
Project Member

Comment 11 by bugdroid1@chromium.org, Sep 5 

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/58c1e33b730d84c14f5d8c4d7e05c99a0b4d0427

commit 58c1e33b730d84c14f5d8c4d7e05c99a0b4d0427
Author: Sam Balana <sbalana@google.com>
Date: Tue Sep 04 22:22:32 2018

Use IndexedDB to cache report data.

After finishing implementation of caching timeseries data (CL 1130105),
the next long-lasting request type is for generating reports.

The main report, "Chrome Performance Overview", takes roughly 6 to 12
seconds to arrive from the backend. This information is not easily
cachable due to varying cross products of available information between
revisions. For example, the latest revision often have new bots that
older revisions did not. The backend takes these inconsistencies into
account while comparing two revision.

The general idea of this CL is to display the information that is
currently available then push an update the the UI once the backend
responses with the fresh statistics.

Early results from Google Analytics show the cache responding in less
than 100ms.

Demo: https://reports-dot-chromeperf.appspot.com

Bug:  chromium:855252 
Change-Id: Ia60c491ac1ec2a552d6fd37cc0c3aabd75c78201
Reviewed-on: https://chromium-review.googlesource.com/1176384
Commit-Queue: Sam Balana <sbalana@google.com>
Reviewed-by: Ben Hayden <benjhayden@chromium.org>

[modify] https://crrev.com/58c1e33b730d84c14f5d8c4d7e05c99a0b4d0427/dashboard/dashboard/spa/report-section.js
[modify] https://crrev.com/58c1e33b730d84c14f5d8c4d7e05c99a0b4d0427/dashboard/dashboard/spa/report-cache.html
[modify] https://crrev.com/58c1e33b730d84c14f5d8c4d7e05c99a0b4d0427/dashboard/dashboard/spa/service-worker.js
[modify] https://crrev.com/58c1e33b730d84c14f5d8c4d7e05c99a0b4d0427/dashboard/dashboard/spa/report-cache.js
[add] https://crrev.com/58c1e33b730d84c14f5d8c4d7e05c99a0b4d0427/dashboard/dashboard/spa/sw-utils/report-cache-request.js
[add] https://crrev.com/58c1e33b730d84c14f5d8c4d7e05c99a0b4d0427/dashboard/dashboard/spa/sw-utils/report-cache-request-test.html
►

Sign in to add a comment