New issue
Advanced search Search tips

Issue 736933 link

Starred by 0 users
Status: Assigned
Owner:
petewil@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Feature



Sign in to add a comment

Improve pages that break when offlined (Page Renovations)

Project Member Reported by collinbaker@google.com, Jun 26 2017 
Many pages that use JavaScript to load content dynamically or to control page presentation break when offlined. This project attempts to fix these pages by preprocessing them before taking a snapshot.

Example of issue: https://bugs.chromium.org/p/chromium/issues/detail?id=733366

Mobile Wikipedia uses JavaScript to collapse and expand article sections. The pages have their sections collapsed upon load, so the page snapshot has collapsed sections that can't be expanded due to the lack of JavaScript.

This framework will work by executing pieces of JavaScript in pages before taking a snapshot. This JavaScript will modify the page to prepare it for offlining. Such scripts could expand Wikipedia sections and force loading of lazily-loaded content, for example.

Design doc:
https://docs.google.com/document/d/1WuwRJaxBKAwVpeAHG0xhq5uRxMTB-j4LFleT4QEqANU

The implementation of this feature will be split into several CLs.

Comment 1 by petewil@chromium.org, Jun 26 2017

Components: UI>Browser>Offline

Comment 2 by petewil@chromium.org, Jun 26 2017

Description: Show this description

Comment 3 by dim...@chromium.org, Jun 28 2017

Labels: -OS-Linux OS-Android
Owner: petewil@chromium.org
Status: Assigned (was: Unconfirmed)
Could not assign to Collin. Pete, could you make sure Collin has bug editing and try bot access?
Project Member

Comment 4 by sheriffbot@chromium.org, Jul 12 2017

Labels: Hotlist-Google

Comment 5 by petewil@chromium.org, Jul 12 2017

Owner: collinbaker@google.com
Project Member

Comment 6 by bugdroid1@chromium.org, Jul 19 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/17f92a54e93cb9e7680355ba9b3a7cb928e58ecb

commit 17f92a54e93cb9e7680355ba9b3a7cb928e58ecb
Author: Collin Baker <collinbaker@google.com>
Date: Wed Jul 19 21:41:25 2017

[Offline pages] Add feature flag for page renovation framework.

This feature flag will be used to enable and disable use of
renovations.

Design doc:
https://docs.google.com/a/google.com/document/d/1WuwRJaxBKAwVpeAHG0xhq5uRxMTB-j4LFleT4QEqANU

Bug: 736933
Change-Id: I2808e734a8a60e7b7eb45efc85b51299500018f4
Reviewed-on: https://chromium-review.googlesource.com/572664
Commit-Queue: Collin Baker <collinbaker@google.com>
Reviewed-by: Yafei Duan <romax@chromium.org>
Reviewed-by: Peter Williamson <petewil@chromium.org>
Cr-Commit-Position: refs/heads/master@{#487998}
[modify] https://crrev.com/17f92a54e93cb9e7680355ba9b3a7cb928e58ecb/chrome/browser/about_flags.cc
[modify] https://crrev.com/17f92a54e93cb9e7680355ba9b3a7cb928e58ecb/chrome/browser/flag_descriptions.cc
[modify] https://crrev.com/17f92a54e93cb9e7680355ba9b3a7cb928e58ecb/chrome/browser/flag_descriptions.h
[modify] https://crrev.com/17f92a54e93cb9e7680355ba9b3a7cb928e58ecb/components/offline_pages/core/offline_page_feature.cc
[modify] https://crrev.com/17f92a54e93cb9e7680355ba9b3a7cb928e58ecb/components/offline_pages/core/offline_page_feature.h
[modify] https://crrev.com/17f92a54e93cb9e7680355ba9b3a7cb928e58ecb/tools/metrics/histograms/enums.xml
Project Member

Comment 7 by bugdroid1@chromium.org, Jul 21 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a453f986cf3aa5c07f5a97d601df02d3b1326804

commit a453f986cf3aa5c07f5a97d601df02d3b1326804
Author: Collin Baker <collinbaker@google.com>
Date: Fri Jul 21 17:40:12 2017

[Offline pages] Change SnapshotController interface for renovations.

Adds methods to SnapshotController and SnapshotController::Client for
the renovation framework. Also adds basic implementations to all
SnapshotController::Client implementors. This doesn't change the
behavior of the SnapshotController yet.

Design doc:
https://docs.google.com/document/d/1WuwRJaxBKAwVpeAHG0xhq5uRxMTB-j4LFleT4QEqANU

Bug: 736933
Change-Id: Iee15c54fa4524b1c09a87f6e6c43a8df8ea630ee
Reviewed-on: https://chromium-review.googlesource.com/580692
Reviewed-by: Peter Williamson <petewil@chromium.org>
Reviewed-by: Dmitry Titov <dimich@chromium.org>
Reviewed-by: Yafei Duan <romax@chromium.org>
Commit-Queue: Collin Baker <collinbaker@google.com>
Cr-Commit-Position: refs/heads/master@{#488696}
[modify] https://crrev.com/a453f986cf3aa5c07f5a97d601df02d3b1326804/chrome/browser/offline_pages/background_loader_offliner.cc
[modify] https://crrev.com/a453f986cf3aa5c07f5a97d601df02d3b1326804/chrome/browser/offline_pages/background_loader_offliner.h
[modify] https://crrev.com/a453f986cf3aa5c07f5a97d601df02d3b1326804/chrome/browser/offline_pages/prerendering_loader.cc
[modify] https://crrev.com/a453f986cf3aa5c07f5a97d601df02d3b1326804/chrome/browser/offline_pages/prerendering_loader.h
[modify] https://crrev.com/a453f986cf3aa5c07f5a97d601df02d3b1326804/chrome/browser/offline_pages/recent_tab_helper.cc
[modify] https://crrev.com/a453f986cf3aa5c07f5a97d601df02d3b1326804/chrome/browser/offline_pages/recent_tab_helper.h
[modify] https://crrev.com/a453f986cf3aa5c07f5a97d601df02d3b1326804/components/offline_pages/core/snapshot_controller.cc
[modify] https://crrev.com/a453f986cf3aa5c07f5a97d601df02d3b1326804/components/offline_pages/core/snapshot_controller.h
[modify] https://crrev.com/a453f986cf3aa5c07f5a97d601df02d3b1326804/components/offline_pages/core/snapshot_controller_unittest.cc
Project Member

Comment 8 by bugdroid1@chromium.org, Jul 25 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/16a090e32e086668f52fec09b28239e01d2d9f8d

commit 16a090e32e086668f52fec09b28239e01d2d9f8d
Author: Collin Baker <collinbaker@google.com>
Date: Tue Jul 25 19:07:29 2017

[Offline pages] Add class for loading renovation scripts.

Adds PageRenovationLoader which will be used for loading page
renovation JavaScript from storage. This implementation doesn't load
from storage yet: it currently uses a script defined in-line that does
nothing. The LoadSource() method will be changed to load the scripts
using ui::ResourceBundle.

Also adds PageRenovation, a class that represents a single renovation
that can be run. Upon PageRenovationLoader construction, a list of
PageRenovation instances will be built for every known renovation.

Design doc:
https://docs.google.com/document/d/1WuwRJaxBKAwVpeAHG0xhq5uRxMTB-j4LFleT4QEqANU

Bug: 736933
Change-Id: Icee9d26f333c46560edbfb6413a6098c6bc93533
Reviewed-on: https://chromium-review.googlesource.com/580727
Reviewed-by: Yafei Duan <romax@chromium.org>
Reviewed-by: Dmitry Titov <dimich@chromium.org>
Reviewed-by: Peter Williamson <petewil@chromium.org>
Commit-Queue: Collin Baker <collinbaker@google.com>
Cr-Commit-Position: refs/heads/master@{#489394}
[modify] https://crrev.com/16a090e32e086668f52fec09b28239e01d2d9f8d/components/offline_pages/core/BUILD.gn
[add] https://crrev.com/16a090e32e086668f52fec09b28239e01d2d9f8d/components/offline_pages/core/renovations/BUILD.gn
[add] https://crrev.com/16a090e32e086668f52fec09b28239e01d2d9f8d/components/offline_pages/core/renovations/page_renovation.h
[add] https://crrev.com/16a090e32e086668f52fec09b28239e01d2d9f8d/components/offline_pages/core/renovations/page_renovation_loader.cc
[add] https://crrev.com/16a090e32e086668f52fec09b28239e01d2d9f8d/components/offline_pages/core/renovations/page_renovation_loader.h
[add] https://crrev.com/16a090e32e086668f52fec09b28239e01d2d9f8d/components/offline_pages/core/renovations/page_renovation_loader_unittest.cc
Project Member

Comment 9 by bugdroid1@chromium.org, Jul 27 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8691148fbfffe99094380748a7a24da672f3f3de

commit 8691148fbfffe99094380748a7a24da672f3f3de
Author: Collin Baker <collinbaker@google.com>
Date: Thu Jul 27 05:57:23 2017

[Offline pages] Manage renovations from SnapshotController

Implements changes to the SnapshotController for running renovations
and waiting for them to complete. This new functionality is enabled
with a boolean flag passed in at construction. When disabled, the
SnapshotController will function just like before.

Design doc:
https://docs.google.com/document/d/1WuwRJaxBKAwVpeAHG0xhq5uRxMTB-j4LFleT4QEqANU

Bug: 736933
Change-Id: I44f3be47f8610876b0506c8811d13c04ac9642a3
Reviewed-on: https://chromium-review.googlesource.com/584650
Reviewed-by: Dmitry Titov <dimich@chromium.org>
Reviewed-by: Yafei Duan <romax@chromium.org>
Commit-Queue: Collin Baker <collinbaker@google.com>
Cr-Commit-Position: refs/heads/master@{#489868}
[modify] https://crrev.com/8691148fbfffe99094380748a7a24da672f3f3de/chrome/browser/offline_pages/background_loader_offliner.cc
[modify] https://crrev.com/8691148fbfffe99094380748a7a24da672f3f3de/chrome/browser/offline_pages/background_loader_offliner_unittest.cc
[modify] https://crrev.com/8691148fbfffe99094380748a7a24da672f3f3de/chrome/browser/offline_pages/prerendering_loader.cc
[modify] https://crrev.com/8691148fbfffe99094380748a7a24da672f3f3de/components/offline_pages/core/snapshot_controller.cc
[modify] https://crrev.com/8691148fbfffe99094380748a7a24da672f3f3de/components/offline_pages/core/snapshot_controller.h
Project Member

Comment 10 by bugdroid1@chromium.org, Aug 2 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9792d744ffe5ceddd86a302b8a1df70a6cae268c

commit 9792d744ffe5ceddd86a302b8a1df70a6cae268c
Author: Collin Baker <collinbaker@google.com>
Date: Wed Aug 02 00:40:57 2017

[Offline pages] Add PageRenovator for script selection and injection.

Implements PageRenovator which is responsible for picking renovations,
injecting them into a page, and notifying the user of script
completion.

Design doc:
https://docs.google.com/document/d/1WuwRJaxBKAwVpeAHG0xhq5uRxMTB-j4LFleT4QEqANU

Bug: 736933
Change-Id: Icbd8b494d75298ed355f9666971ecf918b441d2e
Reviewed-on: https://chromium-review.googlesource.com/585907
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Jochen Eisinger <jochen@chromium.org>
Reviewed-by: Yafei Duan <romax@chromium.org>
Reviewed-by: Justin DeWitt <dewittj@chromium.org>
Commit-Queue: Collin Baker <collinbaker@google.com>
Cr-Commit-Position: refs/heads/master@{#491175}
[modify] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/BUILD.gn
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/content/renovations/BUILD.gn
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/content/renovations/DEPS
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/content/renovations/render_frame_script_injector.cc
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/content/renovations/render_frame_script_injector.h
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/content/renovations/test/DEPS
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/content/renovations/test/page_renovator_browsertest.cc
[modify] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/core/renovations/BUILD.gn
[modify] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/core/renovations/page_renovation_loader.h
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/core/renovations/page_renovator.cc
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/core/renovations/page_renovator.h
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/core/renovations/page_renovator_unittest.cc
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/offline_pages/core/renovations/script_injector.h
[add] https://crrev.com/9792d744ffe5ceddd86a302b8a1df70a6cae268c/components/test/data/offline_pages/renovator_test_page.html

Comment 11 by collinbaker@google.com, Aug 4 2017

Owner: petewil@chromium.org
Transferring to Pete since today is my last day.
Project Member

Comment 12 by bugdroid1@chromium.org, Aug 4 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9

commit 0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9
Author: Collin Baker <collinbaker@google.com>
Date: Fri Aug 04 22:38:05 2017

[Offline pages] Create resources file for renovations.

Creates file that will contain renovation JS code and adds this to the
components_resources pack. This file is to be loaded with the
ui::ResourceBundle. Additionally, this commit sets up the necessary
build dependencies.

Design doc:
https://docs.google.com/document/d/1WuwRJaxBKAwVpeAHG0xhq5uRxMTB-j4LFleT4QEqANU

Bug: 736933
Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation
Change-Id: I0d757fbb06f5cacfd9cd9df2c0c07e4876e62709
Reviewed-on: https://chromium-review.googlesource.com/592349
Commit-Queue: Collin Baker <collinbaker@google.com>
Reviewed-by: Filip Gorski <fgorski@chromium.org>
Reviewed-by: Jochen Eisinger <jochen@chromium.org>
Reviewed-by: Scott Violet <sky@chromium.org>
Cr-Commit-Position: refs/heads/master@{#492155}
[modify] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/offline_pages/content/renovations/test/DEPS
[modify] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/offline_pages/content/renovations/test/page_renovator_browsertest.cc
[modify] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/offline_pages/core/renovations/BUILD.gn
[add] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/offline_pages/core/renovations/DEPS
[modify] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/offline_pages/core/renovations/page_renovation_loader.cc
[add] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/offline_pages/resources/PRESUBMIT.py
[add] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/offline_pages/resources/compiled_resources2.gyp
[add] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/offline_pages/resources/renovations.js
[modify] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/resources/OWNERS
[modify] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/resources/components_resources.grd
[add] https://crrev.com/0ffe2b0c5f0b30dad6975a7d2dcdad73d152d1e9/components/resources/offline_pages_resources.grdp
Project Member

Comment 13 by bugdroid1@chromium.org, Aug 5 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/17ab538d31b7a6225bee73c6fe69873a6362ae54

commit 17ab538d31b7a6225bee73c6fe69873a6362ae54
Author: Collin Baker <collinbaker@google.com>
Date: Sat Aug 05 01:10:32 2017

[Offline pages] Add page renovation for mobile Wikipedia.

Adds a renovation that unfolds sections and forces images to load for
mobile Wikipedia pages. This is part of the renovation framework for
Offline Pages that runs scripts in pages before snapshotting to
prepare them for offline viewing.

Design doc:
https://docs.google.com/document/d/1WuwRJaxBKAwVpeAHG0xhq5uRxMTB-j4LFleT4QEqANU

Bug: 736933, 733366
Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation
Change-Id: I34e4ec97be79ecc43de3519f8e47a489e2732744
Reviewed-on: https://chromium-review.googlesource.com/592347
Commit-Queue: Collin Baker <collinbaker@google.com>
Reviewed-by: Filip Gorski <fgorski@chromium.org>
Reviewed-by: Yafei Duan <romax@chromium.org>
Cr-Commit-Position: refs/heads/master@{#492205}
[modify] https://crrev.com/17ab538d31b7a6225bee73c6fe69873a6362ae54/components/offline_pages/content/renovations/test/page_renovator_browsertest.cc
[modify] https://crrev.com/17ab538d31b7a6225bee73c6fe69873a6362ae54/components/offline_pages/core/renovations/page_renovation_loader.cc
[modify] https://crrev.com/17ab538d31b7a6225bee73c6fe69873a6362ae54/components/offline_pages/resources/renovations.js
[add] https://crrev.com/17ab538d31b7a6225bee73c6fe69873a6362ae54/components/test/data/offline_pages/wikipedia_renovation_test_page.html
Project Member

Comment 14 by bugdroid1@chromium.org, Aug 8 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4f19f1a8a96d3c47df346e26b14d12370adb2d35

commit 4f19f1a8a96d3c47df346e26b14d12370adb2d35
Author: Collin Baker <collinbaker@google.com>
Date: Tue Aug 08 23:49:12 2017

[Offline pages] Use renovations in background loader.

Integrates renovations into the background loader. Use of renovations
is hidden behind a feature flag. When enabled, the renovation
framework will select renovations to run in a page, then inject their
scripts into the page before snapshotting in the background
offliner. At this point, only a renovation for unfolding sections in
mobile Wikipedia pages is implemented.

Design doc:
https://docs.google.com/document/d/1WuwRJaxBKAwVpeAHG0xhq5uRxMTB-j4LFleT4QEqANU

Bug: 736933, 733366
Change-Id: Ia97c32341952f2fce8f5c5a9225e5dc2054c78b7
Reviewed-on: https://chromium-review.googlesource.com/595011
Commit-Queue: Yafei Duan <romax@chromium.org>
Reviewed-by: Filip Gorski <fgorski@chromium.org>
Reviewed-by: Cathy Li <chili@chromium.org>
Cr-Commit-Position: refs/heads/master@{#492777}
[modify] https://crrev.com/4f19f1a8a96d3c47df346e26b14d12370adb2d35/chrome/browser/BUILD.gn
[modify] https://crrev.com/4f19f1a8a96d3c47df346e26b14d12370adb2d35/chrome/browser/offline_pages/background_loader_offliner.cc
[modify] https://crrev.com/4f19f1a8a96d3c47df346e26b14d12370adb2d35/chrome/browser/offline_pages/background_loader_offliner.h

Comment 15 by petewil@chromium.org, Feb 23 2018

Summary: Improve pages that break when offlined (Page Renovations) (was: Improve pages that break when offlined)
►

Sign in to add a comment