All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=633230

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgxtiDuwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgxqqZogoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgxrzRsgoM


Bot(s) for this bug's original alert(s):

android-nexus5
android-nexus6

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9005518691259581808

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9005509420337216800

Bisect failed: Unknown
Failure reason: the build has failed due to infrastructure failure.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9005418195040601040

Bisect failed: Unknown
Failure reason: the build has failed due to infrastructure failure.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9005328713719269072

=== Auto-CCing suspected CL author hanxi@chromium.org ===

Hi hanxi@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Introduce ManifestUpgradeDetector for WebAPK to detect web manifest changes.
Author  : hanxi
Commit description:
  
The ManifestUpgradeDetector fetches the latest web manifest for a WebAPK, compare
with cached one and make a decision of whether to request re-mint.

The ManifestUpgradeDetector is initilized for WebApkActivity when it needs to check
resources changes. Once the detector starts, it observes WebContents's change,
and stops when the manifest file of the WebApk is found and fetched. This is
because not all of the urls within the WebAPK's scope link to its manifest file.
Therefore, the detector needs to follow WebContents's change until find
the manifest file.

TEST=org.chromium.chrome.browser.webapps.ManifestUpgradeDetectorTest
BUG= 624834 

Review-Url: https://codereview.chromium.org/2124513002
Cr-Commit-Position: refs/heads/master@{#408636}
Commit  : e72182e94924a4a8af00b016ed75fb1bdc554500
Date    : Fri Jul 29 14:41:54 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@408599  1201766  2243.47  5  good
chromium@408631  1200128  2896.31  5  good
chromium@408635  1199309  4486.94  5  good
chromium@408636  1216512  8688.93  5  bad    <--
chromium@408637  1218970  3663.57  5  bad
chromium@408639  1214054  4670.16  5  bad
chromium@408647  1216512  8688.93  5  bad
chromium@408662  1232077  5340.53  5  bad
chromium@408725  1229619  3426.96  5  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 633230

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: load_tools-memory:chrome:all_processes:reported_by_os:system_memory:ashmem:private_dirty_size_avg/load_tools_dropbox
Relative Change: 2.32%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2412
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005328713719269072


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5905603331883008

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

It is strange. The changes in my CL haven't been used by Chrome, since it is for WebAPK only, and WebAPK hasn't been enabled in Chrome.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9005162745870463280

I started another bisect to double check.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004782064901611504

=== Auto-CCing suspected CL author engedy@chromium.org ===

Hi engedy@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Framework for indexing of subresource filtering rules.
Author  : engedy
Commit description:
  
Make the RulesetService capable of receiving and indexing Safe Browsing
subresource filtering rules coming in through the component updater. The wire
format is still TBD, but for now consists of a single rule.

Also, change the directory structure so that both indexed and unindexed rules
are stored under a BASE directory at "$USER_DATA_DIR/Subresource Filter/".
Unindexed rules are now stored under "$BASE/Unindexed Rules/$CONTENT_VERSION/";
and indexed rules under "$BASE/Indexed Rules/$FORMAT_VERSION/$CONTENT_VERSION",
instead of "$BASE/$FORMAT_VERSION_$CONTENT_VERSION".

Furthermore, refactor TestRulesetCreator to facilitate testing by supplying
both the indexed/unindexed representation of the testing rules, and making them
available as files, file paths or blobs.

BUG=609747

Review-Url: https://codereview.chromium.org/2182493009
Cr-Commit-Position: refs/heads/master@{#408615}
Commit  : 6cfa34f00f9631c99f8540188e9a5aab7502eebe
Date    : Fri Jul 29 11:20:01 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@408611  1811251  5340.53  5  good
chromium@408614  1815347  7878.81  5  good
chromium@408615  1840742  2243.47  5  bad    <--
chromium@408616  1837466  8973.89  5  bad
chromium@408617  1837466  4670.16  5  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 633230

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: load_tools-memory:chrome:all_processes:reported_by_os:system_memory:ashmem:private_dirty_size_avg/load_tools_weather
Relative Change: 1.45%
Score: 99.5

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/3915
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005418195040601040


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4953050855243776

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

I don't think it's my CL either. That piece of code is also behind a flag.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004719844242326192

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Introduce ManifestUpgradeDetector for WebAPK to detect web manifest changes.
Author  : hanxi
Commit description:
  
The ManifestUpgradeDetector fetches the latest web manifest for a WebAPK, compare
with cached one and make a decision of whether to request re-mint.

The ManifestUpgradeDetector is initilized for WebApkActivity when it needs to check
resources changes. Once the detector starts, it observes WebContents's change,
and stops when the manifest file of the WebApk is found and fetched. This is
because not all of the urls within the WebAPK's scope link to its manifest file.
Therefore, the detector needs to follow WebContents's change until find
the manifest file.

TEST=org.chromium.chrome.browser.webapps.ManifestUpgradeDetectorTest
BUG= 624834 

Review-Url: https://codereview.chromium.org/2124513002
Cr-Commit-Position: refs/heads/master@{#408636}
Commit  : e72182e94924a4a8af00b016ed75fb1bdc554500
Date    : Fri Jul 29 14:41:54 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@408607  1200128  4096.0   5  good
chromium@408635  1201766  3663.57  5  good
chromium@408636  1215693  6730.41  5  bad    <--
chromium@408637  1214874  3663.57  5  bad
chromium@408639  1210778  2243.47  5  bad
chromium@408642  1212416  5016.55  5  bad
chromium@408649  1220608  6476.34  5  bad
chromium@408662  1228800  5016.55  5  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 633230

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: load_tools-memory:chrome:all_processes:reported_by_os:system_memory:ashmem:private_dirty_size_avg/load_tools_dropbox
Relative Change: 2.39%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/3965
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004719844242326192


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5252658680561664

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

[I've broken out the regression in weather.com into  issue 636727 ]

hanxi: Another bisect on a different device (#8: Nexus 6, #16: Nexus 5) confirmed that your patch (r408636) is responsible for a +13.6 KiB regression in ashmem (private dirty) on Dropbox on Android. Please investigate and decide whether any further action is necessary.

Perf sheriff ping: reminder to follow up on possible performance issues

It is really wired since my code is behind a commandline flag and hasn't been used by Chrome.

Could you please try running this locally (ToT vs. ToT + r408636 reverted)? If you don't have a Nexus 5, you can run the benchmark against your local checkout on the trybots:

$ tools/pert/run_benchmark try android-nexus5 system_health.memory_mobile

There are several follow up patches landed based on r408636, it is might not possible to simply to run (ToT vs. ToT + r408636 reverted). I will disable the code in r408636 to run the memory test locally.

This regression has been included in a stable release and that stable channel is now deprecated. I'm closing this so that we won't have unresponsive performance regressions in the future.