All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=732701

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgttTurgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1u-cqAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgtpLCvwsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1o-eoQkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1q3ttwkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1ra7qAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1vniqAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1vCt9woM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1rellAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgls-1sAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1t64-wsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1vn4sQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1qze4goM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1qze4ggM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1qjXqAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1qjDsQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1q-ypwkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1vKk-gsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgls_srgsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1qzeogsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1qnNpAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDglp-4sQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1tbF6wgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgttD9vQkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1t7W9wgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1rru-wkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1s-opQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1vKZrwkM


Bot(s) for this bug's original alert(s):

android-webview-nexus6

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8976919176560333168

=== Auto-CCing suspected CL author mek@chromium.org ===

Hi mek@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Marijn Kruisselbrink
  Commit : dda8780efac5966b275b8b7beeabe951b9d9c133
  Date   : Mon Jun 12 19:42:06 2017
  Subject: Implement retry behavior for several leveldb env methods in mojo env.

Bisect Details
  Configuration: android_webview_nexus6_aosp_perf_bisect
  Benchmark    : system_health.common_mobile
  Metric       : timeToFirstMeaningfulPaint_avg/load_games/load_games_lazors
  Change       : 425.32% | 234.106333333 -> 1229.80483333

Revision             Result                  N
chromium@478718      234.106 +- 28.9664      6      good
chromium@478719      226.053 +- 13.9055      6      good
chromium@478720      1226.12 +- 10.4734      6      bad       <--
chromium@478721      1226.97 +- 24.8674      6      bad
chromium@478723      1226.43 +- 15.4779      6      bad
chromium@478727      1225.25 +- 12.4953      6      bad
chromium@478735      1229.11 +- 14.5925      6      bad
chromium@478752      1229.8 +- 14.9875       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.games.lazors system_health.common_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976919176560333168

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4600418572697600


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Hmm, that's unexpected... is this only happening on android webview, or also on any other platforms? Not sure how I would get that data... if only android webview, that suggests that some operation that should never fail is always failing on android webview (as it retries for ~1 second before giving up, matching the slow down).

And how do I get any data out of these runs? Is there some way to see what UMA data was logged for them for example? Also the "To Run This Test" command seems to be missing some prerequisite setup, so not sure how to reproduce this...

 Issue 732702  has been merged into this issue.

You would need to build android chrome first before running the command. I don't believe UMA data is logged.  You should be able to get traces out of telemetry (I think they're stored somewhere under tools/perf) and if you need additional tracing categories you can edit the telemetry python files. 

 Issue 733281  has been merged into this issue.

 Issue 733297  has been merged into this issue.

 Issue 732721  has been merged into this issue.

 Issue 732721  has been merged into this issue.

 Issue 732771  has been merged into this issue.

 Issue 733648  has been merged into this issue.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/469805742123ec65bdeea766d8960c5abd7f82e8

commit 469805742123ec65bdeea766d8960c5abd7f82e8
Author: Marijn Kruisselbrink <mek@chromium.org>
Date: Thu Jun 15 19:33:41 2017

If the file to rename doesn't exist, return OK immediately.

This matches the behavior of ChromiumEnv::RenameFile and should prevent
at least some unnecesary retries of renames.

Bug:  732701 ,  732863 
Change-Id: I76637ba5f1f081f3709add94f4a085f54be0e096
Reviewed-on: https://chromium-review.googlesource.com/537134
Reviewed-by: Michael Nordman <michaeln@chromium.org>
Commit-Queue: Marijn Kruisselbrink <mek@chromium.org>
Cr-Commit-Position: refs/heads/master@{#479785}
[modify] https://crrev.com/469805742123ec65bdeea766d8960c5abd7f82e8/components/leveldb/env_mojo.cc

 Issue 733729  has been merged into this issue.

I think this is fixed now

 Issue 732771  has been merged into this issue.

Issue 733299 has been merged into this issue.