All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=700902

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg1OqvwQoM


Bot(s) for this bug's original alert(s):

chromium-rel-mac11-air

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8985230014023525600

=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Michael Hablich
  Commit : 7c3936c67a6361371d9a00d83e2c03ecc603ce76
  Date   : Thu Mar 09 08:05:42 2017
  Subject: Version 5.9.34.1 (Turn on I+TF)

Bisect Details
  Configuration: mac_air_perf_bisect
  Benchmark    : smoothness.tough_webgl_cases
  Metric       : frame_times/http___www.khronos.org_registry_webgl_sdk_demos_google_nvidia-vertex-buffer-object_index.html
  Change       : 0.56% | 16.6703216536 -> 16.763290375

Revision                           Result                      N
chromium@455712                    16.6703 +- 0.0101625        6      good
chromium@455718                    16.6698 +- 0.00388037       6      good
chromium@455719                    16.6691 +- 0.00907201       6      good
chromium@455719,v8@7c3936c67a      16.7824 +- 0.00798681       6      bad       <--
chromium@455719,v8@fbffc377e3      16.7641 +- 0.0599482        6      bad
chromium@455720                    16.7641 +- 0.0632073        6      bad
chromium@455721                    16.7611 +- 0.0566364        6      bad
chromium@455723                    16.7809 +- 0.000842157      6      bad
chromium@455733                    16.7729 +- 0.0544594        6      bad
chromium@455754                    16.7813 +- 0.00123416       6      bad
chromium@455796                    16.7633 +- 0.0675306        6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.khronos.org.registry.webgl.sdk.demos.google.nvidia.vertex.buffer.object.index.html smoothness.tough_webgl_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8985230014023525600

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4950112591151104


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Click "Original alerts at time of bug-filing:" to see the original alert. Chromeperf combined everything in one bug automatically.

 Issue 700906  has been merged into this issue.

 Issue 700909  has been merged into this issue.

kbr@, any clue why the WebGL cases suddenly are less smooth? Given that those tests are very flaky I have a hard time interpreting the results.

Issue 700962 has been merged into this issue.

The frame times of Animometer seemed to increase by ~15% from 30->38 ms. This only seems to happen on Mac Air though, which is a bit suspicious. Do we measure Animometer's score elsewhere - did this impact the score?

Issue 700962 has been merged into this issue.

It looks to me like it was actually the nvidia-vertex-buffer-object test which alerted here:

https://chromeperf.appspot.com/report?sid=6c0d91bb6c4862ceca958a00801724b0c0ec40b1a12248537b4041c23754aa58

It looks like it's alerting because of an increase in the mean_frame_time from 16.673 to 16.384 ms.

Here are the same graphs on the Mac Retina bot:

https://chromeperf.appspot.com/report?sid=a11a40c8b35bb3b439fb03e4a8f1919e37fc04fbed0edcdcfbf94c3c40eb1f2d

vertex-buffer-object didn't alert there.

While I can believe that the switch from Crankshaft to I+TF affected the vertex-buffer-object demo -- it does a lot of math and indexing -- I'm not sure a 1% increase in the frame time on this one machine warrants investigation. I would ignore that part of this alert.

-----

The Animometer portion is a different problem; the alert is on smoothness.tough_filter_cases. (Not sure why Animometer is categorized under there.) There do seem to be increases at least on the chromium-rel-mac11-air and chromium-rel-mac-retina bots.

https://chromeperf.appspot.com/report?sid=d75206fc00e0125fb3f67638df5b3d3d26f43007dd4c1075ce9f7ede8cba07e3

Haven't checked the other bots.

I'm going to proactively close this as WontFix. The issue description is wrong -- The WebGL-specific part of the Animometer benchmark that's run as part of tough_webgl_cases doesn't seem to have alerted. The Animometer problem in  Issue 700364  is being investigated separately.

Ken, I'm not sure how  issue 700364  is related? That bug is about some peak memory regressions on mobile with I+TF (note the perf bisect bots keep moving the bugs that the alerts are associated since they all got triggered by the I+TF flag flip).

Is the Animometer problem you are talking about is the increase mean_frame_time? If so, do you have a sense for how high priority this should be in our set of regressions?

Ross, the original description for this bug was:
"1% regression in smoothness.tough_webgl_cases (Animometer and WebGL)"

Different subsets of the Animometer tests are run inside different benchmarks. smoothness.tough_webgl_cases runs a modified version of the WebGL portion of Animometer with a fixed load. mean_frame_times for this benchmark look a little noisy but they didn't alert, so it looked to me like the description for this bug was wrong, and that it should have been focused solely on the nvidia-vertex-buffer-object alert.

In #11 above there was a question about Animometer's frame_times increasing. These graphs from #13 show the regression:

https://chromeperf.appspot.com/report?sid=d75206fc00e0125fb3f67638df5b3d3d26f43007dd4c1075ce9f7ede8cba07e3

The alerts on those graphs point to  Issue 700364 , which is why I blocked this on that other bug. I see that that was probably the wrong thing to do.

The regressions on Animometer (within the smoothness.tough_filters_cases benchmark, not smoothness.tough_webgl_cases) happen on more machines than just the MacBook Air, as demonstrated by the above graphs.

Ross, do you want to investigate those regressions? The nvidia-vertex-buffer-object regression is minimal and I don't intend to spend time diagnosing it. We could rename this bug, but I'm not sure how to modify the chromeperf alerts to point to this bug rather than  Issue 700364 .

Right, yeah it seems the bisect has messed up the bug ids for the graphs. I agree let's not spend time investigating nvidia-vertex-buffer-object, but we should investigate the Animometer regressions. I don't think there is any other bug for this, so I'm going to rename this bug and point the regressions on chromeperf here.

Benedikt, assigning to you for triage, looks like a 20% increase in frame time on Animometer. Might be related to the typed arrays regressions we saw elsewhere?

rmcilroy@: Can you paste the command to reproduce this locally?

Sent over chat, but it should be:
<chrome>/tools/perf/run_benchmark 'smoothness.gpu_rasterization.tough_filters_cases --browser=release

You can also filter so it only runs the Animometer page, but can't remember the exact flag to do that (you can get it with --help)

This benchmark spends almost no time in JS (either optimized or unoptimized), see the attached picture. The time in JS is mostly spend in our builtins, so improving the builtins (Array.prototype.join, Array.prototype.slice, etc.) should recover the regression over time. I don't see anything particular urgent otherwise.

Deleted: smoothness.tough_filters_cases.png 82.1 KB

	smoothness.tough_filters_cases.png 82.1 KB View Download

Adding danno@ who's working on builtins.

Any idea how the switch to I+TF caused a 20% regression in frame time if it doesn't spend any time in JS? Maybe the benchmark spends a lot of time elsewhere but the critical part between frames is JS bound?

Yes, that's quite possible. We know that the performance of the relevant builtins dropped by switching to I+TF.

See attached file for optimized code that we generate. There are a couple of minor things we could do, i.e. optimize new RegExp in TurboFan. But I don't see anything obvious in this.

Deleted: opt-tf.txt 421 KB

opt-tf.txt 421 KB View Download

Perf sheriff ping: it's been several months since the last comment; is this bug still being worked on, or should it be closed?

WontFixing due to inactivity, please reopen if you're still working on this.