New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 758907 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Xoogler
Closed: Aug 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

22.9% regression in rasterize_and_record_micro.top_25 at 496020:496140

Project Member Reported by kraynov@chromium.org, Aug 25 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Aug 25 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=758907

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=594a52f0389dddb2b1fb7b85716d1f5139d26f7a56ee7acc7c1a7a958d5c7372


Bot(s) for this bug's original alert(s):

chromium-rel-mac12-mini-8gb
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Aug 25 2017

Cc: kerrnel@chromium.org
Owner: kerrnel@chromium.org
Status: Assigned (was: Untriaged)

=== Auto-CCing suspected CL author kerrnel@chromium.org ===

Hi kerrnel@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Greg Kerr
  Commit : 66d4a01a3d80c6cb507112d230824d1e9cdafee7
  Date   : Mon Aug 21 22:45:05 2017
  Subject: Mac V2 Sandbox: Disable re-execution of helper executable.

Bisect Details
  Configuration: mac_10_12_mini_8gb_perf_bisect
  Benchmark    : rasterize_and_record_micro.top_25
  Metric       : rasterize_time/Pinterest
  Change       : 22.45% | 0.802666666667 -> 0.982833333333

Revision             Result                      N
chromium@496019      0.802667 +- 0.0225241       6      good
chromium@496080      0.798833 +- 0.019205        6      good
chromium@496095      0.799667 +- 0.00867948      6      good
chromium@496099      0.789333 +- 0.545368        9      good
chromium@496100      1.01533 +- 0.123529         6      bad       <--
chromium@496101      0.981167 +- 0.0126029       6      bad
chromium@496103      0.966333 +- 0.162915        6      bad
chromium@496110      0.997 +- 0.0235372          6      bad
chromium@496140      0.982833 +- 0.0126029       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=Pinterest rasterize_and_record_micro.top_25

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8970294886827306240


For feedback, file a bug with component Speed>Bisection
Owner: kraynov@chromium.org
I reverted the finch trial enabling the feature, so none of that code is executing anymore.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4fa660ddcdc1744e5f4380c336df5c8e2226642c

commit 4fa660ddcdc1744e5f4380c336df5c8e2226642c
Author: Greg Kerr <kerrnel@chromium.org>
Date: Thu Aug 24 20:48:27 2017

Revert "Enable Finch Trial Test for MacV2Sandbox."

This reverts commit 9705b7aaabf4c81d4e9cb13174338df9bde61cc4.

Reason for revert: This CL seems to be causing performance regressions.
 crbug.com/757993 

Original change's description:
> Enable Finch Trial Test for MacV2Sandbox.
>
> Cq-depend: CL:611386
> Bug: 749839
> Change-Id: I03057065380f5e49b899f1c9f9869e175d3223dc
> Reviewed-on: https://chromium-review.googlesource.com/612323
> Reviewed-by: Robert Sesek <rsesek@chromium.org>
> Reviewed-by: Steven Holte <holte@chromium.org>
> Commit-Queue: Greg Kerr <kerrnel@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#494502}

TBR=rsesek@chromium.org,holte@chromium.org

Bug:   756145  , 756623 
Change-Id: Iccada6b649f20f07ea4af4fdc995dc1a8426ded4
Reviewed-on: https://chromium-review.googlesource.com/633623
Reviewed-by: Greg Kerr <kerrnel@chromium.org>
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Commit-Queue: Greg Kerr <kerrnel@chromium.org>
Cr-Commit-Position: refs/heads/master@{#497180}
[modify] https://crrev.com/4fa660ddcdc1744e5f4380c336df5c8e2226642c/testing/variations/fieldtrial_testing_config.json
Cc: vmp...@chromium.org wkorman@chromium.org
kernel: This is strange, looks like the revert landed in r497180, but we've had a few data points come in since then and the numbers didn't go back down. The bisect in #3 points pretty clearly to your CL and has the same magnitude, so it's confusing that the numbers didn't come down.

Kicking off another bisect to see if it reproduces.

Adding rasterize_and_record_micro owners to see if they have any thoughts on what might be going on.
That is strange. Here's what I'm doing:

1) Running a build locally to make sure the V2 sandbox is disabled (through abort() statements)
2) I am happy to revert the CL being blamed by this bisect, I just doubt that it will help. 

But it was only landed to try and get the perf regressions to go down, so if we do think it's still somehow causing issues, I am happy to revert and get all my recent changes out of the picture.
vmpstr, wkorman: Should kernel revert that CL? So far just this single regression on a Mac with spinning disk on pinterest is known.
My only initial thought is that last week I landed a move to WPRgo which re-recorded the Pinterest page producing an actual html page but lacking styles, where prior to that it was basically a static 404 page html. See:

https://bugs.chromium.org/p/chromium/issues/detail?id=756946#c8

So we basically planned to treat the new results for Pinterest as the new baseline but the graph for this issue doesn't seem to reflect the change?
If you look at the graph in https://chromeperf.appspot.com/group_report?bug_id=758907, there is a big jump that links to  bug 756946  and then a smaller jump associated with this one. So it looks like we have a new baseline from that change, then on top of it r496100 seems to have caused an additional 22% regression. But IIUC, it's in an experiment that was later flagged off in r497180, and the graph doesn't go back down at the flag off.

Given that this only happened on one page on one bot, should we investigate further? Or WontFix unless more alerts/bisects come back telling us that r496100 is problematic?
I am ok doing WontFix and waiting to see if the issue is broader than the one Pinterest page/bot.
Thanks everyone. If this reaches a point where it's easier to diagnose by reverting the CL, just send me a bug, I'm happy to do it.
Status: WontFix (was: Assigned)
stgm. WontFix-ing for now.
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Aug 25 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Greg Kerr
  Commit : 66d4a01a3d80c6cb507112d230824d1e9cdafee7
  Date   : Mon Aug 21 22:45:05 2017
  Subject: Mac V2 Sandbox: Disable re-execution of helper executable.

Bisect Details
  Configuration: mac_10_12_mini_8gb_perf_bisect
  Benchmark    : rasterize_and_record_micro.top_25
  Metric       : rasterize_time/Pinterest
  Change       : 20.46% | 0.822666666667 -> 0.991

Revision             Result                     N
chromium@496019      0.822667 +- 0.18351        9      good
chromium@496080      0.803167 +- 0.0157745      6      good
chromium@496095      0.805167 +- 0.0289971      6      good
chromium@496099      0.7995 +- 0.019532         6      good
chromium@496100      0.987833 +- 0.0230398      6      bad       <--
chromium@496101      0.990167 +- 0.0261693      6      bad
chromium@496103      0.953444 +- 0.231781       9      bad
chromium@496110      0.994667 +- 0.0248059      6      bad
chromium@496140      0.991 +- 0.0179444         6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=Pinterest rasterize_and_record_micro.top_25

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8970271514345415056


For feedback, file a bug with component Speed>Bisection
[this bisect is the exact same page/device as before, still WontFix unless more data comes in pointing at this cl]

Sign in to add a comment