New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 756062 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 759682
Owner:
Closed: Jan 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

11.9% regression in blink_perf.parser at 493058:493190

Project Member Reported by ellenpli@google.com, Aug 16 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Aug 16 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=756062

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=51a2fe78f3faa630df8805fc954a0ccde510fc41dfa913b809f76871f2428cce


Bot(s) for this bug's original alert(s):

chromium-rel-mac12
bisect took 24 hours to timeout, that seems really odd for blink_perf tests. kicking off two more, on different bots.
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Sep 22 2017

Cc: ellyjo...@chromium.org
Owner: ellyjo...@chromium.org
Status: Assigned (was: Untriaged)

=== Auto-CCing suspected CL author ellyjones@chromium.org ===

Hi ellyjones@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Elly Fong-Jones
  Commit : e78297f19a802b8e4f26b170f2e4c026a226fba7
  Date   : Wed Aug 09 18:53:09 2017
  Subject: cocoa: draw default favicon in theme color

Bisect Details
  Configuration: mac_10_12_perf_bisect
  Benchmark    : blink_perf.parser
  Metric       : iframe-append-remove/iframe-append-remove
  Change       : 10.94% | 213.771781016 -> 190.385867082

Revision             Result                  N
chromium@493057      213.772 +- 19.2421      6      good
chromium@493074      215.37 +- 15.7806       6      good
chromium@493075      215.043 +- 4.1802       6      good
chromium@493076      184.094 +- 31.1414      6      bad       <--
chromium@493077      191.866 +- 10.934       6      bad
chromium@493079      192.537 +- 7.90342      6      bad
chromium@493083      189.018 +- 6.76059      6      bad
chromium@493091      189.534 +- 6.53586      6      bad
chromium@493124      189.0 +- 9.47407        6      bad
chromium@493190      190.386 +- 10.7501      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8967815259931177088


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Sep 22 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Elly Fong-Jones
  Commit : e78297f19a802b8e4f26b170f2e4c026a226fba7
  Date   : Wed Aug 09 18:53:09 2017
  Subject: cocoa: draw default favicon in theme color

Bisect Details
  Configuration: mac_10_12_mini_8gb_perf_bisect
  Benchmark    : blink_perf.parser
  Metric       : iframe-append-remove/iframe-append-remove
  Change       : 6.13% | 160.097786676 -> 150.287653716

Revision             Result                  N
chromium@493001      160.098 +- 6.75898      6      good
chromium@493075      161.431 +- 4.21318      6      good
chromium@493076      149.453 +- 2.37376      6      bad       <--
chromium@493077      148.415 +- 6.40368      6      bad
chromium@493078      148.349 +- 7.42163      6      bad
chromium@493080      148.489 +- 5.3634       6      bad
chromium@493085      149.633 +- 5.26604      6      bad
chromium@493094      149.592 +- 2.45203      6      bad
chromium@493112      149.222 +- 2.70969      6      bad
chromium@493149      150.288 +- 4.28001      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8967815253058942368


For feedback, file a bug with component Speed>Bisection
Owner: sullivan@chromium.org
It doesn't seem likely that my change affected this benchmark - my change could cause the favicon to be redrawn more than it was previously, but I would be surprised if it affected a blink perf benchmark. Is there a way to find out what this benchmark is actually benchmarking?

owner -> sullivan@ for that question
Owner: jbroman@chromium.org
Assignign to jbroman, owner of the blink_perf.parser benchmark. Jeremy, we're seeing really weird results from bisect where it consistently repros a shift in blink_perf.parser scores, at a CL that makes no sense. You can see that in #6 and #7 here, along with  bug 759682  (which also reproduces at the CL above). Any idea what's going on here? Why would this test be affected by the favicon color?

Additionally in  bug 771253  blink_perf.parser just seems to be really noisy.
Cc: jbroman@chromium.org
Owner: adithyas@chromium.org
Adithya, do you have bandwidth to take a look at this?
Mergedinto: 759682
Status: Duplicate (was: Assigned)

Sign in to add a comment