New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 602318 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: May 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

4.8%-6.2% regression in dromaeo.domcoremodify at 385900:386107

Project Member Reported by rsch...@chromium.org, Apr 11 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=602318

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg2P6vtgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg2JSUrwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg2JL0vQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg2Ja6sAoM


Bot(s) for this bug's original alert(s):

chromium-rel-win7-gpu-ati
chromium-rel-win7-gpu-intel
chromium-rel-win7-gpu-nvidia
chromium-rel-win7-x64-dual
Cc: ashej...@chromium.org
Labels: TE-Triaged
Bisect kicked with wider revision range.

Details on the below link:
=========================
https://chromeperf.appspot.com/group_report?bug_id=602318

Thank you!
Bisect kicked with wider revision range as previous bisect failed.

Details on the below link:
=======================
https://chromeperf.appspot.com/group_report?bug_id=602318

Thank you!
Cc: hayato@chromium.org
Owner: hayato@chromium.org

=== Auto-CCing suspected CL author hayato@chromium.org ===

Hi hayato@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : ASSERT -> {DCHECK|DCHECK_XX}, ENABLE(ASSERT) -> DCHECK_IS_ON() in dom
Author  : hayato
Commit description:
  
There are several places where DCHECK_{EQ/NE} can not be used because
"operator<<(ostream&, T&)" is not defined for T.

To isolate issues, other macros, such as RELEASE_ASSERT, will be replaced in another CL.

BUG= 596760 , 601669 

Review URL: https://codereview.chromium.org/1854423002

Cr-Commit-Position: refs/heads/master@{#385971}
Commit  : 63915eb8cbda0af155de9e774283fa0828e9f446
Date    : Fri Apr 08 03:17:02 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@385830  806.156  7.35459  8  good
chromium@385900  808.126  13.3462  8  good
chromium@385951  801.957  12.4595  8  good
chromium@385964  808.824  11.8429  5  good
chromium@385968  806.605  7.32754  5  good
chromium@385970  806.324  11.78    5  good
chromium@385971  777.679  6.51048  5  bad    <--
chromium@385977  779.311  6.72406  5  bad
chromium@386003  779.526  8.42253  8  bad
chromium@386150  766.36   6.83089  5  bad

Bisect job ran on: winx64ati_perf_bisect
Bug ID: 602318

Test Command: python src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests dromaeo.domcoremodify
Test Metric: dom/dom
Relative Change: 4.72%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1343
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9013588542867668128


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5887025010442240

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: tkent@chromium.org
This CL should not have any performance impact in a release build.
It was a mechanical replace: ASSERT -> DCHECK_XXX.

pinging @tkent, just in case.
Can we revert and see if the graph comes back? This should be easy to test.

Comment 7 by hayato@chromium.org, May 10 2016

Let me start bisect again.
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, May 10 2016


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : ASSERT -> {DCHECK|DCHECK_XX}, ENABLE(ASSERT) -> DCHECK_IS_ON() in dom
Author  : hayato
Commit description:
  
There are several places where DCHECK_{EQ/NE} can not be used because
"operator<<(ostream&, T&)" is not defined for T.

To isolate issues, other macros, such as RELEASE_ASSERT, will be replaced in another CL.

BUG= 596760 , 601669 

Review URL: https://codereview.chromium.org/1854423002

Cr-Commit-Position: refs/heads/master@{#385971}
Commit  : 63915eb8cbda0af155de9e774283fa0828e9f446
Date    : Fri Apr 08 03:17:02 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@385899  787.566  6.31624  5  good
chromium@385951  795.805  6.52794  5  good
chromium@385964  788.151  7.53684  5  good
chromium@385968  796.604  8.123    5  good
chromium@385970  802.717  13.2409  5  good
chromium@385971  761.252  11.6365  5  bad    <--
chromium@385977  758.902  9.98567  5  bad
chromium@386003  763.253  6.41385  5  bad
chromium@386107  751.717  7.39558  5  bad

Bisect job ran on: winx64intel_perf_bisect
Bug ID: 602318

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests dromaeo.domcoremodify
Test Metric: dom/dom
Relative Change: 4.55%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64intel_perf_bisect/builds/918
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9013064243558544832


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5802793655336960

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 9 by danakj@chromium.org, May 10 2016

Cc: danakj@chromium.org
DCHECK should be a no-op on official builds. What does the compiler output look like with ASSERT vs DCHECK? They should both be nothing.
Cc: thakis@chromium.org
speculatively reverting like esprehn suggests sgtm
> DCHECK should be a no-op on official builds. What does the compiler output look like with ASSERT vs DCHECK? They should both be nothing.

Yeah, there is no difference between ASSERT and DCHECK in a release build, as far as I can tell from objdump with --disassembly.

Regarding reverting, we can not revert the CL cleanly because "git revert thisCL" caused a lot of conflicts. It will be yet another replace in a *reverse* order, "DCHECK -> ASSERT".
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, May 11 2016


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : ASSERT -> {DCHECK|DCHECK_XX}, ENABLE(ASSERT) -> DCHECK_IS_ON() in dom
Author  : hayato
Commit description:
  
There are several places where DCHECK_{EQ/NE} can not be used because
"operator<<(ostream&, T&)" is not defined for T.

To isolate issues, other macros, such as RELEASE_ASSERT, will be replaced in another CL.

BUG= 596760 , 601669 

Review URL: https://codereview.chromium.org/1854423002

Cr-Commit-Position: refs/heads/master@{#385971}
Commit  : 63915eb8cbda0af155de9e774283fa0828e9f446
Date    : Fri Apr 08 03:17:02 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@385899  812.567  10.5305  5  good
chromium@385951  806.789  10.2921  8  good
chromium@385964  799.05   8.81095  8  good
chromium@385968  810.831  20.3939  8  good
chromium@385970  818.311  16.151   5  good
chromium@385971  779.543  4.95094  5  bad    <--
chromium@385977  778.564  11.4106  8  bad
chromium@386003  772.42   5.57596  5  bad
chromium@386107  755.495  12.2401  5  bad

Bisect job ran on: winx64nvidia_perf_bisect
Bug ID: 602318

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests dromaeo.domcoremodify
Test Metric: dom/dom
Relative Change: 7.02%
Score: 99.5

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64nvidia_perf_bisect/builds/1633
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9012983647171052368


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5850841991348224

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Status: WontFix (was: Assigned)
I am not sure what I should do it for this regression.

- As far as I confirmed via disassembly, DCHECK_XX would be no-op in a release build.
- As far as I rechecked the CL, this CL did not anything other than changing ASSERT to DCHECK_XX.
- I confirmed that the attempt to revert this CL would cause a lot of conflicts. Thus, we can not isolate the effects of CL cleanly for TOT, I am afraid.

Given that only win bots would regress, I guess this patch is not the real cause of the regression, hopefully. Let me mark this WONTFIX.
Cc: yukishiino@chromium.org haraken@chromium.org bashi@chromium.org
Adding yukishiino, bashi, haraken, owners of dromaeo perf test. Our perf bots and bisectors run release official builds, but it looks like somehow https://codereview.chromium.org/1854423002 has regressed dromaeo performance anyway. Do we care about regressions of this size? Is it okay to WontFix?
I agree that the bisected cl can't possibly cause this -- maybe unless the perf bots build with dchecks enabled, but then their results wouldn't be useful for measuring perf, so I doubt that's the case (can you link to the bot config?).

The most likely explanation is that the bisect bisected to the wrong change.
Is DCHECK completely a no-op on official builds?

If yes, I'm fine with marking this as WontFix. Otherwise, this may be a signal to indicate that we still have some unexpected regression between ASSERT/RELEASE_ASSERT and DCHECK/CHECK.

> Is DCHECK completely a no-op on official builds?

Not only in official builds, in normal release builds as well.

Sign in to add a comment