Issue metadata
Sign in to add a comment
|
4.8%-6.2% regression in dromaeo.domcoremodify at 385900:386107 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Apr 20 2016
Bisect kicked with wider revision range. Details on the below link: ========================= https://chromeperf.appspot.com/group_report?bug_id=602318 Thank you!
,
May 4 2016
Bisect kicked with wider revision range as previous bisect failed. Details on the below link: ======================= https://chromeperf.appspot.com/group_report?bug_id=602318 Thank you!
,
May 4 2016
=== Auto-CCing suspected CL author hayato@chromium.org === Hi hayato@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : ASSERT -> {DCHECK|DCHECK_XX}, ENABLE(ASSERT) -> DCHECK_IS_ON() in dom Author : hayato Commit description: There are several places where DCHECK_{EQ/NE} can not be used because "operator<<(ostream&, T&)" is not defined for T. To isolate issues, other macros, such as RELEASE_ASSERT, will be replaced in another CL. BUG= 596760 , 601669 Review URL: https://codereview.chromium.org/1854423002 Cr-Commit-Position: refs/heads/master@{#385971} Commit : 63915eb8cbda0af155de9e774283fa0828e9f446 Date : Fri Apr 08 03:17:02 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@385830 806.156 7.35459 8 good chromium@385900 808.126 13.3462 8 good chromium@385951 801.957 12.4595 8 good chromium@385964 808.824 11.8429 5 good chromium@385968 806.605 7.32754 5 good chromium@385970 806.324 11.78 5 good chromium@385971 777.679 6.51048 5 bad <-- chromium@385977 779.311 6.72406 5 bad chromium@386003 779.526 8.42253 8 bad chromium@386150 766.36 6.83089 5 bad Bisect job ran on: winx64ati_perf_bisect Bug ID: 602318 Test Command: python src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests dromaeo.domcoremodify Test Metric: dom/dom Relative Change: 4.72% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1343 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9013588542867668128 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5887025010442240 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
May 9 2016
This CL should not have any performance impact in a release build. It was a mechanical replace: ASSERT -> DCHECK_XXX. pinging @tkent, just in case.
,
May 9 2016
Can we revert and see if the graph comes back? This should be easy to test.
,
May 10 2016
Let me start bisect again.
,
May 10 2016
===== BISECT JOB RESULTS =====
Status: completed
===== SUSPECTED CL(s) =====
Subject : ASSERT -> {DCHECK|DCHECK_XX}, ENABLE(ASSERT) -> DCHECK_IS_ON() in dom
Author : hayato
Commit description:
There are several places where DCHECK_{EQ/NE} can not be used because
"operator<<(ostream&, T&)" is not defined for T.
To isolate issues, other macros, such as RELEASE_ASSERT, will be replaced in another CL.
BUG= 596760 , 601669
Review URL: https://codereview.chromium.org/1854423002
Cr-Commit-Position: refs/heads/master@{#385971}
Commit : 63915eb8cbda0af155de9e774283fa0828e9f446
Date : Fri Apr 08 03:17:02 2016
===== TESTED REVISIONS =====
Revision Mean Std Dev N Good?
chromium@385899 787.566 6.31624 5 good
chromium@385951 795.805 6.52794 5 good
chromium@385964 788.151 7.53684 5 good
chromium@385968 796.604 8.123 5 good
chromium@385970 802.717 13.2409 5 good
chromium@385971 761.252 11.6365 5 bad <--
chromium@385977 758.902 9.98567 5 bad
chromium@386003 763.253 6.41385 5 bad
chromium@386107 751.717 7.39558 5 bad
Bisect job ran on: winx64intel_perf_bisect
Bug ID: 602318
Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests dromaeo.domcoremodify
Test Metric: dom/dom
Relative Change: 4.55%
Score: 99.9
Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64intel_perf_bisect/builds/918
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9013064243558544832
Not what you expected? We'll investigate and get back to you!
https://chromeperf.appspot.com/bad_bisect?try_job_id=5802793655336960
| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
| X | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect. Thank you!
,
May 10 2016
DCHECK should be a no-op on official builds. What does the compiler output look like with ASSERT vs DCHECK? They should both be nothing.
,
May 10 2016
,
May 10 2016
speculatively reverting like esprehn suggests sgtm
,
May 11 2016
> DCHECK should be a no-op on official builds. What does the compiler output look like with ASSERT vs DCHECK? They should both be nothing. Yeah, there is no difference between ASSERT and DCHECK in a release build, as far as I can tell from objdump with --disassembly. Regarding reverting, we can not revert the CL cleanly because "git revert thisCL" caused a lot of conflicts. It will be yet another replace in a *reverse* order, "DCHECK -> ASSERT".
,
May 11 2016
===== BISECT JOB RESULTS =====
Status: completed
===== SUSPECTED CL(s) =====
Subject : ASSERT -> {DCHECK|DCHECK_XX}, ENABLE(ASSERT) -> DCHECK_IS_ON() in dom
Author : hayato
Commit description:
There are several places where DCHECK_{EQ/NE} can not be used because
"operator<<(ostream&, T&)" is not defined for T.
To isolate issues, other macros, such as RELEASE_ASSERT, will be replaced in another CL.
BUG= 596760 , 601669
Review URL: https://codereview.chromium.org/1854423002
Cr-Commit-Position: refs/heads/master@{#385971}
Commit : 63915eb8cbda0af155de9e774283fa0828e9f446
Date : Fri Apr 08 03:17:02 2016
===== TESTED REVISIONS =====
Revision Mean Std Dev N Good?
chromium@385899 812.567 10.5305 5 good
chromium@385951 806.789 10.2921 8 good
chromium@385964 799.05 8.81095 8 good
chromium@385968 810.831 20.3939 8 good
chromium@385970 818.311 16.151 5 good
chromium@385971 779.543 4.95094 5 bad <--
chromium@385977 778.564 11.4106 8 bad
chromium@386003 772.42 5.57596 5 bad
chromium@386107 755.495 12.2401 5 bad
Bisect job ran on: winx64nvidia_perf_bisect
Bug ID: 602318
Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests dromaeo.domcoremodify
Test Metric: dom/dom
Relative Change: 7.02%
Score: 99.5
Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64nvidia_perf_bisect/builds/1633
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9012983647171052368
Not what you expected? We'll investigate and get back to you!
https://chromeperf.appspot.com/bad_bisect?try_job_id=5850841991348224
| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
| X | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect. Thank you!
,
May 20 2016
I am not sure what I should do it for this regression. - As far as I confirmed via disassembly, DCHECK_XX would be no-op in a release build. - As far as I rechecked the CL, this CL did not anything other than changing ASSERT to DCHECK_XX. - I confirmed that the attempt to revert this CL would cause a lot of conflicts. Thus, we can not isolate the effects of CL cleanly for TOT, I am afraid. Given that only win bots would regress, I guess this patch is not the real cause of the regression, hopefully. Let me mark this WONTFIX.
,
May 23 2016
Adding yukishiino, bashi, haraken, owners of dromaeo perf test. Our perf bots and bisectors run release official builds, but it looks like somehow https://codereview.chromium.org/1854423002 has regressed dromaeo performance anyway. Do we care about regressions of this size? Is it okay to WontFix?
,
May 23 2016
I agree that the bisected cl can't possibly cause this -- maybe unless the perf bots build with dchecks enabled, but then their results wouldn't be useful for measuring perf, so I doubt that's the case (can you link to the bot config?). The most likely explanation is that the bisect bisected to the wrong change.
,
May 23 2016
Is DCHECK completely a no-op on official builds? If yes, I'm fine with marking this as WontFix. Otherwise, this may be a signal to indicate that we still have some unexpected regression between ASSERT/RELEASE_ASSERT and DCHECK/CHECK.
,
May 23 2016
> Is DCHECK completely a no-op on official builds? Not only in official builds, in normal release builds as well. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by rsch...@chromium.org
, Apr 11 2016