New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 636420 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner: ----
Closed: Aug 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 636755



Sign in to add a comment

12.5% regression in system_health.memory_desktop at 410354:410387

Project Member Reported by petrcermak@chromium.org, Aug 10 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=636420

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgtuq3uAoM


Bot(s) for this bug's original alert(s):

chromium-rel-win7-dual
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Aug 10 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@410353  16384.0  0.0      12  good
chromium@410387  16384.0  0.0      8   bad

Bisect job ran on: win_perf_bisect
Bug ID: 636420

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop
Test Metric: browse_social-memory:chrome:all_processes:reported_by_chrome:v8:allocated_by_malloc:effective_size_avg/browse_social_twitter
Relative Change: 0.00%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6829
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004700642382146240


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5857330214731776

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Blockedon: 636755
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Aug 11 2016

Cc: erikc...@chromium.org
Owner: erikc...@chromium.org

=== Auto-CCing suspected CL author erikchen@chromium.org ===

Hi erikchen@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Revert of Reland #1: "base: Implement GetCurrentThreadPriority." (patchset #2 id:20001 of https://codereview.chromium.org/2215513002/ )
Author  : erikchen
Commit description:
  
Reason for revert:
Still causes startup regressions:

https://bugs.chromium.org/p/chromium/issues/detail?id=635464#c4

Original issue's description:
> Reland #1: "base: Implement GetCurrentThreadPriority."
>
> The Chrome thread priority is saved in the thread dictionary during
> SetCurrentThreadPriority(). This seemed a better approach than using
> thread_policy_get(), since there are 4 Chrome priorities which won't cleanly map
> to thread flavors.
>
> BUG=601270
>
> Committed: https://crrev.com/23d10a36f2e815edd610098a65e5890f279d2183
> Cr-Commit-Position: refs/heads/master@{#410163}

TBR=mark@chromium.org,gab@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=601270

Review-Url: https://codereview.chromium.org/2221083002
Cr-Commit-Position: refs/heads/master@{#410390}
Commit  : 23c669b5b4cf228dab6da2cc47293b4650a7b923
Date    : Mon Aug 08 17:55:57 2016


===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N  Good?
chromium@410386  12640256  0.0      5  good
chromium@410389  12640256  0.0      5  good
chromium@410390  13699232  0.0      5  bad    <--
chromium@410391  13699232  0.0      5  bad
chromium@410395  13699232  0.0      4  bad
chromium@410404  13699232  0.0      5  bad

Bisect job ran on: mac_10_10_perf_bisect
Bug ID: 636420

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop
Test Metric: load_tools-memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/load_tools_stackoverflow
Relative Change: 8.38%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2295
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004634535981878912


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6348294368788480

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: -erikc...@chromium.org sullivan@chromium.org
Components: Tests>Telemetry
Owner: ----
Status: Untriaged (was: Assigned)
Something is really broken with the measurement. Someone from the system health team needs to look into it.
Screen Shot 2016-08-11 at 2.08.14 PM.png
31.2 KB View Download
Owner: petrcermak@chromium.org
Petr, can you take a look? We are also seeing some 16384 values here like in  bug 636755 .
Cc: petrcermak@chromium.org
Owner: erikc...@chromium.org
Status: Assigned (was: Untriaged)
erikchen: What exactly do you mean when you say "something is really broken with the measurement"?

The metric on the screenshot in #7 is memory:chrome:all_processes:reported_by_chrome:v8:ALLOCATED_BY_MALLOC:effective_size_avg
whereas the bisected metric in #6 is memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg (screenshot attached)

The data on the DASHBOARD (https://chromeperf.appspot.com/group_report?bug_id=636420) matches the values in the traces (whose revisions according to their metadata match the range on the dashboard):
r410386: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-08_12-51-00-54722.html
r410404: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-08_15-19-09-68428.html

Similarly, the BISECT values in #6 match the values in the traces (https://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2295):
r410389:
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_08-14-54-12068.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_08-30-16-7639.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_08-45-53-37204.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_09-01-18-72715.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_09-16-50-52424.html
r410390:
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_09-34-11-23055.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_09-49-32-99357.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_10-04-57-62088.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_10-20-19-59964.html
https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_10-36-48-3395.html

Anyway, the difference between the dashboard traces and bisect traces seems to be exactly the same. Given this and the low standard deviation of the results in #6, I think it's safe to conclude that the regression was caused by 410390.

erikchen: Please investigate the issue and decide on further action.

sullivan: I think this might be just a coincidence. 16,384 B = 16 KiB (exactly), which is why it might be quite a common value (because allocations usually happen in chunks that are multiples of page size, which is typically 4 KiB).
Screenshot from 2016-08-12 11:41:41.png
102 KB View Download
Owner: ----
Status: Unconfirmed (was: Assigned)
I landed a CL, and shortly thereafter, reverted it. There should be no net change in performance. The graphs that you linked appear very noisy, as evidenced by both the graph I attached, and the graph you attached. I really have no idea what regression bisect is trying to point out. I also don't see how the revert of a CL (whose landing caused no change in the graphs) could be related to this regression(?).
#10: OK, this is a fair point and I agree with the noise argument. Thanks for looking into the issue!
Status: WontFix (was: Unconfirmed)
Labels: SystemHealth-Sheriff

Sign in to add a comment