New issue
Advanced search Search tips

Issue 633232 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: May 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

2.9% regression in system_health.memory_mobile at 407993:408034

Project Member Reported by petrcermak@chromium.org, Aug 1 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=633232

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgxqrEtQoM


Bot(s) for this bug's original alert(s):

android-nexus6
Cc: robliao@chromium.org
Owner: robliao@chromium.org

=== Auto-CCing suspected CL author robliao@chromium.org ===

Hi robliao@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Revert of Add virtual layout test for GPU accelerated decoding.  (patchset #8 id:140001 of https://codereview.chromium.org/1644763002/ )
Author  : robliao
Commit description:
  
Reason for revert:
Suspected in breaking webkit tests:
https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Linux%20MSAN/builds/11324

unexpected_failures:
virtual/media-gpu-accelerated/http/tests/media/media-source/mediasource-sourcebufferlist.html
virtual/media-gpu-accelerated/http/tests/media/media-source/mediasource-appendstream-quota-exceeded.html
virtual/media-gpu-accelerated/http/tests/media/media-source/mediasource-getvideoplaybackquality.html
and more in mediasource-*.html

Original issue's description:
> Add gpu accelerated config changes to SmokeTests.
>
> SmokeTests are the subset of layout tests run on the android bots. This change
> adds coverage to several historically problematic config-change scenarios.
>
> These tests use a virtual suite that sets the flag "--use-gpu-in-tests". This
> flag overrides the default test behavior of using MesaGL, which is not capable
> of creating surfaces to use with gpu accelerated decoding (at least on Android).
>
> This change also fixes some test/expectations rot.
>
> BUG= 555703 
> TESTS=Config change LayoutTests.
>
> Committed: https://crrev.com/b8bf5eba6b6cd3623f16404c746251bf662d141d
> Cr-Commit-Position: refs/heads/master@{#407938}

TBR=jochen@chromium.org,dpranke@chromium.org,wolenetz@chromium.org,chcunningham@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 555703 

Review-Url: https://codereview.chromium.org/2184003002
Cr-Commit-Position: refs/heads/master@{#408010}
Commit  : a7a23c40bb2e135ea185f171d201232b114afca0
Date    : Wed Jul 27 01:35:07 2016


===== TESTED REVISIONS =====
Revision         Mean    Std Dev  N  Good?
chromium@407992  925696  0.0      5  good
chromium@408003  925696  0.0      5  good
chromium@408008  925696  0.0      5  good
chromium@408009  926515  1831.79  5  good
chromium@408010  929792  0.0      5  bad    <--
chromium@408011  930611  1831.79  5  bad
chromium@408013  929792  0.0      5  bad
chromium@408034  929792  0.0      5  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 633232

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: load_tools-memory:chrome:all_processes:reported_by_os:system_memory:ashmem:private_dirty_size_avg/load_tools_stackoverflow
Relative Change: 0.44%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2395
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005518544591659936


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5801629018750976

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: -robliao@chromium.org
Owner: petrcermak@chromium.org
Just sheriffing that day. The bisect is pointing to a revert.
Owner: chcunningham@chromium.org
Assigning to the original patch author (just an FYI, feel free to close).
The CL that was reverted simply adds new android layout test coverage. The CL, reverted or not (its since relanded), definitely shouldn't have any lasting impact on Android memory. Perhaps running these tests triggers garbage collection to bring memory down?

I'll kick off the bisect again. Petrcermak@, is this ok to close because its resolved itself? I'm not sure if it has. 
Owner: petrcermak@chromium.org
chcunningham: Thanks for kicking off another bisect. It's possibly a different CL in the range then.

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Revert of Add virtual layout test for GPU accelerated decoding.  (patchset #8 id:140001 of https://codereview.chromium.org/1644763002/ )
Author  : robliao
Commit description:
  
Reason for revert:
Suspected in breaking webkit tests:
https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Linux%20MSAN/builds/11324

unexpected_failures:
virtual/media-gpu-accelerated/http/tests/media/media-source/mediasource-sourcebufferlist.html
virtual/media-gpu-accelerated/http/tests/media/media-source/mediasource-appendstream-quota-exceeded.html
virtual/media-gpu-accelerated/http/tests/media/media-source/mediasource-getvideoplaybackquality.html
and more in mediasource-*.html

Original issue's description:
> Add gpu accelerated config changes to SmokeTests.
>
> SmokeTests are the subset of layout tests run on the android bots. This change
> adds coverage to several historically problematic config-change scenarios.
>
> These tests use a virtual suite that sets the flag "--use-gpu-in-tests". This
> flag overrides the default test behavior of using MesaGL, which is not capable
> of creating surfaces to use with gpu accelerated decoding (at least on Android).
>
> This change also fixes some test/expectations rot.
>
> BUG= 555703 
> TESTS=Config change LayoutTests.
>
> Committed: https://crrev.com/b8bf5eba6b6cd3623f16404c746251bf662d141d
> Cr-Commit-Position: refs/heads/master@{#407938}

TBR=jochen@chromium.org,dpranke@chromium.org,wolenetz@chromium.org,chcunningham@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 555703 

Review-Url: https://codereview.chromium.org/2184003002
Cr-Commit-Position: refs/heads/master@{#408010}
Commit  : a7a23c40bb2e135ea185f171d201232b114afca0
Date    : Wed Jul 27 01:35:07 2016


===== TESTED REVISIONS =====
Revision         Mean    Std Dev  N  Good?
chromium@407992  925696  0.0      5  good
chromium@408003  925696  0.0      5  good
chromium@408008  925696  0.0      8  good
chromium@408009  927061  2115.17  6  good
chromium@408010  929792  0.0      8  bad    <--
chromium@408011  929792  0.0      5  bad
chromium@408013  930611  1831.79  5  bad
chromium@408034  929792  0.0      5  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 633232

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: load_tools-memory:chrome:all_processes:reported_by_os:system_memory:ashmem:private_dirty_size_avg/load_tools_stackoverflow
Relative Change: 0.44%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2406
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005410725524994704


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5210052302995456

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: ikilpatrick@chromium.org
Owner: ikilpatrick@chromium.org

=== Auto-CCing suspected CL author ikilpatrick@chromium.org ===

Hi ikilpatrick@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [worklets] Make WorkerThread handle both Worker and Worklet global scopes.
Author  : ikilpatrick
Commit description:
  
This is to allow the WorkerThread infrastructure be used for ThreadedWorkletGlobalScope (see WIP here: https://codereview.chromium.org/2171973002/)

This class could also be renamed to something like ThreadScriptEnvironment or ThreadContextEnvironment, but can also be done later in just a rename patch.

BUG= 567358 

Review-Url: https://codereview.chromium.org/2116113002
Cr-Commit-Position: refs/heads/master@{#407911}
Commit  : 9639129aab764e7597ab62c98dc27605d0a0b668
Date    : Tue Jul 26 21:01:54 2016


===== TESTED REVISIONS =====
Revision         Mean    Std Dev  N  Good?
chromium@407909  901120  0.0      5  good
chromium@407910  901120  0.0      5  good
chromium@407911  913408  0.0      3  bad    <--
chromium@407912  914227  1831.79  5  bad
chromium@407915  913408  0.0      5  bad
chromium@407920  917504  0.0      4  bad
chromium@407930  917504  0.0      5  bad
chromium@407950  925696  0.0      5  bad
chromium@407990  925696  0.0      5  bad
chromium@408071  929792  0.0      5  bad
chromium@408233  946176  0.0      5  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 633232

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: load_tools-memory:chrome:all_processes:reported_by_os:system_memory:ashmem:private_dirty_size_avg/load_tools_stackoverflow
Relative Change: 5.00%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2411
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005329255595728832


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6705734666944512

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Hey, sorry was sick at the end of last week. Above patch shouldn't have any effect of android memory either; is basically just a rename of a method + making some methods pure virtual. Will kick off another bisect.

===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean    Std Dev  N  Good?
chromium@407992  926208  1448.15  8  good
chromium@408034  928882  2730.67  9  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 633232

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: load_tools-memory:chrome:all_processes:reported_by_os:system_memory:ashmem:private_dirty_size_avg/load_tools_stackoverflow
Relative Change: 0.22%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2438
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004893748889091088


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5842567942373376

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
It seems that there are multiple regressions in the whole range:

chromium@407909  901120  0.0      5  good
chromium@407910  901120  0.0      5  good
chromium@407911  913408  0.0      3  bad    <--- REGRESSION 1 (+819)
chromium@407912  914227  1831.79  5  bad
chromium@407915  913408  0.0      5  bad
chromium@407920  917504  0.0      4  bad    <--- REGRESSION 2 (+4,096)
chromium@407930  917504  0.0      5  bad
chromium@407950  925696  0.0      5  bad    <--- REGRESSION 3 (+8,192)
chromium@407990  925696  0.0      5  bad
chromium@408071  929792  0.0      5  bad
chromium@408233  946176  0.0      5  bad    <--- REGRESSION 4 (+16,384)

Let's do each of them separately.

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [worklets] Make WorkerThread handle both Worker and Worklet global scopes.
Author  : ikilpatrick
Commit description:
  
This is to allow the WorkerThread infrastructure be used for ThreadedWorkletGlobalScope (see WIP here: https://codereview.chromium.org/2171973002/)

This class could also be renamed to something like ThreadScriptEnvironment or ThreadContextEnvironment, but can also be done later in just a rename patch.

BUG= 567358 

Review-Url: https://codereview.chromium.org/2116113002
Cr-Commit-Position: refs/heads/master@{#407911}
Commit  : 9639129aab764e7597ab62c98dc27605d0a0b668
Date    : Tue Jul 26 21:01:54 2016


===== TESTED REVISIONS =====
Revision         Mean    Std Dev  N  Good?
chromium@407909  901120  0.0      5  good
chromium@407910  901120  0.0      5  good
chromium@407911  913408  0.0      5  bad    <--
chromium@407912  913408  0.0      5  bad
chromium@407915  913408  0.0      5  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 633232

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: load_tools-memory:chrome:all_processes:reported_by_os:system_memory:ashmem:private_dirty_size_avg/load_tools_stackoverflow
Relative Change: 1.36%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2440
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004807107115630624


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5818152663908352

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Perf sheriff ping: reminder to follow up on possible performance issues
Labels: SystemHealth-Sheriff
Labels: -Performance-Sheriff
Labels: Performance
Status: WontFix (was: Assigned)
This regression has been included in a stable release and that stable channel is now deprecated. I'm closing this so that we won't have unresponsive performance regressions in the future. 

Sign in to add a comment