New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 733700 link

Starred by 1 user
Status: WontFix
Owner: ----
Closed: Jul 2017
Cc:
█ ke...@intel.com
charliea@chromium.org
nedngu...@google.com
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

26.5%-464.4% regression in system_health.common_desktop at 478590:479270

Project Member Reported by majidvp@chromium.org, Jun 15 2017 
See the link to graphs below.

Comment 3 by charliea@chromium.org, Jun 15 2017 

Hi Majid: any context for why this was assigned to me?

Comment 4 by majidvp@chromium.org, Jun 15 2017

Cc: charliea@chromium.org
Owner: ----
Status: Untriaged (was: Assigned)
Ops! I think I put your name in "Component Owner:"** field when creating the bug in perf dashboard. I thought that will cc the benchmark owner (you are listed as owner of system_health.common_desktop) but I should have just added you to cc field.

** It is actually not "Component Owner" field rather two field "Component" and "Owner" but the UI is confusing.

Comment 5 by charliea@chromium.org, Jun 15 2017 

Ah, got it! Just wanted to make sure that there wasn't a regression that I didn't know I caused.
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Jun 15 2017

Cc: ke...@intel.com
Owner: ke...@intel.com

=== Auto-CCing suspected CL author ke.he@intel.com ===

Hi ke.he@intel.com, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Ke He
  Commit : 15092e9195954cbc331cd58e344d0895fe03d0cd
  Date   : Wed Jun 14 03:09:47 2017
  Subject: Rewrite the generic sensor browsertest.

Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : system_health.common_desktop
  Metric       : timeToFirstContentfulPaint_avg/load_search/load_search_taobao
  Change       : 15.25% | 1010.01757143 -> 1164.03964286

Revision             Result                  N
chromium@479147      1010.02 +- 1535.41      14      good
chromium@479209      699.332 +- 1282.01      6       good
chromium@479240      383.617 +- 917.038      6       good
chromium@479255      649.186 +- 1896.26      14      good
chromium@479262      788.828 +- 1897.91      14      good
chromium@479268      880.727 +- 2235.29      21      good
chromium@479269      886.511 +- 1150.91      6       good
chromium@479270      1164.04 +- 979.746      14      bad       <--

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.taobao system_health.common_desktop

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976701844344583072

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6429432476073984


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 7 by ke...@intel.com, Jun 16 2017 

Hi, charliea, my CL only changes the browsertest, I don't think it could affect the benchmark test?

Comment 9 by sullivan@chromium.org, Jun 16 2017

Cc: nedngu...@google.com
Owner: ----
Agreed with ke.he that the CL is unlikely to be responsible. Kicked off another bisect.

Charlie, this one might be hard to track down because it looks like the benchmark suddenly got bimodal. cc-ing Ned too as FYI
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Jun 16 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : system_health.common_desktop
  Metric       : timeToFirstContentfulPaint_avg/load_search/load_search_taobao

Revision             Result                  N
chromium@479147      650.462 +- 2428.27      21      good
chromium@479270      451.031 +- 2034.32      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.taobao system_health.common_desktop

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976667762720053088

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5841236436451328


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Jun 16 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : system_health.common_desktop
  Metric       : timeToFirstContentfulPaint_avg/load_search/load_search_taobao

Revision             Result                  N
chromium@479147      992.418 +- 2016.81      21      good
chromium@479270      741.198 +- 2334.47      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.taobao system_health.common_desktop

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976664605363769040

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5841236436451328


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Jun 16 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : system_health.common_desktop
  Metric       : timeToFirstContentfulPaint_avg/load_search/load_search_taobao

Revision             Result                  N
chromium@479120      806.011 +- 2265.62      21      good
chromium@479270      803.524 +- 2310.14      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.taobao system_health.common_desktop

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976638089488623392

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4898103762616320


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 16 by 42576172...@developer.gserviceaccount.com, Jun 16 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: mac_pro_perf_bisect
  Benchmark    : system_health.common_desktop
  Metric       : timeToFirstContentfulPaint_avg/load_search/load_search_taobao

Revision             Result                  N
chromium@479120      463.486 +- 2019.94      21      good
chromium@479270      402.567 +- 1878.49      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.taobao system_health.common_desktop

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976634406322212944

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4898103762616320


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 17 by toyoshim@chromium.org, Jun 21 2017 

In the dashboard, I removed some test cases that showed a temporary regression that recovered later, because the page says "WARNING: Non-overlapping alerts are grouped together".

The removed test alerts are marked as "ignore".

Comment 19 by toyoshim@chromium.org, Jun 21 2017 

kick another bisect for the remaining test with wider revision range.
Project Member

Comment 20 by 42576172...@developer.gserviceaccount.com, Jun 21 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : system_health.common_mobile
  Metric       : timeToFirstContentfulPaint_avg/browse_news/browse_news_cnn


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.cnn system_health.common_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976197073188245888

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5566212685168640


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 22 by 42576172...@developer.gserviceaccount.com, Jun 21 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : system_health.common_mobile
  Metric       : timeToFirstContentfulPaint_avg/browse_news/browse_news_cnn


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.cnn system_health.common_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976192744352798880

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5566212685168640


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 23 by toyoshim@chromium.org, Jul 4 2017

Status: WontFix (was: Untriaged)
►

Sign in to add a comment