New issue
Advanced search Search tips

Issue 761954 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Nov 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

8.1%-92.7% regression in speedometer at 477411:485714

Project Member Reported by hablich@chromium.org, Sep 5 2017

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=761954

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=69945fd8d3f52ffc2af718abb4f12280222f751b0395d585fa4d1de573a97884


Bot(s) for this bug's original alert(s):

linux-release

=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : speedometer
  Metric       : VanillaJS-TodoMVC/VanillaJS-TodoMVC


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests speedometer

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8969301399016416176


For feedback, file a bug with component Speed>Bisection
Cc: simonhatch@chromium.org
+simonhatch, looks like this is timing out after 24 hours, with a large bisect range. Is there any way to narrow the range?
FYI:These alerts were ignored before. I unignored them and started a bisect to understand why there was such a big regression.

Please note that the normal test stopped working after 477410 until 485714 (the alert) on all of the tests. This had no influence on the ref tests though. Seems like data is missing?
Cc: nedngu...@google.com
Ned, any ideas what caused the speedometer test to break on ToT but not ref?
That would be a bug in Chrome binary. I would say assign this bug to benchmark owner so they can reproduce & address.
re: #c5

Since it didn't get past the good/bad revisions, not that I can think of. Ideally I'd go into the recipe to figure out why it's spending all that time resolving hashes. 

=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : speedometer
  Metric       : BackboneJS-TodoMVC/BackboneJS-TodoMVC


To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests speedometer

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8969195612967660672


For feedback, file a bug with component Speed>Bisection
Owner: mvstan...@chromium.org
Status: Assigned (was: Untriaged)
Assigning to benchmark owner. mvstanton, any clue why in the discussed range Speedometer did not run with Chrome anymore?
Owner: jarin@chromium.org
Jaro, can you look at this one?

Comment 13 by jarin@chromium.org, Sep 18 2017

Cc: jarin@chromium.org
Owner: hablich@chromium.org
I am not quite sure what to look at here? As far as I can see, the regression in ref is pretty much the same as ToT, so no real regression happened in that range.

As for why the test did not run, I have no idea how to find out what is happening on infrastructure...

Assigning to @hablich in case he wants to assign to someone in infrastructure.
Status: WontFix (was: Assigned)
Components: Test>Telemetry
Components: -Speed>Telemetry

Sign in to add a comment