New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 821521 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

v8.browsing benchmarks failing on all platforms due to metric computation error

Project Member Reported by charliea@chromium.org, Mar 13 2018

Issue description

I'm almost certain that the failure was introduced in the huge catapult roll here: https://chromium.googlesource.com/chromium/src/+/a08dc477b554bb564bc861567a54c1f88ced57a2: going to kick off a bisect and see if it can turn something up.
 
Note to self: that means the first bad revision is 	a08dc477b554bb564bc861567a54c1f88ced57a2 and the last good revision is ab595088ec2dd55b76cae534c86af4769a687128.
Summary: v8.browsing benchmarks failing on all platforms due to metric computation error (was: v8.browsing_mobile failing due to metric computation)
Cc: dproy@chromium.org
The check that's firing was introduced in https://chromium-review.googlesource.com/c/catapult/+/939481/10/tracing/tracing/metrics/system_health/breakdown_tree_helpers.html: going to go ahead and revert this CL.
Components: Speed>Benchmarks>Waterfall
Owner: charliea@chromium.org
I checked with Deep, and he's alright with the revert.
An example failing stack trace is 

Failure recorded: Error: Breakdown cannot be negative.
    at addIdleAndBlockByNetworkBreakdown_ (/tracing/metrics/system_health/breakdown_tree_helpers.html:215:13)
    at Object.generateWallClockTimeBreakdownTree (/tracing/metrics/system_health/breakdown_tree_helpers.html:252:5)
    at collectTimeToEvent (/tracing/metrics/system_health/loading_metric.html:151:43)
    at Object.collectLoadingMetricsForRenderer (/tracing/metrics/system_health/loading_metric.html:277:31)
    at Object.getInteractiveTimestamps (/tracing/extras/chrome/estimated_input_latency.html:45:37)
    at new expectedQueueingTimeMetric (/tracing/metrics/system_health/expected_queueing_time_metric.html:84:21)
    at runMetrics (/b/swarming/w/ir/third_party/catapult/tracing/tracing/metrics/metric_map_function.html:47:16)
    at metricMapFunction (/b/swarming/w/ir/third_party/catapult/tracing/tracing/metrics/metric_map_function.html:150:24)
    at Object.mapSingleTrace (/tracing/mre/map_single_trace.html:39:7)
    at eval (/b/swarming/w/ir/third_party/catapult/tracing/tracing/mre/map_single_trace_cmdline.html:61:18)

The trace that triggered this error is ttps://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/browse_shopping_amazon_2018-03-13_06-33-19_92716.html
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Mar 13 2018

Cc: perezju@chromium.org charliea@chromium.org catapult...@skia-buildbots.google.com.iam.gserviceaccount.com
Status: Assigned
📍 Found significant differences after each of 2 commits.
https://pinpoint-dot-chromeperf.appspot.com/job/1484240e440000

[Soundwave] Separate bug id from alert status by perezju@chromium.org
https://chromium.googlesource.com/catapult/+/20f7f3364b73da7df7ffdf74536fc40e40ca3164

Roll src/third_party/catapult/ 406b235a9..6fbfa7cb2 (29 commits) by catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com
https://chromium.googlesource.com/chromium/src/+/a08dc477b554bb564bc861567a54c1f88ced57a2

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

Comment 8 by dtu@chromium.org, Mar 14 2018

Sorry, I think I pushed a bad config change to Pinpoint. Rerunning now.
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Mar 14 2018

Cc: benjhayden@chromium.org fmea...@chromium.org maxlg@chromium.org
Owner: dproy@chromium.org
📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/1003551e440000

More precise self time calculation by dproy@chromium.org
https://chromium.googlesource.com/catapult/+/2210f05b456f5a475229ef9e11ad45461e3136c9

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions
Project Member

Comment 11 by bugdroid1@chromium.org, Mar 23 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6aea1cac122502499713bfc182d0454d0f325fa9

commit 6aea1cac122502499713bfc182d0454d0f325fa9
Author: catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri Mar 23 14:48:06 2018

Roll src/third_party/catapult/ 734f737c6..c4e9b1332 (14 commits)

https://chromium.googlesource.com/catapult.git/+log/734f737c6b57..c4e9b1332818

$ git log 734f737c6..c4e9b1332 --date=short --no-merges --format='%ad %ae %s'
2018-03-23 nednguyen Revert "Add --enable-automation flag in GetFromBrowserOptions()"
2018-03-22 dtu [pinpoint] Set auto_explore == True for "patch jobs".
2018-03-21 ynovikov Remove a note on android_optional_gpu_tests_rel from manual rolls doc.
2018-03-15 dproy Reland "More precise self time calculation"
2018-03-21 nednguyen Ignore fetchts file
2018-03-20 laszio [Telemetry] Support outputing json in benchmark_runner
2018-03-19 kjharland [Dashboard] Whitelist prod Fuchsia Garnet builder.
2018-03-20 benjhayden Fix sorting results.html
2018-03-19 horo Add --enable-automation flag in GetFromBrowserOptions()
2018-03-19 eakuefner [Dashboard] Avoid creating empty rows/histograms/tests if histograms are empty
2018-03-16 dtu [pinpoint] Infer "repository" parameter from configuration.
2018-03-19 chiniforooshan First set of thread times metrics in TBMv2
2018-03-16 erikchen Replace --enable-heap-profiling with --memlog equivalent.
2018-03-19 wangxianzhu Fix visual rect support for cc::DisplayItemList

Created with:
  roll-dep src/third_party/catapult
BUG=chromium:822258, chromium:786572 , chromium:731979 ,chromium:821531,chromium:821521, chromium:819969 ,chromium:822258, chromium:627461 , chromium:822843 


The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=sullivan@chromium.org

Change-Id: I8c0025df6198bf9b1662bb8d78e45741d66f6aff
Reviewed-on: https://chromium-review.googlesource.com/977884
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#545441}
[modify] https://crrev.com/6aea1cac122502499713bfc182d0454d0f325fa9/DEPS

Sign in to add a comment