New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 708249 link

Starred by 1 user

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Discrepancy between values reported on perf waterfall and traces.

Project Member Reported by erikc...@chromium.org, Apr 4 2017

Issue description

Not sure what's going on, can someone from speed ops investigate?

ChromiumPerf/chromium-rel-mac-retina/memory.desktop / memory:chrome:all_processes:reported_by_chrome:malloc:effective_size_avg / TrivialBlinkingCursorPageSharedPageState

https://chromeperf.appspot.com/report?sid=da780d2ad605d0faedd6e10579e4dd4652d8395fd3b7f39de9ba83823a6c5173

Value: 22,843,200 (± 208,412)
Point ID: 461342
Accessing the trace directly shows 21.5 MB.

Value: 22,430,200 (± 223,874)
Point ID: 461346
Accessing the trace directly shows 21.4 MB.

The error bars make me think that there are actually multiple traces being taken, and we're seeing the avg? [hence effective_size_avg]. But how do I get access to these other traces?
 
Cc: eakuefner@chromium.org perezju@chromium.org benjhayden@chromium.org
I wonder whether this is a rounding thing.
The absolute difference in values is >400,000. Assuming rounding on both 21.5 and 21.4, the largest possible difference is 21.55 - 21.35, which is ~200,000.
pageset_repeat = 5
https://cs.corp.google.com/clankium/src/tools/perf/benchmarks/memory.py?q=memory.desktop&l=59

So, yep, tbmv1 summarization is averaging across 5 traces, and only giving you a link to one trace.
I think go/histogram-pipeline should put links to all of the traces in the diagnostics instead of that menu.
https://github.com/catapult-project/catapult/issues/2431

Also, AFAIR, the trace shows values in MiB (= 1024*1024 bytes). So the differences are not as large as you think:

Point ID 461342: 22,843,200 bytes - 21.5 MiB = 298,816 bytes
Point ID 461346: 22,430,200 bytes - 21.4 MiB =  -9,326 bytes

> But how do I get access to these other traces?

It's not pretty, but you can:

- go to "Buildbot stdio" -- oops, doesn't work go/catabug/3472
  - ok, go to "Buildbot status page" link instead,
  - search for the benchmark you were looking at, e.g. "memory.desktop"
  - click on the "stdout" link, e.g.: https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FMac_Retina_Perf%2F506%2F%2B%2Frecipes%2Fsteps%2Fmemory.desktop_on_Intel_GPU_on_Mac_on_Mac-10.11%2F0%2Fstdout
- now you have the log, near the end you'll see:


INFO:py_utils.cloud_storage:Uploading /b/s/w/itvRQFu6/tmpcbtA19.html to gs://chrome-telemetry-output/trace-file-id_0-2017-04-05_00-30-45-66948.html
View generated trace files online at https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_0-2017-04-05_00-30-45-66948.html for page file://trivial_sites/trivial_scrolling_page.html
INFO:py_utils.cloud_storage:Uploading /b/s/w/itvRQFu6/tmp_Kvh9P.html to gs://chrome-telemetry-output/trace-file-id_1-2017-04-05_00-30-47-20958.html
View generated trace files online at https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_1-2017-04-05_00-30-47-20958.html for page file://trivial_sites/trivial_blinking_cursor.html
INFO:py_utils.cloud_storage:Uploading /b/s/w/itvRQFu6/tmpDUwMEY.html to gs://chrome-telemetry-output/trace-file-id_2-2017-04-05_00-30-48-76465.html
View generated trace files online at https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_2-2017-04-05_00-30-48-76465.html for page file://trivial_sites/trivial_canvas.html
[...]

for each page and all of its repeats.

Cc: -eakuefner@chromium.org
Components: Test>Telemetry
Components: -Speed>Telemetry

Sign in to add a comment