+erikchen Do we look at memory.desktop anymore?

(I know on the mobile side we're trying to remove memory.mobile side but are blocked by Android system health).

On the dashboard it looks like there is no data for memory.desktop Linux at all:
https://chromeperf.appspot.com/report?sid=99f9ceeb5eb254b901b1f502195103920508dc76a0c7d84b0959c5c6cece5bf3
Maybe we should stop running it?

There is also no data for system_health.memory_desktop on Linux for since the 18th: https://chromeperf.appspot.com/report?sid=39de33f20e8811f741c97f26740a6ba2d2bdb33a0b03342a0958a0adc592fa6f

As a stop gap maybe we can turn off heap-profiling=native for Linux until OOP heap profiling lands?

+1 to turn off heap profiling for now.

Thanks, I'll turn off heap profiling with https://chromium-review.googlesource.com/c/chromium/src/+/628216 and kick off a return code bisect.

Bisect: https://chromeperf.appspot.com/buildbucket_job_status/8970455036475667584

Thanks for following up. Yes, there's something wrong for Linux + native heap profiling. Exact details uncertain, although I have suspicions about the allocation register sharding parameter...either way, thanks for turning it off for now.

Why was this turned off for all system_health benchmarks if the failure was on memory.desktop on Linux? It provides a much more accurate measurement in the form of shim_allocated_objects_size.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/be262cd3ee73975ced4e339d630b18f3e7e4145b

commit be262cd3ee73975ced4e339d630b18f3e7e4145b
Author: Sami Kyostila <skyostil@chromium.org>
Date: Wed Aug 23 18:53:03 2017

perf: Disable native heap profiling in memory benchmarks

BUG= 757847 

Change-Id: I6c04925f3515231be78b49e4f97322eb6f66671a
Reviewed-on: https://chromium-review.googlesource.com/628216
Commit-Queue: Sami Kyöstilä <skyostil@chromium.org>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#496751}
[modify] https://crrev.com/be262cd3ee73975ced4e339d630b18f3e7e4145b/tools/perf/benchmarks/memory.py
[modify] https://crrev.com/be262cd3ee73975ced4e339d630b18f3e7e4145b/tools/perf/benchmarks/system_health.py

+ oysteine, please see c#5.

#5: Sorry for not mentioning it here before but system_health.memory_desktop was also failing in a very similar way to memory.desktop so I turned heap profiling off there too:

https://sheriff-o-matic.appspot.com/chromium.perf/examine/chromium.perf.Linux%20Perf.system_health.memory_desktop

I think Erik's comment was more about the Win/Mac/Linux distinction than the memory.desktop/system_health.memory_desktop distinction.
My fault, I wrote c#1 like the problem was only on Linux but actually it looks like the problem affects all platforms and most bots on both memory.desktop and system_health.memory_desktop.

Of the 15 bots on system_health.memory_desktop it looks like 10 or so are missing data between the 17th and the 24th (when Sami landed the CL to turn off native heap profiling in memory benchmarks) ~2 are missing data for longer and haven't come back (e.g. chromium-rel-win7-gpu-ati) and ~2 are fine (e.g. chromium-rel-mac11-air) (I lost count towards the end).

See the graphs here https://chromeperf.appspot.com/report?sid=bbb7a123a9547c7931bd1e082a69422f47f48cdf55f0111b16500fed1220f854
It's kind of hard to see which are missing data but if you roll over the points at the end of the graph you should be able to see.

Comparing the Linux build linked above and a random Mac build from mac_retina_perf which failed in the same period it seems like they have similar failure modes:

Linux: https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FLinux_Perf%2F1695%2F%2B%2Frecipes%2Fsteps%2Fsystem_health.memory_desktop_on__102b__GPU_on_Linux%2F0%2Fstdout
Mac: https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FMac_Retina_Perf%2F1033%2F%2B%2Frecipes%2Fsteps%2Fsystem_health.memory_desktop_on_Intel_GPU_on_Mac_on_Mac-10.11%2F0%2Fstdout

got it. We should have a faster replacement ready soon, so lgtm for now.

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8969924756870852592

=== BISECT JOB RESULTS ===
NO Test failure found

Bisect Details
  Configuration: linux_perf_bisect
  Benchmark    : system_health.memory_desktop
  Metric       : benchmark_duration/benchmark_duration

Revision             Exit Code      N
chromium@493996      1 +- N/A       1      good
chromium@496474      1 +- N/A       1      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests system_health.memory_desktop

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8969924756870852592


For feedback, file a bug with component Speed>Bisection