INFO) 2018-12-03 07:17:30,061 trace_data.Serialize:192  Trace sizes in bytes: {'cpuSnapshots': 5044961L, 'traceEvents': 1010115312L, 'telemetry': 739778L}

(INFO) 2018-12-03 07:17:56,525 trace_data.Serialize:201  trace2html finished in 26.46 seconds.

(INFO) 2018-12-03 07:17:56,526 timeline_based_measurement._ComputeTimelineBasedMetrics:317  Starting to compute metrics on trace

(INFO) 2018-12-03 07:38:11,378 timeline_based_measurement._ComputeTimelineBasedMetrics:323  Processing resulting traces took 1214.853 seconds

(ERROR) 2018-12-03 07:38:11,378 page_test_results.Fail:587  Failure recorded: RangeError: Maximum call stack size exceeded

    at Function.findToplevelSchedulerTasks (/tracing/extras/chrome/event_finder_utils.html:221:13)

    at getTasks (/tracing/metrics/system_health/expected_queueing_time_metric.html:121:38)

    at addExpectedQueueingTimeMetric_ (/tracing/metrics/system_health/expected_queueing_time_metric.html:144:21)

    at new expectedQueueingTimeMetric (/tracing/metrics/system_health/expected_queueing_time_metric.html:86:5)

    at runMetrics (c:\b\s\w\ir\third_party\catapult\tracing\tracing\metrics\metric_map_function.html:61:16)

    at metricMapFunction (c:\b\s\w\ir\third_party\catapult\tracing\tracing\metrics\metric_map_function.html:190:24)

    at Object.mapSingleTrace (/tracing/mre/map_single_trace.html:39:7)

    at eval (c:\b\s\w\ir\third_party\catapult\tracing\tracing\mre\map_single_trace_cmdline.html:61:18)

    at Object.runAndConvertErrorsToFailures (/tracing/mre/map_single_trace.html:24:10)

    at mapSingleTraceWithResult (c:\b\s\w\ir\third_party\catapult\tracing\tracing\mre\map_single_trace_cmdline.html:52:12)



perezju: the traces here at 1GB, that seems a little high? :) 

multitab:misc:typical24 has a lot of tabs, so yeah, no wonder. :)

+vovoy and +cbruni who created the original story and did the recent :2108 refresh of it.

What are the alternatives here? Reduce the number of tabs? Can the size of the trace still be reduced?

I will check if the trace size can be reduced.

We could reduce the number of tabs, but that might be a very good stress test for the chrome itself.

Note that just processing these traces on the Telemetry side takes 20 minutes. If we can constrain these tests to collecting some smaller subset of trace data that would definitely help a lot.

An alternative could be to move it out of common_desktop and make a separate multitab_desktop test?

I think making a separate multitab_desktop test is a sane solution. multitab:misc:typical24 should require only the tracingMetric trace data.

As the solution may require changing system_health tests. I might not be the right owner. (You could assign it to me if you think I am the right owner.)

+Speed>Benchmarks and +nednguyen/+tdresser explicitly for queries on #6/#7 re adding a new benchmark.

I would rather reduce the number of tabs than moving it to a separate benchmark. IMO, multi-tab test cases is very important of Chrome browser & thus should be in system health benchmark

Ned, is there any way to have some stories collect different trace data than others? I guess probably there is not. In that case, there is going to be tension between configurability and http://bit.ly/why-merge-benchmarks.

I'm wondering why this just now started failing. It used to work, right?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5a14fdf87276e7460ccd4d18a5eba301514bd3ad

commit 5a14fdf87276e7460ccd4d18a5eba301514bd3ad
Author: Caleb Rouleau <crouleau@chromium.org>
Date: Fri Dec 07 01:32:40 2018

Disable system_health.common_desktop/multitab on Win 10

NOTRY=true

Bug: 911214
Change-Id: Ia853c50260f4a46835818ef4b364b4ee97091244
Reviewed-on: https://chromium-review.googlesource.com/c/1367109
Reviewed-by: Caleb Rouleau <crouleau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#614566}
[modify] https://crrev.com/5a14fdf87276e7460ccd4d18a5eba301514bd3ad/tools/perf/expectations.config

> I'm wondering why this just now started failing. It used to work, right?

I guess it's related to the switch to perfetto for tracing backend, which has caused an increase in trace size (and it particularly magnifies in this story). I believe oysteine has been working on addressing the trace size issue in general.

Could we switch back off of perfetto until it is ready to support this use case? I don't feel good about breaking peoples' tests. Perhaps someone can link to the design/launch/implementation bugs for perfetto?

Perfetto increased trace sizes by about 30% temporarily for traces without memory-infra dumps but is now back to normal as of  https://chromium-review.googlesource.com/c/chromium/src/+/1359304

Perfetto has been enabled for well over a month though, so it wouldn't be the cause of recent failures. If something else caused the trace sizes here to increase recently though, the CL above may have taken it below the failure threshold again so could be worth re-enabling the test to see?

Let's give that a try!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ec24b70caddcb99a897ab89a5bbf60b3ca621349

commit ec24b70caddcb99a897ab89a5bbf60b3ca621349
Author: Juan Antonio Navarro Pérez <perezju@chromium.org>
Date: Mon Dec 10 11:51:20 2018

Revert "Disable system_health.common_desktop/multitab on Win 10"

This reverts commit 5a14fdf87276e7460ccd4d18a5eba301514bd3ad.

Reason for revert: Root cause may have been fixed by:
https://chromium-review.googlesource.com/c/chromium/src/+/1359304

Original change's description:
> Disable system_health.common_desktop/multitab on Win 10
> 
> NOTRY=true
> 
> Bug: 911214
> Change-Id: Ia853c50260f4a46835818ef4b364b4ee97091244
> Reviewed-on: https://chromium-review.googlesource.com/c/1367109
> Reviewed-by: Caleb Rouleau <crouleau@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#614566}

TBR=sullivan@chromium.org,tdresser@chromium.org,charliea@chromium.org,crouleau@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

TBR=nednguyen@google.com
NOTRY=true
Bug: 911214

Change-Id: I0835b698da2cb3f2aec50d12305ef23ed9842295
Reviewed-on: https://chromium-review.googlesource.com/c/1369793
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#615092}
[modify] https://crrev.com/ec24b70caddcb99a897ab89a5bbf60b3ca621349/tools/perf/expectations.config

The NextAction date has arrived: 2018-12-11

Still failing:

(INFO) 2018-12-11 06:02:22,746 trace_data.Serialize:192  Trace sizes in bytes: {'cpuSnapshots': 6822051L, 'traceEvents': 754940138L, 'telemetry': 827447L}

(INFO) 2018-12-11 06:02:45,730 trace_data.Serialize:201  trace2html finished in 22.98 seconds.

(INFO) 2018-12-11 06:02:45,730 timeline_based_measurement._ComputeTimelineBasedMetrics:317  Starting to compute metrics on trace

(INFO) 2018-12-11 06:11:59,328 timeline_based_measurement._ComputeTimelineBasedMetrics:323  Processing resulting traces took 553.598 seconds

(ERROR) 2018-12-11 06:11:59,328 page_test_results.Fail:587  Failure recorded: RangeError: Maximum call stack size exceeded

    at Function.findToplevelSchedulerTasks (/tracing/extras/chrome/event_finder_utils.html:221:13)

    at getTasks (/tracing/metrics/system_health/expected_queueing_time_metric.html:121:38)

    at addExpectedQueueingTimeMetric_ (/tracing/metrics/system_health/expected_queueing_time_metric.html:144:21)

    at new expectedQueueingTimeMetric (/tracing/metrics/system_health/expected_queueing_time_metric.html:86:5)

    at runMetrics (c:\b\s\w\ir\third_party\catapult\tracing\tracing\metrics\metric_map_function.html:61:16)

    at metricMapFunction (c:\b\s\w\ir\third_party\catapult\tracing\tracing\metrics\metric_map_function.html:190:24)

    at Object.mapSingleTrace (/tracing/mre/map_single_trace.html:39:7)

    at eval (c:\b\s\w\ir\third_party\catapult\tracing\tracing\mre\map_single_trace_cmdline.html:61:18)

    at Object.runAndConvertErrorsToFailures (/tracing/mre/map_single_trace.html:24:10)

    at mapSingleTraceWithResult (c:\b\s\w\ir\third_party\catapult\tracing\tracing\mre\map_single_trace_cmdline.html:52:12)

(INFO) 2018-12-11 06:11:59,447 browser.Close:209  Closing browser (pid=2300) ...

(INFO) 2018-12-11 06:12:01,237 desktop_browser_backend._TryCooperativeShutdown:561  Successfully shut down browser cooperatively

(INFO) 2018-12-11 06:12:01,237 browser.Close:223  Browser is closed.

[  FAILED  ] system_health.common_desktop/multitab:misc:typical24@{'case': 'multitab', 'group': 'misc'} (798870 ms)

Too bad :( Let's disable again then.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d136542c0334c29b3531c6d9c400a394e8ec8e85

commit d136542c0334c29b3531c6d9c400a394e8ec8e85
Author: Caleb Rouleau <crouleau@chromium.org>
Date: Thu Dec 13 11:19:42 2018

Reland "Disable system_health.common_desktop/multitab on Win 10"

This is a reland of 5a14fdf87276e7460ccd4d18a5eba301514bd3ad

Original change's description:
> Disable system_health.common_desktop/multitab on Win 10
>
> NOTRY=true
>
> Bug: 911214
> Change-Id: Ia853c50260f4a46835818ef4b364b4ee97091244
> Reviewed-on: https://chromium-review.googlesource.com/c/1367109
> Reviewed-by: Caleb Rouleau <crouleau@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#614566}

TBR=nednguyen@google.com
NOTRY=true

Bug: 911214
Change-Id: I67dc1d3f3b1c797e23a580264726a1b9ffd1884a
Reviewed-on: https://chromium-review.googlesource.com/c/1375710
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#616268}
[modify] https://crrev.com/d136542c0334c29b3531c6d9c400a394e8ec8e85/tools/perf/expectations.config