Issue 667470 link

Starred by 2 users

Issue metadata

Status:	Fixed
Owner:	█ rnep...@chromium.org Last visit > 30 days ago
Closed:	Apr 2017
Cc:	█ mikec...@chromium.org agrieve@chromium.org bpastene@chromium.org nedngu...@google.com █ rnep...@chromium.org charliea@google.com jbudorick@chromium.org lizeb@chromium.org
Components:	Tests>Telemetry
EstimatedDays:	----
NextAction:	----
OS:	Android
Pri:	2
Type:	Bug-Regression
Performance-Sheriff-BotHealth

Blocking:
issue 663748

Simultaneous failure across all device shards seen on nexus 7 devices

Project Member Reported by charliea@chromium.org, Nov 21 2016

Issue description

Revision range first seen: ?????? (before 10/25)
Link to failing step log: https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%282%29/builds/3319/steps/start_with_url.cold.startup_pages.reference/logs/stdio

This seems to be happening across all Chrome devices.

Excerpt from log:

Traceback (most recent call last):
  File "/b/rr/tmpkpY_Zi/w/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 89, in _RunStoryAndProcessErrorIfNeeded
    test.Measure(state.platform, results)
  File "/b/rr/tmpkpY_Zi/w/src/third_party/catapult/telemetry/telemetry/web_perf/timeline_based_measurement.py", line 284, in Measure
    trace_result = platform.tracing_controller.StopTracing()
  File "/b/rr/tmpkpY_Zi/w/src/third_party/catapult/telemetry/telemetry/core/tracing_controller.py", line 47, in StopTracing
    return self._tracing_controller_backend.StopTracing()
  File "/b/rr/tmpkpY_Zi/w/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py", line 108, in StopTracing
    self._IssueClockSyncMarker()
  File "/b/rr/tmpkpY_Zi/w/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py", line 203, in _IssueClockSyncMarker
    self._RecordIssuerClockSyncMarker)
  File "/b/rr/tmpkpY_Zi/w/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/chrome_tracing_agent.py", line 175, in RecordClockSyncMarker
    raise ChromeClockSyncError('Cannot issue clock sync. No devtools clients')
ChromeClockSyncError: Cannot issue clock sync. No devtools clients

This indicates that Chrome is crashing.

The error always seems to be happening on the kapook.com story, so I'm going to go ahead and disable story that for now.

gabadie@, could you please fix this or remove kabook.com if you're no longer interested in this story?

If the test is disabled, please downgrade to Pri-2.

Project Member

Comment 1 by bugdroid1@chromium.org, Nov 21 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d9c54bfd7faa1c00edfab4ea09e39fb4e4d34f66

commit d9c54bfd7faa1c00edfab4ea09e39fb4e4d34f66
Author: charliea <charliea@chromium.org>
Date: Mon Nov 21 23:20:13 2016

Temporarily remove kapook.com from startup_pages page set

The site is crashing on all Android devices and has been since 10/25.

TBR=gabadie@chromium.org,nednguyen@google.com
BUG= 667470 

Review-Url: https://codereview.chromium.org/2521783002
Cr-Commit-Position: refs/heads/master@{#433687}

[modify] https://crrev.com/d9c54bfd7faa1c00edfab4ea09e39fb4e4d34f66/tools/perf/page_sets/startup_pages.py
[modify] https://crrev.com/d9c54bfd7faa1c00edfab4ea09e39fb4e4d34f66/tools/perf/page_sets/startup_pages_record.py

Comment 2 by charliea@chromium.org, Dec 6 2016

It looks like bbc.co.uk is also pretty reliably failing (~75% of the time) on Nexus 9.

Comment 3 by charliea@chromium.org, Dec 6 2016

Owner: pasko@chromium.org

Assigning this to pasko@.

Comment 4 by charliea@chromium.org, Dec 6 2016

Status: Assigned (was: Untriaged)

bbc.co.uk also appears to be failing flakily on Nexus 7v2, but less reliably than on Nexus 9.

Project Member

Comment 5 by bugdroid1@chromium.org, Dec 6 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/29c6b51c2399248a62c4fd184c5e3bd0e8c91814

commit 29c6b51c2399248a62c4fd184c5e3bd0e8c91814
Author: charliea <charliea@chromium.org>
Date: Tue Dec 06 20:46:40 2016

Disable startup_with_url.cold.startup_pages on Nexus 7v2 and Nexus 9

R=sullivan@chromium.org
CC=pasko@chromium.org,nednguyen@google.com,rnephew@chromium.org
BUG= 667470 

Review-Url: https://codereview.chromium.org/2558473002
Cr-Commit-Position: refs/heads/master@{#436709}

[modify] https://crrev.com/29c6b51c2399248a62c4fd184c5e3bd0e8c91814/tools/perf/benchmarks/start_with_url.py

Comment 6 by pasko@chromium.org, Dec 9 2016

Blockedon: 663748
Cc: lizeb@chromium.org rnep...@chromium.org agrieve@chromium.org nednguyen@chromium.org
Labels: -Pri-1 Pri-2

I was OOO.

charliea: are you sure there was a browser crash? Seems like just a renderer/devtools crash.

Link to the build (since link to the log expired): https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%282%29/builds/3319

Looking at symbolized logcats of the build: https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus7v2_Perf__2_%2F3319%2F%2B%2Frecipes%2Fsteps%2Fstack_tool_with_logcat_dump%2F0%2Fstdout

All 'crashes' I can see would look like teh browser being killed from outside:
Stack Trace:
  RELADDR   FUNCTION                                                      FILE:LINE
  00021f90  tgkill+12                                                     /system/lib/libc.so
  00012fe1  pthread_kill+48                                               /system/lib/libc.so
  000131f5  raise+10                                                      /system/lib/libc.so
  00011f2b  <unknown>                                                     /system/lib/libc.so
  00021844  abort+4                                                       /system/lib/libc.so
  00045cc7  dvmAbort+78                                                   /system/lib/libdvm.so
  00001fab  jniRegisterNativeMethods+42                                   /system/lib/libnativehelper.so
  0004cd25  <unknown>                                                     /system/lib/libandroid_runtime.so
  0004d011  android::AndroidRuntime::startReg(_JNIEnv*)+24                /system/lib/libandroid_runtime.so
  0004da61  android::AndroidRuntime::start(char const*, char const*)+184  /system/lib/libandroid_runtime.so

rnephew: symptoms look similar to those in  Issue 663748 . There was some recent ADB version change, could it be releated?
Another similar:  Issue 667489 .
nednguyen: can you remember a recent catapult change that might be related to devtools lifetime?

Comment 7 by rnep...@chromium.org, Dec 9 2016

Cc: jbudorick@chromium.org

Its very possible it is that. I know this CL is fixing one place where we are using the wrong adb:
https://codereview.chromium.org/2544183002/

I dont think that will fix this if thats the issue though. Interestingly multiple test shard have one test failing in the same way. It would be nice if we could generate a trace file to see when tests are running to see if they are running concurrently... I'll talk to John about adding that to the test runner. I definitely think this could be an adb issue since its happening across multiple devices.

I think this is the interesting error:
[1208/093957:ERROR:host_forwarder_main.cc(477)] ERROR: could not unmap port.
ERROR: Existing controllers:


It has a new failure mode now as well:

Traceback (most recent call last):
  File "/b/rr/tmpWmXUqT/w/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 89, in _RunStoryAndProcessErrorIfNeeded
    test.Measure(state.platform, results)
  File "/b/rr/tmpWmXUqT/w/src/third_party/catapult/telemetry/telemetry/web_perf/timeline_based_measurement.py", line 290, in Measure
    trace_result = platform.tracing_controller.StopTracing()
  File "/b/rr/tmpWmXUqT/w/src/third_party/catapult/telemetry/telemetry/core/tracing_controller.py", line 47, in StopTracing
    return self._tracing_controller_backend.StopTracing()
  File "/b/rr/tmpWmXUqT/w/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py", line 130, in StopTracing
    '\n'.join(raised_exception_messages))
Error: Exceptions raised when trying to stop tracing:
Traceback (most recent call last):
  File "/b/rr/tmpWmXUqT/w/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py", line 118, in StopTracing
    agent.CollectAgentTraceData(builder)
  File "/b/rr/tmpWmXUqT/w/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/chrome_tracing_agent.py", line 248, in CollectAgentTraceData
    '\n'.join(raised_exception_messages))
ChromeTracingStoppedError: Exceptions raised when trying to collect Chrome devtool tracing:
Error when collecting Chrome tracing on devtools at port localabstract:chrome_devtools_remote:
Traceback (most recent call last):
  File "/b/rr/tmpWmXUqT/w/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/chrome_tracing_agent.py", line 237, in CollectAgentTraceData
    client.CollectChromeTracingData(trace_data_builder)
  File "/b/rr/tmpWmXUqT/w/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_client_backend.py", line 370, in CollectChromeTracingData
    self._tracing_backend.CollectTraceData(trace_data_builder, timeout)
  File "/b/rr/tmpWmXUqT/w/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/tracing_backend.py", line 229, in CollectTraceData
    self._CollectTracingData(trace_data_builder, timeout)
  File "/b/rr/tmpWmXUqT/w/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/tracing_backend.py", line 267, in _CollectTracingData
    'the timeout amount.' % elapsed_time)
TracingTimeoutException: Only received partial trace data due to timeout after 30.0300691128 seconds. If the trace data is big, you may want to increase the timeout amount.

Comment 8 by pasko@chromium.org, Dec 12 2016

Owner: rnep...@chromium.org

rnephew: since you are driving the ADB migration story, can you take it?

Comment 9 by rnep...@chromium.org, Dec 12 2016

I wouldn't say I'm driving the ADB migration story (that would probably be John), but I certainly don't mind looking at this problem.

Project Member

Comment 10 by bugdroid1@chromium.org, Dec 16 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/293fc82cb952c4c0d0ef816b8b86b7e1a63ada9a

commit 293fc82cb952c4c0d0ef816b8b86b7e1a63ada9a
Author: rnephew <rnephew@chromium.org>
Date: Fri Dec 16 16:46:07 2016

[Android] Add ability to generate test trace json for perf tests runs.

This adds the ability to set --trace-output <dir> and generate a json file that
can be opened in the chrome trace viewer that will display timelines of when
each test is run on which testing shard.

BUG= 667470 

Review-Url: https://codereview.chromium.org/2583613002
Cr-Commit-Position: refs/heads/master@{#439125}

[modify] https://crrev.com/293fc82cb952c4c0d0ef816b8b86b7e1a63ada9a/build/android/PRESUBMIT.py
[modify] https://crrev.com/293fc82cb952c4c0d0ef816b8b86b7e1a63ada9a/build/android/pylib/__init__.py
[modify] https://crrev.com/293fc82cb952c4c0d0ef816b8b86b7e1a63ada9a/build/android/pylib/local/device/local_device_perf_test_run.py
[modify] https://crrev.com/293fc82cb952c4c0d0ef816b8b86b7e1a63ada9a/build/android/pylib/perf/perf_test_instance.py
[modify] https://crrev.com/293fc82cb952c4c0d0ef816b8b86b7e1a63ada9a/build/android/test_runner.py
[modify] https://crrev.com/293fc82cb952c4c0d0ef816b8b86b7e1a63ada9a/build/android/test_runner.pydeps

Project Member

Comment 11 by bugdroid1@chromium.org, Dec 16 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/1cf7ff9fc603cd99c35dec979585f390bb43c244

commit 1cf7ff9fc603cd99c35dec979585f390bb43c244
Author: rnephew <rnephew@chromium.org>
Date: Fri Dec 16 18:54:54 2016

[Android][Perf] Generate test_runner trace file.

BUG= 667470 

Change-Id: I930cc36d672de41c4561ea04525668a3bfeaca69
Reviewed-on: https://chromium-review.googlesource.com/421091
Reviewed-by: Michael Case <mikecase@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: rnephew <rnephew@chromium.org>

[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/api.py
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/enable_platform_mode_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/last_known_devices_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_allow_high_battery_temp_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_allow_low_battery_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_disable_location_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_user_build_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_tests_failure.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_tests_infra_failure.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/resource_size_builder_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/timestamp_as_point_id_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/builder.expected/full_chromium_perf_Android_Builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/builder.expected/full_chromium_perf_Android_arm64_Builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_WebView_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_WebView_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_WebView_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_WebView_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_WebView_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_WebView_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus7v2_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus7v2_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus7v2_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus9_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus9_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus9_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_One_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_One_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_One_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_fyi_Android_Power_Nexus_5X_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/get_perf_test_list_old_data.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/host_info_failure.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/missing_device.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/src_side_script_fails.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/test_failure.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5X_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5X_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5X_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus6_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus6_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus6_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus7v2_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus7v2_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus7v2_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus9_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus9_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus9_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_One_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_One_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_One_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_fyi_Android_Power_Nexus_5X_Perf.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_arm64_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_armv6_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_data_reduction_proxy_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_kitkat_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_lollipop_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_marshmallow_64bit_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_marshmallow_64bit_perf.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_mips_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_x86_builder.json

Project Member

Comment 12 by bugdroid1@chromium.org, Dec 16 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/1cf7ff9fc603cd99c35dec979585f390bb43c244

commit 1cf7ff9fc603cd99c35dec979585f390bb43c244
Author: rnephew <rnephew@chromium.org>
Date: Fri Dec 16 18:54:54 2016

[Android][Perf] Generate test_runner trace file.

BUG= 667470 

Change-Id: I930cc36d672de41c4561ea04525668a3bfeaca69
Reviewed-on: https://chromium-review.googlesource.com/421091
Reviewed-by: Michael Case <mikecase@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: rnephew <rnephew@chromium.org>

[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/api.py
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/enable_platform_mode_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/last_known_devices_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_allow_high_battery_temp_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_allow_low_battery_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_disable_location_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_user_build_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_tests_failure.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/perf_tests_infra_failure.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/resource_size_builder_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipe_modules/chromium_android/example.expected/timestamp_as_point_id_basic.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/builder.expected/full_chromium_perf_Android_Builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/builder.expected/full_chromium_perf_Android_arm64_Builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_WebView_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_WebView_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_WebView_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_WebView_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_WebView_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_WebView_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus7v2_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus7v2_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus7v2_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus9_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus9_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus9_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_One_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_One_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_One_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/full_chromium_perf_fyi_Android_Power_Nexus_5X_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/get_perf_test_list_old_data.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/host_info_failure.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/missing_device.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/src_side_script_fails.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/android/perf.expected/test_failure.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5X_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5X_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5X_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus6_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus6_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus6_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus7v2_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus7v2_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus7v2_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus9_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus9_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus9_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_One_Perf__1_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_One_Perf__2_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_One_Perf__3_.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/chromium.expected/full_chromium_perf_fyi_Android_Power_Nexus_5X_Perf.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_arm64_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_armv6_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_data_reduction_proxy_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_kitkat_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_lollipop_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_marshmallow_64bit_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_marshmallow_64bit_perf.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_mips_builder.json
[modify] https://crrev.com/1cf7ff9fc603cd99c35dec979585f390bb43c244/scripts/slave/recipes/cronet.expected/android_cronet_x86_builder.json

Project Member

Comment 13 by bugdroid1@chromium.org, Dec 19 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/3bf0a397af67b2d4c0537776cec4b56fe60ff48e

commit 3bf0a397af67b2d4c0537776cec4b56fe60ff48e
Author: rnephew <rnephew@chromium.org>
Date: Mon Dec 19 18:32:19 2016

[Android][Perf] Reland Generate test_runner trace file.

BUG= 667470 

Change-Id: I6dbd2c9bf098b7f1e8fd48016abb45c5002a3848
Reviewed-on: https://chromium-review.googlesource.com/421259
Reviewed-by: Michael Case <mikecase@chromium.org>
Commit-Queue: rnephew <rnephew@chromium.org>

[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/api.py
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/enable_platform_mode_basic.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/last_known_devices_basic.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_allow_high_battery_temp_basic.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_allow_low_battery_basic.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_basic.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_disable_location_basic.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/perf_runner_user_build_basic.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/perf_tests_failure.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/perf_tests_infra_failure.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/resource_size_builder_basic.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipe_modules/chromium_android/example.expected/timestamp_as_point_id_basic.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/builder.expected/full_chromium_perf_Android_Builder.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/builder.expected/full_chromium_perf_Android_arm64_Builder.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_WebView_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_WebView_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5X_WebView_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus5_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_WebView_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_WebView_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus6_WebView_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus7v2_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus7v2_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus7v2_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus9_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus9_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_Nexus9_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_One_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_One_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_Android_One_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/full_chromium_perf_fyi_Android_Power_Nexus_5X_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/get_perf_test_list_old_data.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/host_info_failure.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/missing_device.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/src_side_script_fails.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/android/perf.expected/test_failure.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5X_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5X_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5X_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus5_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus6_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus6_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus6_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus7v2_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus7v2_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus7v2_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus9_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus9_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_Nexus9_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_One_Perf__1_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_One_Perf__2_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_Android_One_Perf__3_.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/chromium.expected/full_chromium_perf_fyi_Android_Power_Nexus_5X_Perf.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/cronet.expected/android_cronet_arm64_builder.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/cronet.expected/android_cronet_armv6_builder.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/cronet.expected/android_cronet_data_reduction_proxy_builder.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/cronet.expected/android_cronet_kitkat_builder.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/cronet.expected/android_cronet_lollipop_builder.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/cronet.expected/android_cronet_marshmallow_64bit_builder.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/cronet.expected/android_cronet_marshmallow_64bit_perf.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/cronet.expected/android_cronet_mips_builder.json
[modify] https://crrev.com/3bf0a397af67b2d4c0537776cec4b56fe60ff48e/scripts/slave/recipes/cronet.expected/android_cronet_x86_builder.json

Comment 14 by rnep...@chromium.org, Dec 20 2016

tWith those recent changes, its easy to confirm that this is indeed most likely a problem with the host. Shards 0-3 and 5-6 have tests failing in the same way that are running at the same time. Shard 4 has no test failing, and is done running before the problem arises. Attached you will see the trace of the test run. I will look more into this and try to see if I can figure out which test is causing the problems with the host.

this.html
3.1 MB View Download

Comment 15 by charliea@chromium.org, Dec 28 2016

/bump

rnephew@, any update on this?

Comment 16 by jbudorick@chromium.org, Dec 28 2016

#15: I think you might have to wait until Tuesday.

Comment 17 by nedngu...@google.com, Dec 28 2016

Cc: -nednguyen@chromium.org nedngu...@google.com

Comment 18 by rnep...@chromium.org, Jan 3 2017

Summary: Simultaneous failure across all device shards seen on nexus 7 devices (was: start_with_url.cold.[startup_pages,startup_pages.reference] failure on chromium.perf)

Project Member

Comment 19 by bugdroid1@chromium.org, Jan 3 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3b402227aa7e5c1851893cf8d7090dc456af8ac6

commit 3b402227aa7e5c1851893cf8d7090dc456af8ac6
Author: rnephew <rnephew@chromium.org>
Date: Tue Jan 03 21:54:10 2017

[Android] Disable mid perf run device recovery on n7 devices.

Speculative disabling to see if it causes failures to stop.

BUG= 667470 

Review-Url: https://codereview.chromium.org/2606343003
Cr-Commit-Position: refs/heads/master@{#441217}

[modify] https://crrev.com/3b402227aa7e5c1851893cf8d7090dc456af8ac6/build/android/pylib/local/device/local_device_perf_test_run.py

Comment 20 by rnep...@chromium.org, Jan 4 2017

I think this is going to be an extremely difficult one to nail down. It looks like something is killing adb on the host mid run which is causing failures. I looked at the past 10 runs, and there is no single test in common in all of the failures so it is not a specific test that is failing and causing adb to go down. It is something telemetry does. Since we do not see it on other bots, it is likely N7 specific.

It could also be related to the forwarder.

Project Member

Comment 21 by bugdroid1@chromium.org, Jan 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e5e86244987fa95cb6784aeb06d2004bb44e5f3f

commit e5e86244987fa95cb6784aeb06d2004bb44e5f3f
Author: rnephew <rnephew@chromium.org>
Date: Wed Jan 04 16:23:07 2017

Revert of [Android] Disable mid perf run device recovery on n7 devices. (patchset #2 id:20001 of https://codereview.chromium.org/2606343003/ )

Reason for revert:
Did not fix the issue.

Original issue's description:
> [Android] Disable mid perf run device recovery on n7 devices.
>
> Speculative disabling to see if it causes failures to stop.
>
> BUG= 667470 
>
> Committed: https://crrev.com/3b402227aa7e5c1851893cf8d7090dc456af8ac6
> Cr-Commit-Position: refs/heads/master@{#441217}

TBR=mikecase@chromium.org,jbudorick@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 667470 

Review-Url: https://codereview.chromium.org/2614723002
Cr-Commit-Position: refs/heads/master@{#441386}

[modify] https://crrev.com/e5e86244987fa95cb6784aeb06d2004bb44e5f3f/build/android/pylib/local/device/local_device_perf_test_run.py

Comment 22 by rnep...@chromium.org, Jan 4 2017

Data from runs is being compile here:
https://docs.google.com/a/google.com/spreadsheets/d/11zx0GVofQfao6lPVvuOCIsv-n6HzUkRpbt0pMMvL_OA/edit?usp=sharing

It has links to the test traces. If you select a test and click 'm' it will highlight that time range which makes it easier to see what else is running at the same time. I have been unable to figure out what is going on though.

Comment 23 by rnep...@chromium.org, Jan 4 2017

Deeper analysis of two runs:
https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%282%29/builds/3665
https://storage.cloud.google.com/chromium-testrunner-trace/Android%20Nexus7v2%20Perf%20(2)/trace_3665.html
whatever went wrong happened between 5,057,761.892ms and 5,128,125.019ms.

start_with_url.cold.startup_pages - Devtools Crash
Start time:4,962,141.581
Finishtime:5,135,063.009
[0102/174424:ERROR:host_forwarder_main.cc(477)] ERROR: could not unmap port.
ERROR: Existing controllers:
ERROR: 59170:57868
ERROR: 49823:34604
ERROR: 43108:39496

system_health.common_mobile.reference - Devtools Crash
Start time:4,950,805.748
Finishtime:5,132,824.479
ERROR: Existing controllers:
ERROR: 49823:34604
ERROR: 43108:39496

dromaeo.jslibstylejquery - Devtools Crash
Start time:5,057,761.892 
Finishtime:5,128,125.019
[0102/174421:ERROR:host_forwarder_main.cc(477)] ERROR: could not unmap port.
ERROR: Existing controllers:
ERROR: 49823:34604

page_cycler_v2.intl_ko_th_vi - Devtools Crash
Start time:2,738,648.190
Finishtime:5,488,550.152
[0102/174422:ERROR:host_forwarder_main.cc(477)] ERROR: could not unmap port.
ERROR: Existing controllers:

smoothness.pathological_mobile_sites - Failure when pulling chrome trace (devtools died mid pull?)
Start time:4,915,655.080
Finishtime:5,299,728.178
[0102/174414:ERROR:host_forwarder_main.cc(477)] ERROR: could not unmap port.
ERROR: Existing controllers:

power.gpu_rasterization.typical_10_mobile - Devtools Crash
Start time:4,812,061.740
Finishtime:5,465,560.927
[0102/174419:ERROR:host_forwarder_main.cc(477)] ERROR: could not unmap port.
ERROR: Existing controllers:
ERROR: 49823:34604





https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%282%29/builds/3667
https://storage.cloud.google.com/chromium-testrunner-trace/Android%20Nexus7v2%20Perf%20(2)/trace_3667.html
Whatever went wrong happened between 4,500,670ms and 4,907,159ms

memory.top_10_mobile.reference - devtools crash
Start time:2,864,390.694
Finishtime:4,907,159.837
[0102/174414:ERROR:host_forwarder_main.cc(477)] ERROR: could not unmap port.
ERROR: Existing controllers:

power.gpu_rasterization.typical_10_mobile - devtools crash
Start time:4,500,670.534
Finishtime:5,183,810.842
[0102/174414:ERROR:host_forwarder_main.cc(477)] ERROR: could not unmap port.
ERROR: Existing controllers:
ERROR: 33053:42187
ERROR: 37169:51379
ERROR: 36304:46969
ERROR: 57145:54625

system_health.common_mobile - devtools crash
Start time:1,919,581.485
Finishtime:5,109,633.391
[0102/174414:ERROR:host_forwarder_main.cc(477)] ERROR: could not unmap port.
ERROR: Existing controllers:


If it is a test specific trigger, it has to be part of system_health.common_mobile. It is the only test that has either it or its reference build failing each time. I would like to disable system_health.common_mobile on n7 bots to rule that out. If its a problem with the forwarder I would expect to see it on other bots, and we dont.

Project Member

Comment 24 by bugdroid1@chromium.org, Jan 5 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a73420c80a6038860cc35d1d6f1498c963ba2b52

commit a73420c80a6038860cc35d1d6f1498c963ba2b52
Author: rnephew <rnephew@chromium.org>
Date: Thu Jan 05 17:33:12 2017

[Android] Disable system_health.common_mobile on nexus 7 devices.

BUG= 667470 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq

Review-Url: https://codereview.chromium.org/2611753005
Cr-Commit-Position: refs/heads/master@{#441690}

[modify] https://crrev.com/a73420c80a6038860cc35d1d6f1498c963ba2b52/tools/perf/benchmarks/system_health.py

Comment 25 by rnep...@chromium.org, Jan 6 2017

Wasn't system health. 
https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%282%29/builds/3690

smoothness.gpu_rasterization.top_25_smooth.reference failed in the same way, and was the only thing running at the time (so didn't bring down any other tests with it). At least we know for sure now that that test can trigger the problem. Its not the only one that can though, so disabling it will not help. Reverting system health disabling.

Project Member

Comment 26 by bugdroid1@chromium.org, Jan 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ed8dccd0c958bb372c8f266631e71a471a5b6c17

commit ed8dccd0c958bb372c8f266631e71a471a5b6c17
Author: rnephew <rnephew@chromium.org>
Date: Fri Jan 06 16:48:48 2017

Revert of [Android] Disable system_health.common_mobile on nexus 7 devices. (patchset #1 id:1 of https://codereview.chromium.org/2611753005/ )

Reason for revert:
Didn't fix.

Original issue's description:
> [Android] Disable system_health.common_mobile on nexus 7 devices.
>
> BUG= 667470 
> CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq
>
> Review-Url: https://codereview.chromium.org/2611753005
> Cr-Commit-Position: refs/heads/master@{#441690}
> Committed: https://chromium.googlesource.com/chromium/src/+/a73420c80a6038860cc35d1d6f1498c963ba2b52

TBR=nednguyen@google.com,sullivan@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 667470 

Review-Url: https://codereview.chromium.org/2618013002
Cr-Commit-Position: refs/heads/master@{#441950}

[modify] https://crrev.com/ed8dccd0c958bb372c8f266631e71a471a5b6c17/tools/perf/benchmarks/system_health.py

Project Member

Comment 27 by bugdroid1@chromium.org, Jan 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f08e309602f10cd50a64919fbfecf181fda2d2ac

commit f08e309602f10cd50a64919fbfecf181fda2d2ac
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Fri Jan 06 22:41:26 2017

Roll src/third_party/catapult/ f84aaa04d..fa6d5ce49 (4 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/f84aaa04d4d0..fa6d5ce49e34

$ git log f84aaa04d..fa6d5ce49 --date=short --no-merges --format='%ad %ae %s'
2017-01-06 jessimb Adding skeleton for /speed_releasing endpoint.
2017-01-06 rnephew [Android][Telemetry] Log processes running during Nexus 7 runs.
2017-01-06 benjhayden Fix some old style violations in UserModelBuilder.
2017-01-06 rnephew [BattOr] Upload Windows AVRDude binary for flashing BattOr firmware.

BUG= 667470 

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2617873003
Cr-Commit-Position: refs/heads/master@{#442078}

[modify] https://crrev.com/f08e309602f10cd50a64919fbfecf181fda2d2ac/DEPS

Comment 28 by rnep...@chromium.org, Jan 9 2017

The new logging confirms that the adb server is being killed and restarted.

Before:
chrome-+ 14583     1 12 03:45 ?        00:09:57 adb -P 5037 fork-server server

After:
chrome-+   328     1  4 05:14 ?        00:00:00 adb -P 5037 fork-server server

Comment 29 by rnep...@chromium.org, Jan 31 2017

Cc: bpastene@chromium.org mikec...@chromium.org

 Issue 687008  has been merged into this issue.

Comment 30 by rnep...@chromium.org, Mar 29 2017

Blocking: 663748

Comment 31 by rnep...@chromium.org, Mar 29 2017

Blockedon: -663748

Comment 32 by rnep...@chromium.org, Apr 3 2017

Status: Fixed (was: Assigned)

As discussed in  crbug.com/691654  john landed a fix to the forwarder leaking ports. This bot stopped having issues immediately after that landed. I believe this is fixed, I dont remember the last time I saw that many green runs in a row on this bot.

Project Member

Comment 33 by bugdroid1@chromium.org, Apr 19 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d132421d7a007dacb588ca4b098f38759a5a5481

commit d132421d7a007dacb588ca4b098f38759a5a5481
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Wed Apr 19 17:57:37 2017

Roll src/third_party/catapult/ 24e36dadf..121ef2b4a (1 commit)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/24e36dadf5b1..121ef2b4a8c6

$ git log 24e36dadf..121ef2b4a --date=short --no-merges --format='%ad %ae %s'
2017-04-19 rnephew [Telemetry] Get rid of old debugging information.

Created with:
  roll-dep src/third_party/catapult
BUG= 667470 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I20439992b215d7734dad275d7a9ebe0cb27b3d11
Reviewed-on: https://chromium-review.googlesource.com/481900
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#465667}
[modify] https://crrev.com/d132421d7a007dacb588ca4b098f38759a5a5481/DEPS

Project Member

Comment 34 by bugdroid1@chromium.org, Nov 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ee30418fb92368068c8c1facd2e694ee9cc14315

commit ee30418fb92368068c8c1facd2e694ee9cc14315
Author: Egor Pasko <pasko@chromium.org>
Date: Mon Nov 06 17:03:57 2017

start_with_url benchmarks: remove kapook.com

Remove loading of kapook.com when benchmarking startup.

Reasons:
* it has been disabled for a while due to failures on N7, looks cleaner
  without the override
* it was intended as a 'stress test', but the 'stress' parts of the test
  would kick in after all actionable metrics relevant to startup are
  recorded

Bug:  667470 
Change-Id: I793a4fbf080928242d81b713035c2d57c843a3cc
Reviewed-on: https://chromium-review.googlesource.com/754819
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Egor Pasko <pasko@chromium.org>
Cr-Commit-Position: refs/heads/master@{#514154}
[modify] https://crrev.com/ee30418fb92368068c8c1facd2e694ee9cc14315/tools/perf/benchmarks/start_with_url.py
[modify] https://crrev.com/ee30418fb92368068c8c1facd2e694ee9cc14315/tools/perf/page_sets/startup_pages.py

►

Issue 667470 link

Issue metadata

Simultaneous failure across all device shards seen on nexus 7 devices

Issue description

Comment 1 by bugdroid1@chromium.org, Nov 21 2016

Comment 1 Deleted

Comment 2 by charliea@chromium.org, Dec 6 2016

Comment 2 Deleted

Comment 3 by charliea@chromium.org, Dec 6 2016

Comment 3 Deleted

Comment 4 by charliea@chromium.org, Dec 6 2016

Comment 4 Deleted

Comment 5 by bugdroid1@chromium.org, Dec 6 2016

Comment 5 Deleted

Comment 6 by pasko@chromium.org, Dec 9 2016

Comment 6 Deleted

Comment 7 by rnep...@chromium.org, Dec 9 2016

Comment 7 Deleted

Comment 8 by pasko@chromium.org, Dec 12 2016

Comment 8 Deleted

Comment 9 by rnep...@chromium.org, Dec 12 2016

Comment 9 Deleted

Comment 10 by bugdroid1@chromium.org, Dec 16 2016

Comment 10 Deleted

Comment 11 by bugdroid1@chromium.org, Dec 16 2016

Comment 11 Deleted

Comment 12 by bugdroid1@chromium.org, Dec 16 2016

Comment 12 Deleted

Comment 13 by bugdroid1@chromium.org, Dec 19 2016

Comment 13 Deleted

Comment 14 by rnep...@chromium.org, Dec 20 2016

Comment 14 Deleted

Comment 15 by charliea@chromium.org, Dec 28 2016

Comment 15 Deleted

Comment 16 by jbudorick@chromium.org, Dec 28 2016

Comment 16 Deleted

Comment 17 by nedngu...@google.com, Dec 28 2016

Comment 17 Deleted

Comment 18 by rnep...@chromium.org, Jan 3 2017

Comment 18 Deleted

Comment 19 by bugdroid1@chromium.org, Jan 3 2017

Comment 19 Deleted

Comment 20 by rnep...@chromium.org, Jan 4 2017

Comment 20 Deleted

Comment 21 by bugdroid1@chromium.org, Jan 4 2017

Comment 21 Deleted

Comment 22 by rnep...@chromium.org, Jan 4 2017

Comment 22 Deleted

Comment 23 by rnep...@chromium.org, Jan 4 2017

Comment 23 Deleted

Comment 24 by bugdroid1@chromium.org, Jan 5 2017

Comment 24 Deleted

Comment 25 by rnep...@chromium.org, Jan 6 2017

Comment 25 Deleted

Comment 26 by bugdroid1@chromium.org, Jan 6 2017

Comment 26 Deleted

Comment 27 by bugdroid1@chromium.org, Jan 6 2017

Comment 27 Deleted

Comment 28 by rnep...@chromium.org, Jan 9 2017

Comment 28 Deleted

Comment 29 by rnep...@chromium.org, Jan 31 2017

Comment 29 Deleted

Comment 30 by rnep...@chromium.org, Mar 29 2017

Comment 30 Deleted

Comment 31 by rnep...@chromium.org, Mar 29 2017

Comment 31 Deleted

Comment 32 by rnep...@chromium.org, Apr 3 2017

Comment 32 Deleted

Comment 33 by bugdroid1@chromium.org, Apr 19 2017

Comment 33 Deleted

Comment 34 by bugdroid1@chromium.org, Nov 6 2017

Comment 34 Deleted

Sign in to add a comment