Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Issue 637904 "telemetry_perf_unittests (with patch)" is flaky
Starred by 1 user Project Member Reported by chromium...@appspot.gserviceaccount.com, Aug 15 2016 Back to list
Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug


Sign in to add a comment
"telemetry_perf_unittests (with patch)" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 4 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA.



This flaky test/step was previously tracked in issue 635255.
 
Project Member Comment 1 by chromium...@appspot.gserviceaccount.com, Aug 16 2016
Detected 18 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Owner: bcwh...@chromium.org
Status: Started
Cc: gab@chromium.org
Owner: asvitk...@chromium.org
Status: Assigned
This has apparently been happening since at least mid-May.

According to https://codereview.chromium.org/2236383002 ...
Some attempts have been made to fix this but a real fix is coming from asvitkine.

Labels: -Sheriff-Chromium
I don't think it's caused by the same issue. What's the stack trace?
Also, I'm ooo until August 28 - so if you want someone to look at this before then, i suggest finding a different owner.
Project Member Comment 7 by chromium...@appspot.gserviceaccount.com, Aug 18 2016
Detected 13 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 8 by chromium...@appspot.gserviceaccount.com, Aug 21 2016
Detected 11 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 9 by chromium...@appspot.gserviceaccount.com, Aug 25 2016
Detected 46 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Cc: asvitk...@chromium.org
Labels: Sheriff-Chromium
Owner: ----
Status: Available
Marking available and putting the Sheriff-Chromium label back.

I don't think this is the same issue as https://bugs.chromium.org/p/chromium/issues/detail?id=635255 unless there's evidence otherwise.

These tests can fail if there's a startup crash. So sheriff need to get the stack trace for that to figure out who should own this. 
Components: Infra>Client>Android
I perused some of these failures during my sheriff stint today. There are various errors. Today the flakes stopped happening on non-android. I am not very familiar with android so I'm hoping someone from Infra can triage these.

https://chromium-swarm.appspot.com/user/task/30efee9b3f25b210
ctrl-f 232/
(1) "Failed to install test certificate authority on target platform. Browsers may fall back to ignoring certificate errors."
(2) Cascades to: Traceback (most recent call last):
    _RunStoryAndProcessErrorIfNeeded at /b/swarm_slave/w/irUDEYK1/third_party/catapult/telemetry/telemetry/internal/story_runner.py:104
      None
    DidRunStory at /b/swarm_slave/w/irUDEYK1/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:159
      None
  AttributeError: 'NoneType' object has no attribute 'credentials'
(3) Then many following tests fail in story_set.py, like:
 File "/b/swarm_slave/w/irUDEYK1/tools/perf/measurements/v8_gc_times_unittest.py", line 93, in testWithGarbageCollectionEvents
    File "/b/swarm_slave/w/irUDEYK1/third_party/catapult/telemetry/telemetry/testing/page_test_test_case.py", line 45, in CreateEmptyPageSet
    File "/b/swarm_slave/w/irUDEYK1/third_party/catapult/telemetry/telemetry/story/story_set.py", line 41, in __init__
  ValueError: Invalid directory path of base_dir: /b/swarm_slave/w/irUDEYK1/third_party/catapult/telemetry/telemetry/internal/testing

Maybe the scripts don't handle test-CA installation failures very gracefully? And who knows why the test CAs weren't installed correctly in the first place.

https://chromium-swarm.appspot.com/user/task/30ede4911aed9e10
Tests were disabled in issue 637938, so yay.

Will look again tomorrow.
Project Member Comment 12 by chromium...@appspot.gserviceaccount.com, Aug 30 2016
Detected 7 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 13 by chromium...@appspot.gserviceaccount.com, Aug 31 2016
Detected 9 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Comment 14 by gab@chromium.org, Sep 1 2016
Cc: sullivan@chromium.org
I don't think these are the same flakes anymore.
Cc: nednguyen@chromium.org
Cc: petrcermak@chromium.org
I filed issue 643320 about recent flakes of telemetry_perf_unittest
Project Member Comment 18 by chromium...@appspot.gserviceaccount.com, Sep 1 2016
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 19 by chromium...@appspot.gserviceaccount.com, Sep 3 2016
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Labels: -Sheriff-Chromium
The very last failure is timeout on Android (https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/136335)
Comment 22 by gab@chromium.org, Sep 6 2016
Cc: -gab@chromium.org
Removing self per #14.
Blockedon: 629434 643320
Blockedon: 644270
Project Member Comment 25 by chromium...@appspot.gserviceaccount.com, Sep 6 2016
Labels: Sheriff-Chromium
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Project Member Comment 26 by chromium...@appspot.gserviceaccount.com, Sep 7 2016
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
All of the last 5 failures seem to be due to issue 643320.
Project Member Comment 28 by chromium...@appspot.gserviceaccount.com, Sep 8 2016
Detected 7 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 29 by bugdroid1@chromium.org, Sep 10 2016
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c4b1e75d9334ca63343d5e44768c246d796c8386

commit c4b1e75d9334ca63343d5e44768c246d796c8386
Author: nednguyen <nednguyen@google.com>
Date: Sat Sep 10 12:00:05 2016

Manually roll src/third_party/catapult/ da6d44e4a..c0b988891 (3 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/da6d44e4a2a5..c0b988891fd1

$ git log da6d44e4a..c0b988891 --date=short --no-merges --format='%ad %ae %s'
2016-09-09 erikchen Add a new cpu time TBMv2 system health metric.
2016-09-09 nednguyen Update the lock operation in cloud_storage with better implementation using py_utils.lock
2016-09-09 rnephew [BattOr] Kill BattOr shell if there is a problem with communicating during clock sync.

BUG= 640312 ,637904, 645106 , 645720 

TBR=catapult-sheriff@chromium.org, jbudorick@chromium.org

Review-Url: https://codereview.chromium.org/2326063004
Cr-Commit-Position: refs/heads/master@{#417833}

[modify] https://crrev.com/c4b1e75d9334ca63343d5e44768c246d796c8386/DEPS
[modify] https://crrev.com/c4b1e75d9334ca63343d5e44768c246d796c8386/build/android/test_runner.pydeps

Project Member Comment 30 by chromium...@appspot.gserviceaccount.com, Sep 10 2016
Detected 6 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Labels: -Sheriff-Chromium
Two recent failures:
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/291856
https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/140017
shows problems in the SystemHealthBenchmarkSmokeTest tests:

[7/23] benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.load:social:facebook failed unexpectedly 0.0310s:
  Chrome build location for win_AMD64 not found. Browser will be run without Flash.
  Chrome build location for win_AMD64 not found. Browser will be run without Flash.
  Chrome build location for win_AMD64 not found. Browser will be run without Flash.
  Traceback (most recent call last):
    File "e:\b\swarm_slave\w\irv8mnl9\tools\perf\core\path_util.pyc\..\..\benchmarks\system_health_smoke_test.py", line 85, in RunTest
      self.assertEqual(0, SinglePageBenchmark().Run(options),
    File "e:\b\swarm_slave\w\irv8mnl9\third_party\catapult\telemetry\telemetry\benchmark.py", line 91, in Run
      return story_runner.RunBenchmark(self, finder_options)
    File "e:\b\swarm_slave\w\irv8mnl9\third_party\catapult\telemetry\telemetry\internal\story_runner.py", line 304, in RunBenchmark
      stories = benchmark.CreateStorySet(finder_options)
    File "e:\b\swarm_slave\w\irv8mnl9\tools\perf\core\path_util.pyc\..\..\benchmarks\system_health_smoke_test.py", line 69, in CreateStorySet
      story_set = super(SinglePageBenchmark, self).CreateStorySet(options)
    File "e:\b\swarm_slave\w\irv8mnl9\tools\perf\core\path_util.pyc\..\..\benchmarks\system_health.py", line 103, in CreateStorySet
      take_memory_measurement=True)
    File "e:\b\swarm_slave\w\irv8mnl9\tools\perf\page_sets\system_health\system_health_stories.py", line 31, in __init__
      self.AddStory(story_class(self, take_memory_measurement))
    File "e:\b\swarm_slave\w\irv8mnl9\tools\perf\page_sets\system_health\system_health_story.py", line 75, in __init__
      grouping_keys={'case': case, 'group': group})
    File "e:\b\swarm_slave\w\irv8mnl9\third_party\catapult\telemetry\telemetry\page\__init__.py", line 43, in __init__
      cloud_storage.GetIfChanged(credentials_path, credentials_bucket)
    File "e:\b\swarm_slave\w\irv8mnl9\third_party\catapult\common\py_utils\py_utils\cloud_storage.py", line 339, in GetIfChanged
      with _FileLock(file_path):
    File "e:\b\depot_tools\python276_bin\lib\contextlib.py", line 17, in __enter__
      return self.gen.next()
    File "e:\b\swarm_slave\w\irv8mnl9\third_party\catapult\common\py_utils\py_utils\cloud_storage.py", line 243, in _FileLock
      fd = open(pseudo_lock_path, 'w')
  IOError: [Errno 13] Permission denied: 'e:\\b\\swarm_slave\\w\\irv8mnl9\\tools\\perf\\page_sets\\system_health\\../data/credentials.json.pseudo_lock'

Is this an infra problem?

Android one:
[143/284] benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.load:games:lazors failed unexpectedly 41.9931s:
  [ RUN      ] load:games:lazors
  Traceback (most recent call last):
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 78, in _RunStoryAndProcessErrorIfNeeded
      test.WillRunStory(state.platform)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/telemetry/telemetry/web_perf/timeline_based_measurement.py", line 274, in WillRunStory
      platform.tracing_controller.StartTracing(self._tbm_options.config)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/telemetry/telemetry/core/tracing_controller.py", line 43, in StartTracing
      self._tracing_controller_backend.StartTracing(tracing_config, timeout)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py", line 91, in StartTracing
      if agent.StartAgentTracing(config, timeout):
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/chrome_tracing_agent.py", line 113, in StartAgentTracing
      self._platform_backend.SetGraphicsMemoryTrackingEnabled(True)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/telemetry/telemetry/internal/platform/android_platform_backend.py", line 238, in SetGraphicsMemoryTrackingEnabled
      '-d'], as_root=True, check_return=True)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper
      retry_if_func=retry_if_func)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/utils/timeout_retry.py", line 157, in Run
      error_log_func=error_log_func)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 186, in JoinAll
      self._JoinAll(watcher, timeout)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 158, in _JoinAll
      thread.ReraiseIfException()
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 81, in run
      self._ret = self._func(*self._args, **self._kwargs)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/utils/timeout_retry.py", line 150, in <lambda>
      child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs),
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
      return f(*args, **kwargs)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/android/device_utils.py", line 904, in RunShellCommand
      output = handle_large_output(cmd, large_output)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/android/device_utils.py", line 882, in handle_large_output
      return handle_large_command(cmd)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/android/device_utils.py", line 864, in handle_large_command
      return handle_check_return(cmd)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/android/device_utils.py", line 855, in handle_check_return
      return run(cmd)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/android/device_utils.py", line 851, in run
      return self.adb.Shell(cmd)
    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 494, in Shell
      command, output, status=status, device_serial=self._device_serial)
  AdbShellCommandFailedError: (device: 060f09f513c8594a) shell command run via adb failed on the device:
    command: /data/local/tmp/profilers/memtrack_helper -d
    exit status: 1
    output:
    - FATAL: /data/local/tmp/profilers/memtrack_helper must be run as root!
  
  
  [  FAILED  ] load:games:lazors (7307 ms)


Cc: jbudorick@chromium.org
To #31, the failure on win is tracked in issue 643320.

The failure on Android seems like an infra failure with us not having root on the device.
#31/#32: I don't think this is an infra failure. We do have root on the device, and we're attempting to run the command as root:

    File "/b/swarm_slave/w/ir0OxkXR/third_party/catapult/telemetry/telemetry/internal/platform/android_platform_backend.py", line 238, in SetGraphicsMemoryTrackingEnabled
      '-d'], as_root=True, check_return=True)

I'm guessing that we think we're running as root though we aren't, so we don't use su? Need to look into this a bit more.
er, actually, we may not have root on the device. If that's the case, though, devil is still doing a poor job of messaging.
These, on the other hand, are more indicative of possible device failure:

CRITICAL:root:(TimeoutThread-1-for-delete_temporary_file(060f09f513c8594a)) Exception on _RunAdbCmd(<class 'devil.android.sdk.adb_wrapper.AdbWrapper'>, ['shell', 'rm -f /data/local/tmp/temp_file-db4f5fe7fb44f'], check_error=False, retries=2, device_serial=060f09f513c8594a, timeout=30), attempt 1 of 3: AdbCommandFailedError("(device: 060f09f513c8594a) adb shell 'rm -f /data/local/tmp/temp_file-db4f5fe7fb44f': failed with exit status 255 and output:\n- error: protocol fault (couldn't read status): Success\n",)
INFO:root:[host]> /b/swarm_slave/w/ir0OxkXR/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 060f09f513c8594a shell 'rm -f /data/local/tmp/temp_file-db4f5fe7fb44f'
CRITICAL:root:(TimeoutThread-2-for-delete_temporary_file(060f09f513c8594a)) Exception on _RunAdbCmd(<class 'devil.android.sdk.adb_wrapper.AdbWrapper'>, ['shell', 'rm -f /data/local/tmp/temp_file-db4f5fe7fb44f'], check_error=False, retries=2, device_serial=060f09f513c8594a, timeout=30), attempt 2 of 3: AdbCommandFailedError("(device: 060f09f513c8594a) adb shell 'rm -f /data/local/tmp/temp_file-db4f5fe7fb44f': failed with exit status 255 and output:\n- error: device '060f09f513c8594a' not found\n",)

...

  CommandFailedError: (device: 060f09f513c8594a) pm path returned: 'Error: Could not access the Package Manager.  Is the system running?'
Project Member Comment 36 by chromium...@appspot.gserviceaccount.com, Sep 13 2016
Labels: Sheriff-Chromium
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Comment 37 by grt@chromium.org, Sep 14 2016
Labels: -Sheriff-Chromium
The last 5 flakes are issue 643320, which is unrelated to device problem. Popping out of sheriff queue.
Project Member Comment 38 by chromium...@appspot.gserviceaccount.com, Sep 19 2016
Labels: Sheriff-Chromium
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Blockedon: 649167
Project Member Comment 40 by chromium...@appspot.gserviceaccount.com, Sep 22 2016
Detected 34 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Blockedon: 649216
Blockedon: 649217
Labels: -Sheriff-Chromium
The failures in #38 and #40:

* Timeout in benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.load:social:twitter on android_n5x_swarming_rel. Only seen once.
https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_swarming_rel/builds/32924

* "FATAL:navigation_handle_impl.cc(507)] Check failed: state_ != WILL_REDIRECT_REQUEST || next_index_ == 0". Only seen once.
https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_swarming_rel/builds/32913

* Issue 649167.

* Windows failures: issue 643320, afaict.

nednguyen@: Can you verify that the Windows failures are the same as that issue?
issue 643320 only happen once on win recently. Instead, I see issue 649217 & issue 649216 are the major cause of telemetry_perf_unittest' on win
Project Member Comment 45 by chromium...@appspot.gserviceaccount.com, Sep 23 2016
Labels: Sheriff-Chromium
Detected 37 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Project Member Comment 46 by chromium...@appspot.gserviceaccount.com, Sep 23 2016
Cc: stale-flakes-reports@google.com
Reporting to stale-flakes-reports@google.com to investigate why this issue has been in the appropriate queue 5 times or more.
Seems the problem is the infrastructure can't tell which specific part is flaky. Ideally, it should be able to file more targeted issues like it does for other test binaries.
Cc: st...@chromium.org
+Shuotao: do you know why the flakiness dashboard fails to show which specific tests in telemetry_perf_unittest suite are flaky? 
#48: because the step doesn't report its results to test-results in the gtest JSON format.
Comment 50 by st...@chromium.org, Sep 23 2016
Cc: kbr@chromium.org estaab@chromium.org
#48: I think John is right.

Probably we'd better switch telemetry test_runner to use gtest JSON format and gain the benefit automatically here.
Blockedon: 649762
Sounds good to me. I will file a different bug about that (issue 649762)
Cc: -stale-flakes-reports@google.com
Thank you for looking into this and filing a separate bug.
Project Member Comment 53 by chromium...@appspot.gserviceaccount.com, Sep 24 2016
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 54 by chromium...@appspot.gserviceaccount.com, Sep 26 2016
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 55 by chromium...@appspot.gserviceaccount.com, Sep 26 2016
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 56 by chromium...@appspot.gserviceaccount.com, Sep 27 2016
Detected 68 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 57 by chromium...@appspot.gserviceaccount.com, Sep 28 2016
Detected 9 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Cc: stip@chromium.org
Majority of the flakes of linux_android_rel_ng recently seems like infra/lab issue to me:
https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/148010
https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/147899

John, Mike: do we have a tracking bug for it?
Project Member Comment 59 by chromium...@appspot.gserviceaccount.com, Sep 29 2016
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 13 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Some of the Friday night flakes on the Android bots were due to me messing up the forwarder binary upgrade.

#58: that was https://bugs.chromium.org/p/chromium/issues/detail?id=650141
Detected 38 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Cc: eyaich@chromium.org
Cc: -petrcermak@chromium.org
Cc: -petrcermak@chromium.org
Detected 55 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 35 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 25 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Labels: -Sheriff-Chromium
I am chromium sheriff today.

It's difficult to follow this discussion. Let me remove Sheriff-Chromium since it looks there are already filed bugs other than this.

Please feel free to add Sheriff-Chromium again if my understanding is wrong.
Labels: Sheriff-Chromium
Detected 8 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Labels: -Sheriff-Chromium
Labels: Sheriff-Chromium
Detected 6 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Cc: -estaab@chromium.org
Labels: -Sheriff-Chromium
Same comment as in #71. Not sure who to assign this issue to. Those of you who know the topic and history, please assign to suitable owner.
Blockedon: 649762
Owner: nedngu...@google.com
Status: Assigned
I am working on addressing the flakiness reporting mechanism in issue 649762 so that we report which tests are failing, not the whole suite.
Detected 10 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Is any progress being made on the flakes on the various platforms? See for example this random failure from an unrelated CL:
https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/164497

Ken, most of the flakes here don't happen very frequently, the one you just have is probably the more serious issue in issue 619189
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 15 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 10 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 13 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 6 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 7 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 8 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 21 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Blockedon: 662021
Ok so I got a chance to look at this closer this afternoon ( I was waiting for https://codereview.chromium.org/2472123002 to land to see what it did) and it seems that cleared up the json errors, but we are still seeing some pretty consistent flakes.  

For windows most commonly it is smoke test system_health.memory_desktop.browse:news:nytimes

Example failing: https://chromium-swarm.appspot.com/user/task/324943564d061210

An snippet of the failing log: 
      expr, context_id=context_id, timeout=timeout)
    File "e:\b\swarm_slave\w\irhz58th\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "e:\b\swarm_slave\w\irhz58th\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_backend.py", line 36, in inner
      return func(inspector_backend, *args, **kwargs)
    File "e:\b\swarm_slave\w\irhz58th\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_backend.py", line 199, in ExecuteJavaScript
      self._runtime.Execute(expr, context_id, timeout)
    File "e:\b\swarm_slave\w\irhz58th\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_runtime.py", line 21, in Execute
      self.Evaluate(expr + '; 0;', context_id, timeout)
    File "e:\b\swarm_slave\w\irhz58th\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_runtime.py", line 45, in Evaluate
      res = self._inspector_websocket.SyncRequest(request, timeout)
    File "e:\b\swarm_slave\w\irhz58th\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 110, in SyncRequest
      res = self._Receive(timeout)
    File "e:\b\swarm_slave\w\irhz58th\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 166, in _Receive
      self._HandleNotification(result)
    File "e:\b\swarm_slave\w\irhz58th\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 179, in _HandleNotification
      self._domain_handlers[domain_name](result)
    File "e:\b\swarm_slave\w\irhz58th\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "e:\b\swarm_slave\w\irhz58th\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_backend.py", line 365, in _HandleInspectorDomainNotification
      raise exception
  DevtoolsTargetCrashException: Devtools target crashed
  ********************************************************************************
  (e:\b\swarm_slave\w\irhz58th\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_backend.py:418 _AddDebuggingInformation) Received a socket error in the browser connection and the tab no longer exists. The tab probably crashed.
  *******************************************************************************

The tab seems to be crashing but I can't tell if it is random failure or if something else is causing it. 


Here is an example of a browser log of one of these failures: 
 [8176:4916:1104/093737:ERROR:memory_mapped_file.cc(52)] Couldn't open e:\b\swarm_slave\w\irsz8uni\out\Release_x64\chrome_200_percent.pak
  [8176:4916:1104/093737:ERROR:data_pack.cc(79)] Failed to mmap datapack
  [8176:4916:1104/093737:WARNING:chrome_browser_main_win.cc(419)] Command line too long for RegisterApplicationRestart
  [7932:7784:1104/093737:ERROR:angle_platform_impl.cc(33)] ANGLE Display::initialize error 4: Could not create D3D11 device.
  [7932:7784:1104/093737:ERROR:gl_surface_egl.cc(630)] eglInitialize D3D11 failed with error EGL_NOT_INITIALIZED, trying next display type
  [5168:4348:1104/093737:ERROR:angle_platform_impl.cc(33)] ANGLE Display::initialize error 4: Could not create D3D11 device.
  [5168:4348:1104/093737:ERROR:gl_surface_egl.cc(630)] eglInitialize D3D11 failed with error EGL_NOT_INITIALIZED, trying next display type
  [7932:7784:1104/093738:ERROR:angle_platform_impl.cc(33)] ANGLE Display::initialize error 4: Renderer does not support PS 3.0.aborting!
  [7932:7784:1104/093738:ERROR:gl_surface_egl.cc(630)] eglInitialize D3D9 failed with error EGL_NOT_INITIALIZED
  [7932:7784:1104/093738:ERROR:gl_initializer_win.cc(272)] GLSurfaceEGL::InitializeOneOff failed.
  [7932:7784:1104/093738:INFO:media_foundation_video_encode_accelerator_win.cc(290)] Windows versions earlier than 8 are not supported.
  [7932:7784:1104/093738:ERROR:gpu_child_thread.cc(328)] Exiting GPU process due to errors during initialization
  [8176:4880:1104/093738:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
  [5168:4348:1104/093738:ERROR:angle_platform_impl.cc(33)] ANGLE Display::initialize error 4: Renderer does not support PS 3.0.aborting!
  [5168:4348:1104/093738:ERROR:gl_surface_egl.cc(630)] eglInitialize D3D9 failed with error EGL_NOT_INITIALIZED
  [5168:4348:1104/093738:ERROR:gl_initializer_win.cc(272)] GLSurfaceEGL::InitializeOneOff failed.
  [5168:4348:1104/093738:INFO:media_foundation_video_encode_accelerator_win.cc(290)] Windows versions earlier than 8 are not supported.
  [5168:4348:1104/093738:ERROR:gpu_child_thread.cc(328)] Exiting GPU process due to errors during initialization

I am not sure if these errors are typical or if one of them could be causing it.




Android failures on linux_android_rel_ng seem to be different tests and different reasons, the most recent related to adb failures: 

https://chromium-swarm.appspot.com/user/task/32495f84f5938310

    File "/b/swarm_slave/w/ir3qKHC9/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 81, in run
      self._ret = self._func(*self._args, **self._kwargs)
    File "/b/swarm_slave/w/ir3qKHC9/third_party/catapult/devil/devil/utils/timeout_retry.py", line 152, in <lambda>
      child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs),
    File "/b/swarm_slave/w/ir3qKHC9/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
      return f(*args, **kwargs)
    File "/b/swarm_slave/w/ir3qKHC9/third_party/catapult/devil/devil/android/device_utils.py", line 657, in Install
      reinstall=reinstall, permissions=permissions)
    File "/b/swarm_slave/w/ir3qKHC9/third_party/catapult/devil/devil/android/device_utils.py", line 743, in _InstallInternal
      base_apk.path, reinstall=reinstall, allow_downgrade=allow_downgrade)
    File "/b/swarm_slave/w/ir3qKHC9/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 697, in Install
      cmd, output, device_serial=self._device_serial)
  AdbCommandFailedError: (device: 0609d298003bc315) adb install /b/swarm_slave/w/ir3qKHC9/out/Release/apks/ChromePublic.apk: failed and output:
  - 4752 KB/s (58587431 bytes in 12.038s)
  - 	pkg: /data/local/tmp/ChromePublic.apk
  - Failure [INSTALL_FAILED_UID_CHANGED]
  

I didn't get very far in my investigations but wondered if anyone had any thoughts on the browser log errors.  Otherwise I will keep digging.

Detected 50 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
jbudorick@ would be the best person to help diagnose the Android failures.

It would be good to prove or disprove whether the GPU process launch failures are the cause of these test failures. Are they seen on any passing test runs?

I'll look into the INSTALL_FAILED_UID_CHANGED errors on Monday.
Detected 6 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
So I am not sure how to look at the browser log of a passing test to see if they are seen. 

It is interesting to note that we often seen nytimes failing with the same stack trace but then succeeding on a retry so that is why it is still green.  

For instance the most recent run of windowns_rel_ng: 

https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/326793

Here is a link to the telemetry_perf_unittests swarming task: 
https://chromium-swarm.appspot.com/user/task/325825ba3c80ca10

You see the stack trace of the failing nytimes test with the same browser output, but then you see it succeeed on retry so the builder is green. 

Where as in another recent build: 
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/326777

You see from the swarming task that it succeeded the first time: 
https://chromium-swarm.appspot.com/user/task/32580b0e52b2c810

and you never see that stack trace.

Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Cc: bpastene@chromium.org
Looking a bit at the android flakes, a lot of the recent failures are all localized to a single swarming bot: https://chromium-swarm.appspot.com/restricted/bot/build3-b4. It's been consistently failing all of its telemetry_perf_unittests runs for the same reason: "INSTALL_FAILED_UID_CHANGED"

That usually means a previous version of an app, in this case ChromePublic.apk, wasn't properly removed from the device before reinstalling. I'll reflash all phones on that host, that should clear it up.

Though, we should look at how we're cleaning these apps from the phones and see if we can't avoid the errors without having to trigger a reflash.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 13 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
All recent flakes are due to timeouts. Most likely caused by https://codereview.chromium.org/2493203002/ which increased the number of tests ran. I'll bump the timeout on the suite.
Project Member Comment 109 by bugdroid1@chromium.org, Nov 14
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1ebc90106e99c94b774f9de0f10197185500de0e

commit 1ebc90106e99c94b774f9de0f10197185500de0e
Author: bpastene <bpastene@chromium.org>
Date: Mon Nov 14 23:27:14 2016

Bump timeout of telemetry_perf_unittests on android_n5x bot to 20 min.

BUG=637904

Review-Url: https://codereview.chromium.org/2498903004
Cr-Commit-Position: refs/heads/master@{#431966}

[modify] https://crrev.com/1ebc90106e99c94b774f9de0f10197185500de0e/testing/buildbot/chromium.android.json

Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 14 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 18 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 6 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Almost all the new flakes are caused by issue 671018
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 6 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 20 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 9 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 9 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 29 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 6 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Cc: -nednguyen@chromium.org
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 8 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 14 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 16 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Almost all the new flakiness are due to telemetry_perf_unittest timed out on android_n5x_swarming after adb keeps failing. Filed issue 680141 on this.
Blockedon: 680141
Those aren't adb failures. Those are timeouts. The timing out is the problem.
Detected 13 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 35 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 48 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 16 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 14 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 8 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 60 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Blockedon: 669242
60 flakes in a day is a huge number. It looks to me like the most recent source of crashes might be a V8 bug. See Issue 669242.

https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_rel_ng/builds/376089 has this failure log:

Retrying failed tests (attempt #3 of 3)...

[1/1] benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.browse:news:nytimes failed unexpectedly 133.5537s:
[ RUN ] browse:news:nytimes
Traceback (most recent call last):
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 94, in _RunStoryAndProcessErrorIfNeeded
state.RunStory(results)
File "/b/s/w/irhu4lH5/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
return func(*args, **kwargs)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 299, in RunStory
self._current_page.Run(self)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/page/__init__.py", line 111, in Run
self.RunPageInteractions(action_runner)
File "/b/s/w/irhu4lH5/tools/perf/page_sets/system_health/system_health_story.py", line 112, in RunPageInteractions
self._DidLoadDocument(action_runner)
File "/b/s/w/irhu4lH5/tools/perf/page_sets/system_health/browsing_stories.py", line 80, in _DidLoadDocument
self._NavigateBack(action_runner)
File "/b/s/w/irhu4lH5/tools/perf/page_sets/system_health/browsing_stories.py", line 49, in _NavigateBack
action_runner.NavigateBack()
File "/b/s/w/irhu4lH5/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
return func(*args, **kwargs)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 183, in NavigateBack
self.ExecuteJavaScript2('window.history.back()')
File "/b/s/w/irhu4lH5/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
return func(*args, **kwargs)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 216, in ExecuteJavaScript2
return self._tab.ExecuteJavaScript2(*args, **kwargs)
File "/b/s/w/irhu4lH5/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
return func(*args, **kwargs)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 152, in ExecuteJavaScript2
return self._inspector_backend.ExecuteJavaScript2(*args, **kwargs)
File "/b/s/w/irhu4lH5/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
return func(*args, **kwargs)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 37, in inner
return func(inspector_backend, *args, **kwargs)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 216, in ExecuteJavaScript2
self._runtime.Execute(statement, context_id, timeout)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_runtime.py", line 21, in Execute
self.Evaluate(expr + '; 0;', context_id, timeout)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_runtime.py", line 45, in Evaluate
res = self._inspector_websocket.SyncRequest(request, timeout)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 110, in SyncRequest
res = self._Receive(timeout)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 166, in _Receive
self._HandleNotification(result)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 179, in _HandleNotification
self._domain_handlers[domain_name](result)
File "/b/s/w/irhu4lH5/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
return func(*args, **kwargs)
File "/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 456, in _HandleInspectorDomainNotification
raise exception
DevtoolsTargetCrashException: Devtools target crashed
********************************************************************************
(/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py:509 _AddDebuggingInformation) Received a socket error in the browser connection and the tab no longer exists. The tab probably crashed.
********************************************************************************
(/b/s/w/irhu4lH5/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py:510 _AddDebuggingInformation) Debugger url: ws://127.0.0.1:36850/devtools/page/212b6012-aeb5-444f-8364-5c35ead4f0c8
Found Minidump: True
Stack Trace:
********************************************************************************
Operating system: Linux
0.0.0 Linux 3.13.0-46-generic #79~precise1-Ubuntu SMP Tue Mar 10 20:25:33 UTC 2015 x86_64
CPU: amd64
family 6 model 62 stepping 4
8 CPUs

GPU: UNKNOWN

Crash reason: SIGSEGV
Crash address: 0x0
Process uptime: not available

Thread 0 (crashed)
0 chrome!StringCharacterPosition [objects-inl.h : 3987 + 0x0]
rax = 0x6a00001e9b7c2023 rdx = 0x00007f9c7b91e2c0
rcx = 0x000007d6367c6088 rbx = 0x0000000000000000
rsi = 0x0000000000000000 rdi = 0x0000161a1b068191
rbp = 0x00007fffd6666b58 rsp = 0x00007fffd6666a60
r8 = 0x00000000000003ff r9 = 0x000007d6367a4000
r10 = 0x0000000000000008 r11 = 0x00007f9c75306f9f
r12 = 0x0000000000000000 r13 = 0x00007fffd6666b18
r14 = 0x00007fffd6666b50 r15 = 0x00007fffd6666b68
rip = 0x00007f9c7d0cd57a
Found by: given as instruction pointer in context
1 chrome!CheckStackGuardState [regexp-macro-assembler.cc : 204 + 0x9]
rbx = 0x0000000000000015 rbp = 0x00007fffd6666b58
rsp = 0x00007fffd6666a70 r12 = 0x0000000000000000
r13 = 0x00007fffd6666b18 r14 = 0x00007fffd6666b50
r15 = 0x00007fffd6666b68 rip = 0x00007f9c7d0cd75e
Found by: call frame info
2 chrome!CheckStackGuardState [regexp-macro-assembler-x64.cc : 1222 + 0xb]
rbx = 0x00000abc3f0778c0 rbp = 0x00007fffd6666b70
rsp = 0x00007fffd6666b00 r12 = 0x0000161a1b0681bd
r13 = 0x0000161a1b0681a8 r14 = 0x000019797d778839
r15 = 0x000007d636754bf4 rip = 0x00007f9c7d30b90e
Found by: call frame info
3 0xabc3f077cc7
rbx = 0x00000abc3f0778c0 rbp = 0x00007fffd6666b70
rsp = 0x00007fffd6666b20 r12 = 0x0000161a1b0681bd
r13 = 0x0000161a1b0681a8 r14 = 0x000019797d778839
r15 = 0x000007d636754bf4 rip = 0x00000abc3f077cc7
Found by: call frame info
4 chrome!Execute [regexp-macro-assembler.cc : 283 + 0x1d]
rsp = 0x00007fffd6666b80 rip = 0x00007f9c7d0cdabf
Found by: stack scanning
5 chrome!Match [regexp-macro-assembler.cc : 257 + 0x11]
rbx = 0x0000000000000004 rbp = 0x0000000000000000
rsp = 0x00007fffd6666be0 r12 = 0x0000000000000015
r13 = 0x000007d63674f000 r14 = 0x0000161a1b068191
r15 = 0x0000000000000000 rip = 0x00007f9c7d0cd96c
Found by: call frame info
6 chrome!IrregexpExecRaw [jsregexp.cc : 500 + 0x17]
rbx = 0x0000000000000000 rbp = 0x00007fffd6666d78
rsp = 0x00007fffd6666c40 r12 = 0x000007d63674f000
r13 = 0x00007fffd6666d80 r14 = 0x000007d6367c6070
r15 = 0x00000abc3f077841 rip = 0x00007f9c7d0b43c4
Found by: call frame info
7 chrome!IrregexpExec [jsregexp.cc : 592 + 0x15]
rbx = 0x000007d63674f000 rbp = 0x000007d636754bf4
rsp = 0x00007fffd6666c90 r12 = 0x00007fffd6666d78
r13 = 0x00007fffd6666d80 r14 = 0x00007fffd6666cf8
r15 = 0x0000000000000004 rip = 0x00007f9c7d0b2925
Found by: call frame info
8 chrome!Exec [jsregexp.cc : 198 + 0x8]
rbx = 0x00007fffd6666cf8 rbp = 0x000007d636756000
rsp = 0x00007fffd6666ce0 r12 = 0x000007d63676c000
r13 = 0x000007d63674f0c8 r14 = 0x000007d63674f000
r15 = 0x00007fffd6666d80 rip = 0x00007f9c7d0b2356
Found by: call frame info
9 chrome!__RT_impl_Runtime_RegExpExec [runtime-regexp.cc : 822 + 0xb]
rbx = 0x00007fffd6666d78 rbp = 0x000007d636756000
rsp = 0x00007fffd6666cf0 r12 = 0x000007d63676c000
r13 = 0x000007d63674f0c8 r14 = 0x000007d63674f000
r15 = 0x00007fffd6666d80 rip = 0x00007f9c7d170e15
Found by: call frame info
10 0xabc3df0426e
rbx = 0x00007f9c7d170960 rbp = 0x00007fffd6666d58
rsp = 0x00007fffd6666d40 r12 = 0x000007d63676c000
r13 = 0x000007d63674f0c8 r14 = 0x0000000000000004
r15 = 0x00007fffd6666d80 rip = 0x00000abc3df0426e
Found by: call frame info
11 0xabc3df9519c
rbp = 0x00007fffd6666e20 rsp = 0x00007fffd6666d68
rip = 0x00000abc3df9519c
Found by: previous frame's frame pointer
12 0xabc3e7964f6
rbp = 0x00007fffd6666e60 rsp = 0x00007fffd6666e30
rip = 0x00000abc3e7964f6
Found by: previous frame's frame pointer
13 0xabc3f0774e5
rbp = 0x00007fffd6666ea8 rsp = 0x00007fffd6666e70
rip = 0x00000abc3f0774e5
Found by: previous frame's frame pointer
14 0xabc3e62eb5f
rbp = 0x00007fffd6666f18 rsp = 0x00007fffd6666eb8
rip = 0x00000abc3e62eb5f
Found by: previous frame's frame pointer
15 0xabc3f7e4479
rbp = 0x00007fffd6666f58 rsp = 0x00007fffd6666f28
rip = 0x00000abc3f7e4479
Found by: previous frame's frame pointer
16 0xabc3df06555
rbp = 0x00007fffd6666f88 rsp = 0x00007fffd6666f68
rip = 0x00000abc3df06555
Found by: previous frame's frame pointer
17 0xabc3e62e1f2
rbp = 0x00007fffd6666fd0 rsp = 0x00007fffd6666f98
rip = 0x00000abc3e62e1f2
Found by: previous frame's frame pointer
18 0xabc3f785687
rbp = 0x00007fffd6667028 rsp = 0x00007fffd6666fe0
rip = 0x00000abc3f785687
Found by: previous frame's frame pointer
19 0xabc3f7866de
rbp = 0x00007fffd6667088 rsp = 0x00007fffd6667038
rip = 0x00000abc3f7866de
Found by: previous frame's frame pointer
20 0xabc3f784564
rbp = 0x00007fffd66670c0 rsp = 0x00007fffd6667098
rip = 0x00000abc3f784564
Found by: previous frame's frame pointer
21 0xabc3df06555
rbp = 0x00007fffd66670f0 rsp = 0x00007fffd66670d0
rip = 0x00000abc3df06555
Found by: previous frame's frame pointer
22 0xabc3f5aca28
rbp = 0x00007fffd6667128 rsp = 0x00007fffd6667100
rip = 0x00000abc3f5aca28
Found by: previous frame's frame pointer
23 0xabc3dfcbec3
rbp = 0x00007fffd6667158 rsp = 0x00007fffd6667138
rip = 0x00000abc3dfcbec3
Found by: previous frame's frame pointer
24 0xabc3df34981
rbp = 0x00007fffd66671c0 rsp = 0x00007fffd6667168
rip = 0x00000abc3df34981
Found by: previous frame's frame pointer
25 chrome!Invoke [execution.cc : 144 + 0x15]
rsp = 0x00007fffd66671d0 rip = 0x00007f9c7ce642fb
Found by: stack scanning
26 chrome!CallInternal [execution.cc : 180 + 0x1e]
rbx = 0x000007d63674f068 rbp = 0x000007d63674f000
rsp = 0x00007fffd6667280 r12 = 0x000007d6367c6008
r13 = 0x00007fffd6667308 r14 = 0x0000000000000000
r15 = 0x0000000000000000 rip = 0x00007f9c7ce63c91
Found by: call frame info
27 chrome!v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) + 0x14
rbx = 0x00007fffd6667308 rbp = 0x0000000000000000
rsp = 0x00007fffd66672e0 r12 = 0x000007d6367c6028
r13 = 0x000007d6367c6008 r14 = 0x0000000000000000
r15 = 0x0000000000000004 rip = 0x00007f9c7ce63bb4
Found by: call frame info
28 chrome!Call [api.cc : 5101 + 0x14]
rbx = 0x000007d63674f000 rbp = 0x0000000000000000
rsp = 0x00007fffd6667300 r12 = 0x000007d6367c6028
r13 = 0x000007d6367c6008 r14 = 0x0000000000000000
r15 = 0x0000000000000004 rip = 0x00007f9c7ca9c6ed
Found by: call frame info
29 chrome!callFunction [V8ScriptRunner.cpp : 650 + 0x16]
rbx = 0x000015baa136a9c0 rbp = 0x000007d6367c6008
rsp = 0x00007fffd66673e0 r12 = 0x0000000000000000
r13 = 0x000007d6367c6028 r14 = 0x000007d63674f000
r15 = 0x0000000000000000 rip = 0x00007f9c80695b09
Found by: call frame info
30 chrome!execute [ScheduledAction.cpp : 135 + 0xe]
rbx = 0x000015baa136a9c0 rbp = 0x000015baa136a9c0
rsp = 0x00007fffd6667610 r12 = 0x00003e2e5a1f8248
r13 = 0x00007fffd66681b0 r14 = 0x00002274a7b61f50
r15 = 0x000007d6367c6008 rip = 0x00007f9c82b05645

Detected 23 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 31 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 23 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 17 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 33 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 36 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 58 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
According to the flakiness dashboard, most of the new flakes are due to benchmarks/system_health_smoke_test/SystemHealthBenchmarkSmokeTest/system_health/memory_mobile/browse:news:washingtonpost. 

Track the disabling & fixing it in issue 688190
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 12 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 10 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Cc: -stip@chromium.org
Detected 12 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 20 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 18 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 6 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 11 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 14 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 9 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 10 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 11 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 12 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 194 by chromium...@appspot.gserviceaccount.com, Apr 19 (6 days ago)
Detected 8 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Comment 195 by nedngu...@google.com, Apr 19 (6 days ago)
Blockedon: 713048
#194 is tracked in issue 713048
Project Member Comment 196 by chromium...@appspot.gserviceaccount.com, Apr 21 (4 days ago)
Detected 3 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 197 by chromium...@appspot.gserviceaccount.com, Apr 22 (3 days ago)
Detected 5 new flakes for test/step "telemetry_perf_unittests (with patch)". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMAsSBUZsYWtlIiV0ZWxlbWV0cnlfcGVyZl91bml0dGVzdHMgKHdpdGggcGF0Y2gpDA. This message was posted automatically by the chromium-try-flakes app.
Sign in to add a comment