New issue
Advanced search Search tips

Issue 693078 link

Starred by 3 users
Status: Duplicate
Merged: issue 725516
Owner: ----
Closed: Jun 2017
Components:
EstimatedDays: ----
NextAction: 2017-06-24
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

telemetry_perf_unittests is super flaky on Linux due to cloud storage lock failure

Project Member Reported by nedngu...@google.com, Feb 16 2017 
Flakiness dashboard:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=telemetry_perf_unittests&builder=tryserver.chromium.linux%3Alinux_chromium_rel_ng


An example run log:
https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_rel_ng/builds/389926/steps/telemetry_perf_unittests%20%28with%20patch%29/logs/stdio

Assign to Charlie since you last work on this. I think if it's too hard to diagnose, maybe we should just run this test in serial on linux bot & feel free to reassign to me in that case.

Comment 1 by charliea@chromium.org, Feb 16 2017 

Stack trace from log:

  Traceback (most recent call last):
    RunBenchmark at /b/s/w/irUW2HFD/third_party/catapult/telemetry/telemetry/internal/story_runner.py:380
      benchmark.ShouldTearDownStateAfterEachStorySetRun())
    Run at /b/s/w/irUW2HFD/third_party/catapult/telemetry/telemetry/internal/story_runner.py:220
      story_set.archive_data_file, story_set.wpr_archive_info, stories):
    _UpdateAndCheckArchives at /b/s/w/irUW2HFD/third_party/catapult/telemetry/telemetry/internal/story_runner.py:420
      wpr_archive_info.DownloadArchivesIfNeeded()
    DownloadArchivesIfNeeded at /b/s/w/irUW2HFD/third_party/catapult/telemetry/telemetry/wpr/archive_info2.py:110
      download_if_needed(archive_path)
    download_if_needed at /b/s/w/irUW2HFD/third_party/catapult/telemetry/telemetry/wpr/archive_info2.py:89
      cloud_storage.GetIfChanged(path, self._bucket)
    GetIfChanged at /b/s/w/irUW2HFD/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:406
      with _FileLock(file_path):
    __enter__ at /usr/lib/python2.7/contextlib.py:17
      return self.gen.next()
    _FileLock at /b/s/w/irUW2HFD/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:259
      LOCK_ACQUISITION_TIMEOUT)
    WaitFor at /b/s/w/irUW2HFD/third_party/catapult/common/py_utils/py_utils/__init__.py:132
      (timeout, GetConditionString()))
  TimeoutException: Timed out while waiting 10s for py_utils.WaitFor(lambda: _AttemptPseudoLockAcquisition(pseudo_lock_path,
                                                           pseudo_lock_fd_return),

Comment 2 by charliea@chromium.org, Feb 16 2017 

Also, note that this is only responsible for some of the telemetry_perf_unittests flakes. Chrome crashes are also responsible for some. (Obviously, this should still be fixed, though.)

Comment 3 by charliea@chromium.org, Jun 22 2017

NextAction: 2017-06-24
Owner: ----
Status: (was: Assigned)
Ned, is this fixed now?

Comment 4 by nedngu...@google.com, Jun 22 2017

Mergedinto: 725516
Status: Duplicate

Comment 5 by monor...@bugs.chromium.org, Jun 24 2017 

The NextAction date has arrived: 2017-06-24
►

Sign in to add a comment