New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 746717 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jul 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

[Findit] Flake Analyzer - Majority of flake analyses stuck due to TomstonedTaskError

Project Member Reported by lijeffrey@chromium.org, Jul 20 2017

Issue description

Generator waterfall.process_flake_swarming_task_result_pipeline.ProcessFlakeSwarmingTaskResultPipeline(*(u'chromium.webkit', u'WebKit Win x64 Builder (dbg)', 111157, u'webkit_unit_tests', u'3770d332ddc93410', 111158, u'All/ParameterizedWebFrameTest.DISABLE_ON_TSAN(TestNonCompositedOverlayScrollbarsFade)... (207 bytes), **{})#1b56f172e0aa4ebdabfe14c39f83362d raised exception. TombstonedTaskError:  (/base/data/home/apps/s~findit-for-me/waterfall-backend:10776-641ea88.402742370387407601/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py:2434)
Traceback (most recent call last):
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:10776-641ea88.402742370387407601/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py", line 2158, in evaluate
    self, pipeline_key, root_pipeline_key, caller_output)
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:10776-641ea88.402742370387407601/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py", line 1113, in _run_internal
    return self.run(*self.args, **self.kwargs)
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:10776-641ea88.402742370387407601/waterfall/process_base_swarming_task_result_pipeline.py", line 364, in run
    name=task_id + '_cleanup_task')
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:10776-641ea88.402742370387407601/waterfall/process_base_swarming_task_result_pipeline.py", line 93, in delay_callback
    task.add(queue_name=constants.WATERFALL_ANALYSIS_QUEUE)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/taskqueue/taskqueue.py", line 1213, in add
    return self.add_async(queue_name, transactional).get_result()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/taskqueue/taskqueue.py", line 2162, in ResultHook
    raise exception
TombstonedTaskError

This is affecting nearly all flake analyses for some reason.
 
Status: Fixed (was: Assigned)

Sign in to add a comment