New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 703761 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Mar 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocking:
issue 703294



Sign in to add a comment

Process swarming result pipeline callback failures

Project Member Reported by robert...@chromium.org, Mar 21 2017

Issue description

https://pantheon.corp.google.com/errors/16044591379238181608?time=P30D&filter&project=findit-for-me


The root cause seems to be similar to the one for  bug 700498 


Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/gae_ts_mon/config.py", line 257, in dispatch
    time_fn=time_fn)
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/gae_ts_mon/config.py", line 219, in _instrumented_dispatcher
    ret = dispatcher(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py", line 2786, in get
    self.run_callback()
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py", line 2863, in run_callback
    callback_result = perform_callback()
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py", line 2853, in perform_callback
    return stage._callback_internal(kwargs)
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py", line 1099, in _callback_internal
    return self.callback(**kwargs)
  File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/waterfall/process_base_swarming_task_result_pipeline.py", line 98, in callback
    task = self._GetSwarmingTask(*call_args)
TypeError: _GetSwarmingTask() takes exactly 5 arguments (75 given)

 
Status: Started (was: Untriaged)
Project Member

Comment 2 by bugdroid1@chromium.org, Mar 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/d04f00e7b37e86c42176889917e4766f9732f959

commit d04f00e7b37e86c42176889917e4766f9732f959
Author: Roberto Carrillo <robertocn@google.com>
Date: Tue Mar 21 22:25:48 2017

[Findit] Callback now accepts a dict or its json.

Instead of doing complex casting sequences, all parameters for
the callback are taken from a single dictionary param that can
be json-serialized ahead of converting the callback to a url.

R=chanli,stgao,lijeffrey
BUG= 703761 

Change-Id: I261c05f19833fef5f81896d48a32ada0238f42db
Reviewed-on: https://chromium-review.googlesource.com/457479
Reviewed-by: Chan Li <chanli@chromium.org>
Reviewed-by: Jeffrey Li <lijeffrey@chromium.org>
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>

[modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/test/process_swarming_task_result_pipeline_test.py
[modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/monitor_try_job_pipeline.py
[modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/process_base_swarming_task_result_pipeline.py
[modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/test/process_base_swarming_task_result_pipeline_test.py
[modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/test/monitor_try_job_pipeline_test.py
[modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/test/process_flake_swarming_task_result_pipeline_test.py

This did not solve the problem. 500s were thrown this morning, requiring a rollback.
https://pantheon.corp.google.com/errors/1343419634688374971?time=P30D&filter&project=findit-for-me
Blocking: 703294
Project Member

Comment 5 by bugdroid1@chromium.org, Mar 24 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/4bd07b5148508d3d100d9d4eafb9f4d233d7fa15

commit 4bd07b5148508d3d100d9d4eafb9f4d233d7fa15
Author: Roberto Carrillo <robertocn@google.com>
Date: Fri Mar 24 01:07:39 2017

[Findit] Remove unnecessary datetime callback parameter

The parameter does not need to be passed between calls.

R=chanli,stgao,lijeffrey
BUG= 703761 

Change-Id: I40bc1e744806f2c4aadf0ce6609aaa61b4019fa7
Reviewed-on: https://chromium-review.googlesource.com/458107
Reviewed-by: Chan Li <chanli@chromium.org>
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>

[modify] https://crrev.com/4bd07b5148508d3d100d9d4eafb9f4d233d7fa15/appengine/findit/waterfall/monitor_try_job_pipeline.py
[modify] https://crrev.com/4bd07b5148508d3d100d9d4eafb9f4d233d7fa15/appengine/findit/libs/time_util.py
[modify] https://crrev.com/4bd07b5148508d3d100d9d4eafb9f4d233d7fa15/appengine/findit/waterfall/test/monitor_try_job_pipeline_test.py

Status: Fixed (was: Started)

Sign in to add a comment