Process swarming result pipeline callback failures |
|||
Issue descriptionhttps://pantheon.corp.google.com/errors/16044591379238181608?time=P30D&filter&project=findit-for-me The root cause seems to be similar to the one for bug 700498 Traceback (most recent call last): File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__ rv = self.handle_exception(request, response, e) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__ rv = self.router.dispatch(request, response) File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/gae_ts_mon/config.py", line 257, in dispatch time_fn=time_fn) File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/gae_ts_mon/config.py", line 219, in _instrumented_dispatcher ret = dispatcher(request, response) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher return route.handler_adapter(request, response) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__ return handler.dispatch() File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch return self.handle_exception(e, self.app.debug) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch return method(*args, **kwargs) File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py", line 2786, in get self.run_callback() File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py", line 2863, in run_callback callback_result = perform_callback() File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py", line 2853, in perform_callback return stage._callback_internal(kwargs) File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/first_party/pipeline_utils/appengine_third_party_pipeline_python_src_pipeline/pipeline.py", line 1099, in _callback_internal return self.callback(**kwargs) File "/base/data/home/apps/s~findit-for-me/waterfall-backend:ea9e1af37.399984549731806797/waterfall/process_base_swarming_task_result_pipeline.py", line 98, in callback task = self._GetSwarmingTask(*call_args) TypeError: _GetSwarmingTask() takes exactly 5 arguments (75 given)
,
Mar 21 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/d04f00e7b37e86c42176889917e4766f9732f959 commit d04f00e7b37e86c42176889917e4766f9732f959 Author: Roberto Carrillo <robertocn@google.com> Date: Tue Mar 21 22:25:48 2017 [Findit] Callback now accepts a dict or its json. Instead of doing complex casting sequences, all parameters for the callback are taken from a single dictionary param that can be json-serialized ahead of converting the callback to a url. R=chanli,stgao,lijeffrey BUG= 703761 Change-Id: I261c05f19833fef5f81896d48a32ada0238f42db Reviewed-on: https://chromium-review.googlesource.com/457479 Reviewed-by: Chan Li <chanli@chromium.org> Reviewed-by: Jeffrey Li <lijeffrey@chromium.org> Commit-Queue: Roberto Carrillo <robertocn@chromium.org> [modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/test/process_swarming_task_result_pipeline_test.py [modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/monitor_try_job_pipeline.py [modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/process_base_swarming_task_result_pipeline.py [modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/test/process_base_swarming_task_result_pipeline_test.py [modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/test/monitor_try_job_pipeline_test.py [modify] https://crrev.com/d04f00e7b37e86c42176889917e4766f9732f959/appengine/findit/waterfall/test/process_flake_swarming_task_result_pipeline_test.py
,
Mar 23 2017
This did not solve the problem. 500s were thrown this morning, requiring a rollback. https://pantheon.corp.google.com/errors/1343419634688374971?time=P30D&filter&project=findit-for-me
,
Mar 24 2017
,
Mar 24 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/4bd07b5148508d3d100d9d4eafb9f4d233d7fa15 commit 4bd07b5148508d3d100d9d4eafb9f4d233d7fa15 Author: Roberto Carrillo <robertocn@google.com> Date: Fri Mar 24 01:07:39 2017 [Findit] Remove unnecessary datetime callback parameter The parameter does not need to be passed between calls. R=chanli,stgao,lijeffrey BUG= 703761 Change-Id: I40bc1e744806f2c4aadf0ce6609aaa61b4019fa7 Reviewed-on: https://chromium-review.googlesource.com/458107 Reviewed-by: Chan Li <chanli@chromium.org> Commit-Queue: Roberto Carrillo <robertocn@chromium.org> [modify] https://crrev.com/4bd07b5148508d3d100d9d4eafb9f4d233d7fa15/appengine/findit/waterfall/monitor_try_job_pipeline.py [modify] https://crrev.com/4bd07b5148508d3d100d9d4eafb9f4d233d7fa15/appengine/findit/libs/time_util.py [modify] https://crrev.com/4bd07b5148508d3d100d9d4eafb9f4d233d7fa15/appengine/findit/waterfall/test/monitor_try_job_pipeline_test.py
,
Mar 30 2017
|
|||
►
Sign in to add a comment |
|||
Comment 1 by robert...@chromium.org
, Mar 21 2017