Report swarming tasks to event_mon
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/a2d4431bc6d86f6b95d680206b177640d102ce26 commit a2d4431bc6d86f6b95d680206b177640d102ce26 Author: Paweł Hajdan, Jr <phajdan.jr@chromium.org> Date: Thu Oct 27 14:55:16 2016 Update chrome_infra_log_pb2.py to include SwarmingTaskEvent BUG= 660014 Change-Id: I6771fae7a8a77f50a8270a7e8460ef23f78de02e Reviewed-on: https://chromium-review.googlesource.com/404249 Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Commit-Queue: Paweł Hajdan Jr. <phajdan@google.com> [modify] https://crrev.com/a2d4431bc6d86f6b95d680206b177640d102ce26/packages/infra_libs/infra_libs/event_mon/protos/chrome_infra_log_pb2.py
The following revision refers to this bug: https://chromium.googlesource.com/external/github.com/luci/luci-py.git/+/62d9a940b73d2d1497d8dc07b36417be136eb8b1 commit 62d9a940b73d2d1497d8dc07b36417be136eb8b1 Author: phajdan.jr <phajdan.jr@chromium.org> Date: Tue Nov 29 15:18:04 2016 Upload basic info about completed swarming tasks to event_mon More CLs will follow to fill in more details. BUG= 660014 Review-Url: https://codereview.chromium.org/2534943002 [add] https://crrev.com/62d9a940b73d2d1497d8dc07b36417be136eb8b1/appengine/swarming/event_mon_metrics.py [add] https://crrev.com/62d9a940b73d2d1497d8dc07b36417be136eb8b1/appengine/swarming/gae_event_mon [modify] https://crrev.com/62d9a940b73d2d1497d8dc07b36417be136eb8b1/appengine/swarming/main.py [modify] https://crrev.com/62d9a940b73d2d1497d8dc07b36417be136eb8b1/appengine/swarming/server/task_scheduler.py [modify] https://crrev.com/62d9a940b73d2d1497d8dc07b36417be136eb8b1/appengine/swarming/server/task_scheduler_test.py [modify] https://crrev.com/62d9a940b73d2d1497d8dc07b36417be136eb8b1/appengine/swarming/test_env_handlers.py
The following revision refers to this bug: https://chromium.googlesource.com/external/github.com/luci/luci-py.git/+/8e285a8f793c3e98218ea994cad186f946482a7e commit 8e285a8f793c3e98218ea994cad186f946482a7e Author: phajdan.jr <phajdan.jr@chromium.org> Date: Thu Dec 08 08:45:00 2016 swarming: fill the rest of SwarmingTaskEvent proto BUG= 660014 Review-Url: https://codereview.chromium.org/2556763002 [modify] https://crrev.com/8e285a8f793c3e98218ea994cad186f946482a7e/appengine/swarming/event_mon_metrics.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/b114d0689a3cfc2b9d44972baba0e2458aeb66aa commit b114d0689a3cfc2b9d44972baba0e2458aeb66aa Author: Paweł Hajdan, Jr <phajdan.jr@chromium.org> Date: Fri Dec 09 19:26:16 2016 Update infra log protos BUG= 660014 Change-Id: I008b77f7802f4cc60b09cd24216da1eede0ff6d6 Reviewed-on: https://chromium-review.googlesource.com/418736 Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org> [modify] https://crrev.com/b114d0689a3cfc2b9d44972baba0e2458aeb66aa/packages/infra_libs/infra_libs/event_mon/protos/chrome_infra_log_pb2.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/066f135e603f90945be764884a3d2863f72d21c1 commit 066f135e603f90945be764884a3d2863f72d21c1 Author: Sergey Berezin <sergeyberezin@google.com> Date: Wed Dec 14 22:50:32 2016 Roll infra_libs in infra.git to 277501d Relevant changes: 277501d Update infra log protos f8283fa ts_mon: Continue flushing metrics beyond serialization error [2/2] 14fc8d6 ts_mon: Continue flushing metrics beyond serialization error. 4e71072 Roll infra_libs in infra.git BUG= 660014 TBR=maruel@chromium.org Change-Id: I0ed303bcb1b51f0a123c80b2d4de11b4e6b9c2f5 Reviewed-on: https://chromium-review.googlesource.com/420403 Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org> Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org> [modify] https://crrev.com/066f135e603f90945be764884a3d2863f72d21c1/bootstrap/deps.pyl
The following revision refers to this bug: https://chromium.googlesource.com/external/github.com/luci/luci-py.git/+/7ca3ff6b75b75429a75a80c7100e8cc262e98628 commit 7ca3ff6b75b75429a75a80c7100e8cc262e98628 Author: sergeyberezin <sergeyberezin@chromium.org> Date: Thu Dec 15 09:46:30 2016 Roll infra_libs and its dependencies to 066f135 Relevant commits: b114d06 Update infra log protos 23de8a9 ts_mon: Continue flushing metrics beyond serialization error [2/2] 038c15d ts_mon: Continue flushing metrics beyond serialization error. 07bcb6b Roll infra_libs in infra.git 733c409 Fix a bug where data for different targets would be added to the same MetricsDataSet. BUG= 660014 R=maruel@chromium.org,phajdan.jr@chromium.org Review-Url: https://codereview.chromium.org/2573343002 [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/appengine/third_party/gae_event_mon/README.swarming [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/appengine/third_party/gae_ts_mon/README.swarming [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/appengine/third_party/gae_ts_mon/config.py [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/client/third_party/infra_libs/README.swarming [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/client/third_party/infra_libs/event_mon/protos/chrome_infra_log_pb2.py [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/client/third_party/infra_libs/ts_mon/common/errors.py [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/client/third_party/infra_libs/ts_mon/common/interface.py [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/client/third_party/infra_libs/ts_mon/common/metric_store.py [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/client/third_party/infra_libs/ts_mon/common/metrics.py [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/client/third_party/infra_libs/ts_mon/common/monitors.py [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/client/third_party/infra_libs/ts_mon/common/pb_to_popo.py [modify] https://crrev.com/7ca3ff6b75b75429a75a80c7100e8cc262e98628/client/third_party/infra_libs/ts_mon/config.py
The following revision refers to this bug: https://chromium.googlesource.com/external/github.com/luci/luci-py.git/+/0ed8407e43d4e67200fd4b5aa4f014c0d8809065 commit 0ed8407e43d4e67200fd4b5aa4f014c0d8809065 Author: phajdan.jr <phajdan.jr@chromium.org> Date: Fri Dec 16 20:15:19 2016 swarming: send task tags to event_mon BUG= 660014 Review-Url: https://codereview.chromium.org/2584663002 [modify] https://crrev.com/0ed8407e43d4e67200fd4b5aa4f014c0d8809065/appengine/swarming/event_mon_metrics.py
Is this completed? Moved to Infra>Platform>Swarming since this is a service specific alert update. Added label "Ops-AddMonitoring" to track monitoring tasks over services.
It is completed but it seems to be having issues with issue 704907 so we can't close it yet.
There's also some concern about possibly missing data (e.g. BOT_DIED events), see https://goto.google.com/xhdhq .
I'd say "reporting" part is done. If there are issues with the pipeline, I suggest tackling them separately.
Comment 1 by bugdroid1@chromium.org
, Oct 28 2016