New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 687521 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner: ----
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 1
Type: Bug



Sign in to add a comment

Fix bugdroid's monitoring

Project Member Reported by dsansome@chromium.org, Feb 1 2017

Issue description

A counter of attempted and succeeded commits, and maybe some other good stuff like processing time if you feel like it.

Add alerts to detect Issue 687515 before our users do.
 
Components: -Infra Infra>Platform
Cc: aga...@chromium.org
Owner: dsansome@chromium.org
Status: Assigned (was: Unconfirmed)
Summary: Fix bugdroid's monitoring (was: Add monitoring to bugdroid)
Bugdroid already has basic monitoring. It uses the outer_loop library, which sends some default metrics. Unfortunately it appears that it fails to flush those default metrics every time:
https://luci-logdog.appspot.com/v/?s=infra-internal%2Fbb%2Finternal.infra.cron%2Fbugdroid%2F38282%2F%2B%2Frecipes%2Fsteps%2Fbugdroid%2F0%2Fstdout

I don't know what to do about that, since it seems to be using the same stuff as gsubtreed, which sends metrics just fine.

Once that's fixed, I'll be happy to add additional metrics for things we care about, like success/failure of parsing logs and success/failure of posting updates.

I'd noticed that error as well in some other logs :/  Looks like it's been broken for a while - weird we're only just noticing it now.  I'll fix it.
Project Member

Comment 5 by bugdroid1@chromium.org, Feb 2 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/a8d18a31ba11b7a62018326b3002e8278746ccda

commit a8d18a31ba11b7a62018326b3002e8278746ccda
Author: David Sansome <dsansome@chromium.org>
Date: Thu Feb 02 08:16:22 2017

Roll infra_libs in infra and fix outer_loop's usage of DistributionMetric

BUG= 687521 , 684667 
TBR=sergeyberezin

Change-Id: I1b57d624ed54cfc42204ef1242ba9ff8627a0583
Reviewed-on: https://chromium-review.googlesource.com/435121
Reviewed-by: Dave Sansome <dsansome@chromium.org>
Commit-Queue: Dave Sansome <dsansome@chromium.org>

[modify] https://crrev.com/a8d18a31ba11b7a62018326b3002e8278746ccda/appengine_module/gae_ts_mon/__init__.py
[modify] https://crrev.com/a8d18a31ba11b7a62018326b3002e8278746ccda/bootstrap/deps.pyl
[modify] https://crrev.com/a8d18a31ba11b7a62018326b3002e8278746ccda/infra/libs/service_utils/outer_loop.py

Owner: ----
Status: Available (was: Assigned)

Comment 7 by aga...@chromium.org, Oct 30 2017

Status: Fixed (was: Available)

Sign in to add a comment