New issue
Advanced search Search tips

Issue 704321 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Closed: Apr 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug
som

Blocked on:
issue 639901
issue 704600



Sign in to add a comment

Enable sheriff-o-matic for the chromeos pfq-informational builders

Project Member Reported by steve...@chromium.org, Mar 22 2017

Issue description

Forking from issue 639901

We would like to add a PFQ Informational section to sheriff-o-matic tracking the following waterfalls:

(public - chromium) https://build.chromium.org/p/chromiumos.chromium/waterfall
(internal - chrome) https://uberchromegw.corp.google.com/i/chromeos.chrome/waterfall

The most important builders are the ones ending in -pfq-informational

This could either be a subsection of Chromeos, or one or two separate top level sections, what ever is the most straightforward to implement. 

(If it is all the same, a single 'Pfq-informational' top level section would be preferred).


 
From issue 639901#51:

You'll also need to get dnj@ to turn on Logdog for some of these builds like peach_pit-tot-chrome-pfq-informational.

Will need seanmcullough@ to create the new tree for you.

I need to land some changes I just made to:
- allow the dispatcher to generate alerts for non-important builds
- allow the dispatcher to send alerts to another tree than 'chromeos'
- specify this full set of builds to explictly watch

akeshet@/dgarrett@ to find an appropriate time and place to call this different invocation of som_alerts_dispatcher.  We might just be able to call it after each of those builds finishing in the report stage -- maybe the load of running it 13 times isn't that bad since it's just looking at 13 builds each of those times.  This will get better once we allow individuals build failures to be posted.

My quick changes and manually running them results in: http://davidriley-lab.mtv.corp.google.com:8080/pfq

seanmccullough/dnj: Do you guys need separate bugs for this, or is this fine?

I don't think we should merge it with chromeos, so I think we should create a new tree "PFQ Informational" as per stevenjb@.  It won't be an issue having builds from both waterfalls end up on the same tree.
I'd like to lobby for "Chrome PFQ Informational" or something like that. PFQ is an overloaded term (see also Android PFQ).
To keep things reasonably short, how about one 'Informational' section with clearly named sub sections?

* Chrome OS Informational
** Chromium PFQ Informational
** Chrome PFQ Informational

Then if we have one we can add e.g. Android PFQ Informational builders we can put them there.

I'm not quite sure what you mean by section and subsections.

It's currently divided into trees (eg chromeos, chromium, trooper) which are basically split based on whom is responsible for watching over that tree.  Within that tree the alerts are divided into different categories/severities based on the order that they should be triaged/investigated.

I suggest one new tree, either "Gardener" or "Chromium PFQ Informational".  Within it, we can have all the alerts be the same or different severities.  I'd probably say have one for PFQ Failures and one for Informational.
"secton" = "tree". 

"Chrome OS Gardener" would be fine, but "Chrome OS Informational" might be better since these are a subset of what the gardener needs to look at (the subset is more or less exclusive to the gardener, but the gardener also needs to pay attention to Chrome OS/PFQ and Chromium/anything cros related).

"subsection" = "category"

* 'Chromium PFQ Informaitonal' = https://build.chromium.org/p/chromiumos.chromium/waterfall
* 'Chrome PFQ Informational' = https://uberchromegw.corp.google.com/i/chromeos.chrome/waterfall

Blockedon: 704600
Regarding the categories, to confirm that Chromium should appear before Chrome then?

Given that the subset is exclusive to the gardener, I think Gardener Informational is more inline with the other trees, and wouldn't confuse people following the Chrome OS trees?

If I could move the current PFQ builds (children of master-chromium-pfq) to this new tree, would that be desirable?  It would mean that sheriffs would no longer look at them (assuming they're using sheriff-o-matic as the basis of their investigations).
I don't really care much about the ordering, but yeah that sounds good.

"Gardener Informational" is good.

If we move the PFQ builders there, which SGTM, we could then just call it "Gardener". 

(The builders that gardeners look at on the Chromium tree are also monitored by the Chrome sheriffs and failures are rare since we have tryserver coverage and generally result in immediate reverts).

Cheers!

Project Member

Comment 10 by bugdroid1@chromium.org, Mar 24 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/5de2a731af4b0a1c59d83565b6aa4f72b024b031

commit 5de2a731af4b0a1c59d83565b6aa4f72b024b031
Author: David Riley <davidriley@chromium.org>
Date: Fri Mar 24 19:24:00 2017

[som] Add Gardener PFQ severities.

BUG= 704321 

Change-Id: I09d9368d7037e980702894d4f6ecbe22327f4d4f
Reviewed-on: https://chromium-review.googlesource.com/459066
Reviewed-by: Sean McCullough <seanmccullough@chromium.org>
Commit-Queue: Sean McCullough <seanmccullough@chromium.org>

[modify] https://crrev.com/5de2a731af4b0a1c59d83565b6aa4f72b024b031/go/src/infra/appengine/sheriff-o-matic/elements/som-app/som-app.js

Project Member

Comment 11 by bugdroid1@chromium.org, Mar 24 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/5de2a731af4b0a1c59d83565b6aa4f72b024b031

commit 5de2a731af4b0a1c59d83565b6aa4f72b024b031
Author: David Riley <davidriley@chromium.org>
Date: Fri Mar 24 19:24:00 2017

[som] Add Gardener PFQ severities.

BUG= 704321 

Change-Id: I09d9368d7037e980702894d4f6ecbe22327f4d4f
Reviewed-on: https://chromium-review.googlesource.com/459066
Reviewed-by: Sean McCullough <seanmccullough@chromium.org>
Commit-Queue: Sean McCullough <seanmccullough@chromium.org>

[modify] https://crrev.com/5de2a731af4b0a1c59d83565b6aa4f72b024b031/go/src/infra/appengine/sheriff-o-matic/elements/som-app/som-app.js

Project Member

Comment 12 by bugdroid1@chromium.org, Mar 24 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/5de2a731af4b0a1c59d83565b6aa4f72b024b031

commit 5de2a731af4b0a1c59d83565b6aa4f72b024b031
Author: David Riley <davidriley@chromium.org>
Date: Fri Mar 24 19:24:00 2017

[som] Add Gardener PFQ severities.

BUG= 704321 

Change-Id: I09d9368d7037e980702894d4f6ecbe22327f4d4f
Reviewed-on: https://chromium-review.googlesource.com/459066
Reviewed-by: Sean McCullough <seanmccullough@chromium.org>
Commit-Queue: Sean McCullough <seanmccullough@chromium.org>

[modify] https://crrev.com/5de2a731af4b0a1c59d83565b6aa4f72b024b031/go/src/infra/appengine/sheriff-o-matic/elements/som-app/som-app.js

Project Member

Comment 13 by bugdroid1@chromium.org, Mar 31 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/c2a28eb1dc405864a6dcdbb9fcced3307803156a

commit c2a28eb1dc405864a6dcdbb9fcced3307803156a
Author: David Riley <davidriley@chromium.org>
Date: Fri Mar 31 04:21:58 2017

som: Add support for gardener tree.

Add support for Sheriff-o-Matic for PFQ by allowing alerts
to be generated for experimental builds and posted to a
different tree.  Move PFQ builds to new gardener tree.

BUG= chromium:704321 
TEST=som_alerts_dispatcher --allow_experimental --som_tree gardener CREDS_DIR

Change-Id: Ieb89463528c58fb01bdaf819ab65d082924c0bf4
Reviewed-on: https://chromium-review.googlesource.com/457845
Commit-Ready: David Riley <davidriley@chromium.org>
Tested-by: David Riley <davidriley@chromium.org>
Reviewed-by: David Riley <davidriley@chromium.org>

[modify] https://crrev.com/c2a28eb1dc405864a6dcdbb9fcced3307803156a/lib/som.py
[modify] https://crrev.com/c2a28eb1dc405864a6dcdbb9fcced3307803156a/lib/constants.py
[modify] https://crrev.com/c2a28eb1dc405864a6dcdbb9fcced3307803156a/scripts/som_alerts_dispatcher.py
[modify] https://crrev.com/c2a28eb1dc405864a6dcdbb9fcced3307803156a/cbuildbot/stages/report_stages_unittest.py
[modify] https://crrev.com/c2a28eb1dc405864a6dcdbb9fcced3307803156a/cbuildbot/stages/report_stages.py

I broke SoM with my change in c#13.  Currently manually generating alerts for both chromeos and gardener trees, so once the fix (https://chromium-review.googlesource.com/c/465506/) lands, I think the gardener tree should be up and good.  (It should be up and going right now through my manual dispatching if any gardeners want to take a look).
Labels: som
Status: Fixed (was: Assigned)
This should be alive now via go/som/gardener.

@stevenjb: Can you share this with the gardeners?

Comment 16 by dchan@google.com, May 30 2017

Labels: VerifyIn-60
Labels: VerifyIn-61

Comment 18 by dchan@chromium.org, Jan 22 2018

Status: Archived (was: Fixed)

Sign in to add a comment