Is this a duplicate of https://bugs.chromium.org/p/chromium/issues/detail?id=814791 ?

no, Ned intentionally filed this one for whatever we want to do on SoM: https://bugs.chromium.org/p/chromium/issues/detail?id=814791#c17

Oh right. 

I think you can exclude this step in gatekeeper config to make SoM ignore it. https://cs.chromium.org/chromium/build/scripts/slave/gatekeeper.json?q=gatekeeper&sq=package:chromium&l=493

I'd rather not have SoM try to extract any information from step names or do any other special case filtering.

The point here is that it'd be nice to be able to mark steps as experimental in src w/o needing a corresponding change to gatekeeper.json every time we do so.

Are gatekeeper and SoM the only two apps that should ignore these steps or are there others?

Ping. This is still coming up and alerting the sheriffs. 

jbudorick, nednguyen: please propose where you want to store these "ignore this step" flags? This is blocked on that.

AFAIK gatekeeper makes the most sense. We aren't going to parse your source code or try to extract this from text in step names.

I don't know much about what data format we send to the test results server, so deferring this to John

i'll exclude telemetry_perf_unittests on gatekeeper for now, but I don't think the maintenance burden of handling these in gatekeeper is acceptable long-term.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/28fec8a7ceb0ba2db1d04e5dda71cbb696ebd64a

commit 28fec8a7ceb0ba2db1d04e5dda71cbb696ebd64a
Author: John Budorick <jbudorick@chromium.org>
Date: Thu Mar 22 22:19:35 2018

Exclude currently experimental tests from gatekeeper.

Bug:  815006 
Change-Id: I8b6e27e017ca962d9b8052b195894866de7212f5
Reviewed-on: https://chromium-review.googlesource.com/976649
Reviewed-by: Sean McCullough <seanmccullough@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>

[modify] https://crrev.com/28fec8a7ceb0ba2db1d04e5dda71cbb696ebd64a/scripts/slave/unittests/gatekeeper_ng_test.py
[modify] https://crrev.com/28fec8a7ceb0ba2db1d04e5dda71cbb696ebd64a/scripts/slave/gatekeeper.json
[modify] https://crrev.com/28fec8a7ceb0ba2db1d04e5dda71cbb696ebd64a/scripts/slave/gatekeeper_ng.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/570e82a588399004ddb2dcf5308b527bdc5e8346

commit 570e82a588399004ddb2dcf5308b527bdc5e8346
Author: John Budorick <jbudorick@chromium.org>
Date: Fri Mar 23 16:21:39 2018

gatekeeper: add experimental tests to all chromium tree closers

Bug:  815006 
Change-Id: If175a437ad28b5b33039cc439fd0f2593e0f4641
Reviewed-on: https://chromium-review.googlesource.com/977895
Reviewed-by: Sean McCullough <seanmccullough@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>

[modify] https://crrev.com/570e82a588399004ddb2dcf5308b527bdc5e8346/scripts/slave/gatekeeper.json

This is still showing up on SoM https://screenshot.googleplex.com/czZNcmPOvbv.png

 Issue 849533  has been merged into this issue.

updating SoM's handling of gatekeeper configs in https://chromium-review.googlesource.com/c/infra/infra/+/1119106

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/6babb8cb91aca3fd51b616214b6efcba69a75330

commit 6babb8cb91aca3fd51b616214b6efcba69a75330
Author: John Budorick <jbudorick@chromium.org>
Date: Mon Jul 02 18:23:24 2018

Fix som handling of gatekeeper glob-based exclusions.

Bug:  815006 
Change-Id: Idd26b2d2fdccd47fae1eebf1bacd5a198d4ec042
Reviewed-on: https://chromium-review.googlesource.com/1119106
Reviewed-by: Sean McCullough <seanmccullough@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>

[modify] https://crrev.com/6babb8cb91aca3fd51b616214b6efcba69a75330/go/src/infra/appengine/sheriff-o-matic/som/analyzer/analyzer_test.go
[modify] https://crrev.com/6babb8cb91aca3fd51b616214b6efcba69a75330/go/src/infra/appengine/sheriff-o-matic/som/analyzer/gatekeeper.go

Fix is live in SoM as of today.

 Issue 833669  has been merged into this issue.

 Issue 868535  has been merged into this issue.

Fix seems to not be effective.

What makes you believe that? The two bugs you just duped in are from a while ago, and might not still be active?

https://bugs.chromium.org/p/chromium/issues/detail?id=868535 is from july 27 (after #18), and https://bugs.chromium.org/p/chromium/issues/detail?id=876193 was duped into it today.

Seems that SoM ignores categories, including things like experimental_tests.

teaching SoM about categories in https://chromium-review.googlesource.com/c/infra/infra/+/1184235

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/4d90c8b790a3ea712967e5462d7b85793b1fdb95

commit 4d90c8b790a3ea712967e5462d7b85793b1fdb95
Author: John Budorick <jbudorick@chromium.org>
Date: Mon Aug 27 22:25:19 2018

sheriff-o-matic: implement support for gatekeeper categories.

Bug:  815006 
Change-Id: If6d8a0ff35346e0bf9815ab26de852bf7b735974
Reviewed-on: https://chromium-review.googlesource.com/1184235
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Sean McCullough <seanmccullough@chromium.org>

[modify] https://crrev.com/4d90c8b790a3ea712967e5462d7b85793b1fdb95/go/src/infra/appengine/sheriff-o-matic/som/analyzer/analyzer_test.go
[modify] https://crrev.com/4d90c8b790a3ea712967e5462d7b85793b1fdb95/go/src/infra/appengine/sheriff-o-matic/som/analyzer/gatekeeper.go
[modify] https://crrev.com/4d90c8b790a3ea712967e5462d7b85793b1fdb95/go/src/infra/monitoring/messages/gatekeeper.go

New version of SoM containing #26 was pushed on 2018-09-28.