All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=816492

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=bd91d740795af6ea9af2dd92485767544a3cadb82b5e12dbe731768532ebe015


Bot(s) for this bug's original alert(s):

android-nexus5
android-nexus5X
android-one
android-webview-nexus5X
chromium-rel-mac11-pro

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14ea7598440000

Stumbled on this in graphs : pretty sure this is https://chromium.googlesource.com/v8/v8/+/3c62f7ae0744ebf95769429e5b2b3c83fc384fdf given it also lines up with its revert.

sullivan,simonhatch: pinpoint is still running after 17 hours, seems pretty clear to me from the graphs, why is it taking so long?

dtu, simonhatch, can one of you take a look?

I think the android-one bots are just overloaded at the moment, there are only 3 devices: https://chromium-swarm.appspot.com/botlist?c=id&c=os&c=task&c=status&f=pool%3AChrome-perf-pinpoint&f=device_type%3AAndroid%20One%20(sprout)&l=100&s=id%3Aasc

All of which are running tasks, and looking at a couple there seemed to be about 40 more test runs queued each.

From the Pinpoint results so far, it looks like one of these 3:
7507763 cros: Workaround crash in Feedback tool under --mash by James Cook · 5 days ago
fecb9c3ee Handle errors properly in node.py. by Torne (Richard Coles) · 5 days ago
68d73438 Add MemoryAblation to field_trial_config by Maria Khomenko · 5 days ago

(Yes, they're way overloaded.)

mariakhomenko@'s blamed CL was already reverted and I'm tracking mine elsewhere.

It's unfortunate that pinpoint jobs can't be preempted when a blame CL was identified, doing so would help relieve the bots (especially in scenarios like this where a single CL regresses *many* benchmarks which result in *many* pinpoint jobs).

That's an interesting idea, maybe we could put in something manual to go shut off other jobs? Dave, wdyt?

Yes, we can include a manual "Stop" button. I don't think we could do it automatically, since if the job is still running we don't yet know what the culprits are.