All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=613703

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZydgsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSENocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvQVZfYnVmZmVyX3Bvc3RfbG9hZAwLEg1TdG9wcGFnZUFsZXJ0GN2EGAw,agxzfmNocm9tZXBlcmZyegsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiTENocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvQVZfYnVmZmVyX3Bvc3RfbG9hZF9yZWYMCxINU3RvcHBhZ2VBbGVydBjdhBgM,agxzfmNocm9tZXBlcmZydQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiR0Nocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvQVZfYnVmZmVyX3ByZV9sb2FkDAsSDVN0b3BwYWdlQWxlcnQY3YQYDA,agxzfmNocm9tZXBlcmZyeQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiS0Nocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvQVZfYnVmZmVyX3ByZV9sb2FkX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GN2EGAw,agxzfmNocm9tZXBlcmZydQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiR0Nocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvQV9idWZmZXJfcG9zdF9sb2FkDAsSDVN0b3BwYWdlQWxlcnQY3YQYDA,agxzfmNocm9tZXBlcmZyeQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiS0Nocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvQV9idWZmZXJfcG9zdF9sb2FkX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GN2EGAw,agxzfmNocm9tZXBlcmZydAsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiRkNocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvQV9idWZmZXJfcHJlX2xvYWQMCxINU3RvcHBhZ2VBbGVydBjdhBgM,agxzfmNocm9tZXBlcmZyeAsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSkNocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvQV9idWZmZXJfcHJlX2xvYWRfcmVmDAsSDVN0b3BwYWdlQWxlcnQY3YQYDA,agxzfmNocm9tZXBlcmZydQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiR0Nocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvVl9idWZmZXJfcG9zdF9sb2FkDAsSDVN0b3BwYWdlQWxlcnQY3YQYDA,agxzfmNocm9tZXBlcmZyeQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiS0Nocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvVl9idWZmZXJfcG9zdF9sb2FkX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GN2EGAw,agxzfmNocm9tZXBlcmZydAsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiRkNocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvVl9idWZmZXJfcHJlX2xvYWQMCxINU3RvcHBhZ2VBbGVydBjdhBgM,agxzfmNocm9tZXBlcmZyeAsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSkNocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvVl9idWZmZXJfcHJlX2xvYWRfcmVmDAsSDVN0b3BwYWdlQWxlcnQY3YQYDA,agxzfmNocm9tZXBlcmZyfwsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiUUNocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvc3RhcnR1cF90ZXN0Lmh0bWw_dGVzdFR5cGVfQQwLEg1TdG9wcGFnZUFsZXJ0GN2EGAw,agxzfmNocm9tZXBlcmZygAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlJDaHJvbWl1bVBlcmYvYW5kcm9pZC1uZXh1czd2Mi9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL3N0YXJ0dXBfdGVzdC5odG1sP3Rlc3RUeXBlX0FWDAsSDVN0b3BwYWdlQWxlcnQY3YQYDA,agxzfmNocm9tZXBlcmZynAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50Im5DaHJvbWl1bVBlcmYvYW5kcm9pZC1uZXh1czd2Mi9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL3N0YXJ0dXBfdGVzdC5odG1sP3Rlc3RUeXBlX0FWX2RvTm90V2FpdEZvckJvZHlPbkxvYWRfdHJ1ZQwLEg1TdG9wcGFnZUFsZXJ0GN2EGAw,agxzfmNocm9tZXBlcmZyoAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50InJDaHJvbWl1bVBlcmYvYW5kcm9pZC1uZXh1czd2Mi9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL3N0YXJ0dXBfdGVzdC5odG1sP3Rlc3RUeXBlX0FWX2RvTm90V2FpdEZvckJvZHlPbkxvYWRfdHJ1ZV9yZWYMCxINU3RvcHBhZ2VBbGVydBjdhBgM,agxzfmNocm9tZXBlcmZyhAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlZDaHJvbWl1bVBlcmYvYW5kcm9pZC1uZXh1czd2Mi9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL3N0YXJ0dXBfdGVzdC5odG1sP3Rlc3RUeXBlX0FWX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GN2EGAw,agxzfmNocm9tZXBlcmZymwELEhNTdG9wcGFnZUFsZXJ0UGFyZW50Im1DaHJvbWl1bVBlcmYvYW5kcm9pZC1uZXh1czd2Mi9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL3N0YXJ0dXBfdGVzdC5odG1sP3Rlc3RUeXBlX0FfZG9Ob3RXYWl0Rm9yQm9keU9uTG9hZF90cnVlDAsSDVN0b3BwYWdlQWxlcnQY3YQYDA,agxzfmNocm9tZXBlcmZynwELEhNTdG9wcGFnZUFsZXJ0UGFyZW50InFDaHJvbWl1bVBlcmYvYW5kcm9pZC1uZXh1czd2Mi9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL3N0YXJ0dXBfdGVzdC5odG1sP3Rlc3RUeXBlX0FfZG9Ob3RXYWl0Rm9yQm9keU9uTG9hZF90cnVlX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GN2EGAw,agxzfmNocm9tZXBlcmZygwELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlVDaHJvbWl1bVBlcmYvYW5kcm9pZC1uZXh1czd2Mi9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL3N0YXJ0dXBfdGVzdC5odG1sP3Rlc3RUeXBlX0FfcmVmDAsSDVN0b3BwYWdlQWxlcnQY3YQYDA,agxzfmNocm9tZXBlcmZyfwsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiUUNocm9taXVtUGVyZi9hbmRyb2lkLW5leHVzN3YyL21lZGlhLm1zZV9jYXNlcy9hcHBlbmQvc3RhcnR1cF90ZXN0Lmh0bWw_dGVzdFR5cGVfVgwLEg1TdG9wcGFnZUFsZXJ0GN2EGAw,agxzfmNocm9tZXBlcmZydwsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSUNocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FWX2J1ZmZlcl9wb3N0X2xvYWQMCxINU3RvcHBhZ2VBbGVydBjngxgM,agxzfmNocm9tZXBlcmZyhAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlZDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL21lZGlhc291cmNlX29wZW5pbmcvQVZfYnVmZmVyX3Bvc3RfbG9hZAwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZyhAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlZDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9WX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZyoAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50InJDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9WX2RvTm90V2FpdEZvckJvZHlPbkxvYWRfdHJ1ZV9yZWYMCxINU3RvcHBhZ2VBbGVydBi_7RcM,agxzfmNocm9tZXBlcmZynAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50Im5DaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9WX2RvTm90V2FpdEZvckJvZHlPbkxvYWRfdHJ1ZQwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZygAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlJDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9WDAsSDVN0b3BwYWdlQWxlcnQYv-0XDA,agxzfmNocm9tZXBlcmZyhAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlZDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZyhQELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IldDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BVl9yZWYMCxINU3RvcHBhZ2VBbGVydBi_7RcM,agxzfmNocm9tZXBlcmZyoQELEhNTdG9wcGFnZUFsZXJ0UGFyZW50InNDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BVl9kb05vdFdhaXRGb3JCb2R5T25Mb2FkX3RydWVfcmVmDAsSDVN0b3BwYWdlQWxlcnQYv-0XDA,agxzfmNocm9tZXBlcmZynQELEhNTdG9wcGFnZUFsZXJ0UGFyZW50Im9DaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BVl9kb05vdFdhaXRGb3JCb2R5T25Mb2FkX3RydWUMCxINU3RvcHBhZ2VBbGVydBi_7RcM,agxzfmNocm9tZXBlcmZygAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlJDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BDAsSDVN0b3BwYWdlQWxlcnQYv-0XDA,agxzfmNocm9tZXBlcmZyegsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiTENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL1ZfYnVmZmVyX3Bvc3RfbG9hZF9yZWYMCxINU3RvcHBhZ2VBbGVydBi_7RcM,agxzfmNocm9tZXBlcmZydgsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL1ZfYnVmZmVyX3Bvc3RfbG9hZAwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZyeQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiS0Nocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FfYnVmZmVyX3ByZV9sb2FkX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZydQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiR0Nocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FfYnVmZmVyX3ByZV9sb2FkDAsSDVN0b3BwYWdlQWxlcnQYv-0XDA,agxzfmNocm9tZXBlcmZyegsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiTENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FfYnVmZmVyX3Bvc3RfbG9hZF9yZWYMCxINU3RvcHBhZ2VBbGVydBi_7RcM,agxzfmNocm9tZXBlcmZyegsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiTENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FWX2J1ZmZlcl9wcmVfbG9hZF9yZWYMCxINU3RvcHBhZ2VBbGVydBi_7RcM,agxzfmNocm9tZXBlcmZydgsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FfYnVmZmVyX3Bvc3RfbG9hZAwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZydQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiR0Nocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL1ZfYnVmZmVyX3ByZV9sb2FkDAsSDVN0b3BwYWdlQWxlcnQYv-0XDA,agxzfmNocm9tZXBlcmZyeQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiS0Nocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL1ZfYnVmZmVyX3ByZV9sb2FkX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZygQELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlNDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BVgwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZynAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50Im5DaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BX2RvTm90V2FpdEZvckJvZHlPbkxvYWRfdHJ1ZQwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZyoAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50InJDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BX2RvTm90V2FpdEZvckJvZHlPbkxvYWRfdHJ1ZV9yZWYMCxINU3RvcHBhZ2VBbGVydBi_7RcM,agxzfmNocm9tZXBlcmZydgsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FWX2J1ZmZlcl9wcmVfbG9hZAwLEg1TdG9wcGFnZUFsZXJ0GL_tFww,agxzfmNocm9tZXBlcmZyewsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiTUNocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FWX2J1ZmZlcl9wb3N0X2xvYWRfcmVmDAsSDVN0b3BwYWdlQWxlcnQYv-0XDA,agxzfmNocm9tZXBlcmZydwsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSUNocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FWX2J1ZmZlcl9wb3N0X2xvYWQMCxINU3RvcHBhZ2VBbGVydBi_7RcM,agxzfmNocm9tZXBlcmZygAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlJDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9WDAsSDVN0b3BwYWdlQWxlcnQY4OYXDA,agxzfmNocm9tZXBlcmZyhAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlZDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GODmFww,agxzfmNocm9tZXBlcmZyoAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50InJDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BX2RvTm90V2FpdEZvckJvZHlPbkxvYWRfdHJ1ZV9yZWYMCxINU3RvcHBhZ2VBbGVydBjg5hcM,agxzfmNocm9tZXBlcmZyhQELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IldDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BVl9yZWYMCxINU3RvcHBhZ2VBbGVydBjg5hcM,agxzfmNocm9tZXBlcmZyoQELEhNTdG9wcGFnZUFsZXJ0UGFyZW50InNDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BVl9kb05vdFdhaXRGb3JCb2R5T25Mb2FkX3RydWVfcmVmDAsSDVN0b3BwYWdlQWxlcnQY4OYXDA,agxzfmNocm9tZXBlcmZynQELEhNTdG9wcGFnZUFsZXJ0UGFyZW50Im9DaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BVl9kb05vdFdhaXRGb3JCb2R5T25Mb2FkX3RydWUMCxINU3RvcHBhZ2VBbGVydBjg5hcM,agxzfmNocm9tZXBlcmZygQELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlNDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BVgwLEg1TdG9wcGFnZUFsZXJ0GODmFww,agxzfmNocm9tZXBlcmZygAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50IlJDaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BDAsSDVN0b3BwYWdlQWxlcnQY4OYXDA,agxzfmNocm9tZXBlcmZyeQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiS0Nocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL1ZfYnVmZmVyX3ByZV9sb2FkX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GODmFww,agxzfmNocm9tZXBlcmZyegsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiTENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL1ZfYnVmZmVyX3Bvc3RfbG9hZF9yZWYMCxINU3RvcHBhZ2VBbGVydBjg5hcM,agxzfmNocm9tZXBlcmZydgsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL1ZfYnVmZmVyX3Bvc3RfbG9hZAwLEg1TdG9wcGFnZUFsZXJ0GODmFww,agxzfmNocm9tZXBlcmZyeQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiS0Nocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FfYnVmZmVyX3ByZV9sb2FkX3JlZgwLEg1TdG9wcGFnZUFsZXJ0GODmFww,agxzfmNocm9tZXBlcmZydQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiR0Nocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FfYnVmZmVyX3ByZV9sb2FkDAsSDVN0b3BwYWdlQWxlcnQY4OYXDA,agxzfmNocm9tZXBlcmZyegsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiTENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FfYnVmZmVyX3Bvc3RfbG9hZF9yZWYMCxINU3RvcHBhZ2VBbGVydBjg5hcM,agxzfmNocm9tZXBlcmZydgsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FfYnVmZmVyX3Bvc3RfbG9hZAwLEg1TdG9wcGFnZUFsZXJ0GODmFww,agxzfmNocm9tZXBlcmZyegsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiTENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FWX2J1ZmZlcl9wcmVfbG9hZF9yZWYMCxINU3RvcHBhZ2VBbGVydBjg5hcM,agxzfmNocm9tZXBlcmZydgsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSENocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FWX2J1ZmZlcl9wcmVfbG9hZAwLEg1TdG9wcGFnZUFsZXJ0GODmFww,agxzfmNocm9tZXBlcmZyewsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiTUNocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FWX2J1ZmZlcl9wb3N0X2xvYWRfcmVmDAsSDVN0b3BwYWdlQWxlcnQY4OYXDA,agxzfmNocm9tZXBlcmZydwsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiSUNocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL0FWX2J1ZmZlcl9wb3N0X2xvYWQMCxINU3RvcHBhZ2VBbGVydBjg5hcM,agxzfmNocm9tZXBlcmZydQsSE1N0b3BwYWdlQWxlcnRQYXJlbnQiR0Nocm9taXVtUGVyZi9hbmRyb2lkLWdhbGF4eS1zNS9tZWRpYS5tc2VfY2FzZXMvYXBwZW5kL1ZfYnVmZmVyX3ByZV9sb2FkDAsSDVN0b3BwYWdlQWxlcnQY4OYXDA,agxzfmNocm9tZXBlcmZynAELEhNTdG9wcGFnZUFsZXJ0UGFyZW50Im5DaHJvbWl1bVBlcmYvYW5kcm9pZC1nYWxheHktczUvbWVkaWEubXNlX2Nhc2VzL2FwcGVuZC9zdGFydHVwX3Rlc3QuaHRtbD90ZXN0VHlwZV9BX2RvTm90V2FpdEZvckJvZHlPbkxvYWRfdHJ1ZQwLEg1TdG9wcGFnZUFsZXJ0GODmFww


Bot(s) for this bug's original alert(s):

android-galaxy-s5
android-nexus7v2

Why are you ignoring the alerts? For example, just looking at the first one, no data has come in since May 19:
https://chromeperf.appspot.com/report?sid=90ad2e41a095db573fae09ec4631a8f78aa6859ff12788606e85447d50e96037

Would you prefer to disable this test?

Adding Dan back.

Dan can speak to this as well. We currently have a rotation of people to look into the alerts. I think that we have been having trouble understanding exactly what to do to triage the alerts, and so people are simply ignoring them. It's a problem that we are hoping to fix. 

I wrote a doc on triaging these, it's a bit verbose, but maybe it's a starting point? https://docs.google.com/document/d/1cF2Ny3UYbXq2y3fZaygUSz_3lVc0SOrSRZwXqGjjKgo/edit

Looks like the test has recovered, but I don't know why it was failing. For example, this is a failure. runtest.py saw an exit code of 87 from something and "adb devices" looks to be the last thing that ran.

I guess we can see if this happens again and try to debug the cause.

/usr/bin/python /b/build/slave/Android_Galaxy_S5_Perf__3_/build/src/build/android/test_runner.py perf --print-step media.mse_cases --verbose --blacklist-file /b/build/slave/Android_Galaxy_S5_Perf__3_/build/src/out/bad_devices.json --output-chartjson-data=/tmp/tmpZIz3St
I    0.004s Main  command: /b/build/slave/Android_Galaxy_S5_Perf__3_/build/src/build/android/test_runner.py perf --print-step media.mse_cases --verbose --blacklist-file /b/build/slave/Android_Galaxy_S5_Perf__3_/build/src/out/bad_devices.json --output-chartjson-data=/tmp/tmpZIz3St
I    0.004s TimeoutThread-1-for-MainThread  [host]> /b/build/slave/Android_Galaxy_S5_Perf__3_/build/src/third_party/android_tools/sdk/platform-tools/adb devices
E    0.012s Main  File not found /b/build/slave/Android_Galaxy_S5_Perf__3_/build/src/out/step_results/media.mse_cases
<Thread(Thread-1, started 140718020540160)> ProcessRead: proc.stdout finished.
<Thread(Thread-1, started 140718020540160)> ProcessRead: cleaning up.
<Thread(Thread-2, started daemon 140718012147456)> TimedFlush: Finished
<Thread(Thread-1, started 140718020540160)> ProcessRead: finished.
exit code (as seen by runtest.py): 87
@@@STEP_FAILURE@@@

Moving this nonessential bug to the next milestone.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

This issue has been moved once and is lower than Pri-1. Removing the milestone.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Annie, any update on this bug?

Assignign back to reporter to debug with the doc linked in #6 and info in #7 if it happens again.

Pinging Dan.  Could you comment on #12?

Since the data is currently flowing there is not too much I can do. I'll leave this open until the next time it fails.

Leaving this open as per #14.

Dan, is this something we should still be working on?

+Frank.

+ Thomas

 Issue 669312  has been merged into this issue.

Stale.