Send alert emails when tests failures exceed a threshold |
||||||
Issue descriptionSome tests are flaky and we don't have the time to fix them all. So we ended up muting the alert emails because there were too many. I still want to be notified if the failures are stable. For example, video_VideoDecodeAccelerator fails on all devices or fails on one device for three builds in a row. Maybe we can write a script to filter emails from crosinfra@gmail.com. https://groups.google.com/a/google.com/forum/#!topic/chromeos-video-test-failures/bEc_KffiYRA
,
Jan 20 2017
There's time-driven trigger similar to a cron job. https://developers.google.com/apps-script/guides/triggers/installable
,
Jan 23 2017
Thanks for the information. The idea we draft first is to combine Google-App-Script to Gmail. I could run a script on chromeos-test-video-failure archived mails of my account, to parse the valuable information.
,
Jan 23 2017
Pawel has an idea. We can make a summary of test failures like the number of devices and the number of failure in a row.
,
Jan 26 2017
,
Feb 15 2017
Johny. Any update?
,
Feb 15 2017
Code implementation is on-going. My idea now is using Google Apps Script to parse the following two steps: aggregate cros mails --(1)-> (SpreadSheet) dashboard --(2)-> (Mail) sends alert mails Step(1) coding is finished. Here is the implkementation: https://drive.google.com/open?id=1W2VTwjuTniTm6OCWH2ViXOzXHgpGkFbpmiIyyOh_4tlgNt6hlrXSTSFW
,
Feb 20 2017
I finished an alpha version of mail parser. I just found crosinfra doesn't autofile failure mails for suite:bvt-perbuild (https://bugs.chromium.org/p/chromium/issues/detail?id=694139), if this could be fixed then we can get both suite:video and suite:bvt-perbuild failures.
,
Mar 6 2017
There are two bugs tracking why bvt-cq and bvt-perbuild didn't send mails to chromeos-video-test-failures@google.com https://bugs.chromium.org/p/chromium/issues/detail?id=694139 https://bugs.chromium.org/p/chromium/issues/detail?id=697742 Since we still have many test failures, I set up a priority test group and right now it will report test failures in the priority group only (for others it still collects data, but not report on alert mail) Priority group right now: ["video_VideoEncodeAccelerator", "video_VideoDecodeAccelerator", "video_ChromeRTCHWDecodeUsed", "video_VEAPerf", "video_JpegDecodeAccelerator"]
,
Mar 7 2017
Please add video_VideoDecodeAccelerator to video suite before 694139 is fixed.
,
Mar 14 2017
,
May 30 2017
,
Aug 1 2017
,
Nov 6 2017
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by wuchengli@chromium.org
, Jan 20 2017Status: Assigned (was: Available)