[WPT Import] Handle expectations for other build types besides basic Release builds. |
||||||||
Issue descriptionThis is a general bug, covering the below specific cases: - Some tests behave differently with DCHECK on, as is the case for CQ bots (bug 674692). - Relatedly, some tests behave differently in debug builds. - Some tests fail in ASAN/MSAN or Leak bots ( bug 662570 ). My thoughts about the general way to solve these problems is: We need to use more try bots and use their results when updating expectations. For example: - *_chromium_rel_ng CQ bots could be used to get the results when DCHECK is enabled - *_blink_dbg bots on tryserver.blink could also provide results for debug builds - new try bots could be added for ASAN/MSAN or Leak bots In the wpt-update-expectations, some important changes will need to be made: - "Debug" specifiers should be added for Debug-specific failures/timeouts/crashes (but we never add debug-specific baselines). - ASAN/MSAN/Leak expectations for Timeout/Failure/Crash also need to be added to their own special expectations files (again, we don't want to add baselines).
,
Jun 16 2017
Issue 662570 has been merged into this issue.
,
Jun 16 2017
,
Jul 3 2017
,
Jul 3 2017
,
Jul 3
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue. Sorry for the inconvenience if the bug really should have been left as Available. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jul 18
robertma@, what do you think about this avenue? It seems to me that right now our biggest problem is actually that we don't consider CQ bots, and that simply by triggering those and using the results, we'd have sufficient coverage?
,
Jul 18
Re #7: using the CQ bot to rebaseline is specifically issue 739522, which would solve the DCHECK problem; I might try to find some time to think about the implementation there. On the other hand, this issue has a much larger scope and won't be happening any time soon. |
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by qyears...@chromium.org
, May 22 2017