New issue
Advanced search Search tips

Issue 850204 link

Starred by 1 user
Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

update_expectations.py --type=fail removes Android-only expectations

Project Member Reported by qyears...@chromium.org, Jun 6 2018 
This is an interesting case for blink/tools/update_expectations.py.

Tests that have no results on all relevant platforms are removed. In practice, this means that Android-only expectations for tests that
are not listed in SmokeTests are removed, which is not desirable because the tests may actually fail on Android when run manually, and the expectations may still be accurate.

Proposed possible fix: Add a flag to enable removing lines with no results, and by default don't remove such lines.
Project Member

Comment 1 by bugdroid1@chromium.org, Jun 14 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/92fe9e08e40c716a95c46139b88c461d8b36f4d6

commit 92fe9e08e40c716a95c46139b88c461d8b36f4d6
Author: Quinten Yearsley <qyearsley@chromium.org>
Date: Thu Jun 14 22:44:37 2018

[blinkpy update_expectations] Keep lines with no results

By default, keep lines for tests that have no results;
Add a flag to turn on the removal of tests that have
no results.

Reason: For Android-only expectations for tests that
aren't in the SmokeTests file, we probably usually want
to keep the lines.

But for lines for tests that actually don't exist anymore,
we may want to actually remove the lines, so I wanted
to keep the option of possibly trying to remove such lines
(added a flag --remove-missing).

Bug: 850204
Change-Id: I2eabca25ef59373aedb84ec2165271302ddd50a2
Reviewed-on: https://chromium-review.googlesource.com/1089642
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: David Bokan <bokan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#567449}
[modify] https://crrev.com/92fe9e08e40c716a95c46139b88c461d8b36f4d6/third_party/blink/tools/blinkpy/web_tests/update_expectations.py
[modify] https://crrev.com/92fe9e08e40c716a95c46139b88c461d8b36f4d6/third_party/blink/tools/blinkpy/web_tests/update_expectations_unittest.py
►

Sign in to add a comment