New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 669704 link

Starred by 1 user
Status: Duplicate
Merged: issue 642980
Owner:
qyears...@chromium.org
Closed: Nov 2016
Cc:
█ wkorman@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

should rebaseline-cl ignore cases which fail before/after patch

Project Member Reported by ericrk@chromium.org, Nov 29 2016 
When using the rebaseline-cl tool, I hit a case where there were a number of tests that failed both before and after my change. Rebaseline-cl added new baselines for these cases, which seems to be undesirable (adds noise to the submit, makes it harder to understand what my change actually impacts).

My patch is here:
  https://codereview.chromium.org/2514263002/#ps180001

The bot in question is mac10.11_retina_blink_rel - results here:
  https://build.chromium.org/p/tryserver.blink/builders/mac10.11_retina_blink_rel/builds/1329

Note that the retry summary reports:

  webkit_tests (retry summary) webkit_tests (retry summary)

  failures:
  fast/hidpi/image-set-border-image-button.html
  fast/borders/border-image-scale-transform.html

  ignored:
  fast/forms/radio/radio-appearance-basic.html
  fast/forms/input-appearance-height.html
  fast/forms/checkbox/checkbox-appearance-basic.html
  fast/text/whitespace/normal-after-nowrap-breaking.html
  fast/forms/input-value.html
  fast/forms/001.html

From this, I would only have expected two new baselines added to my CL for the failures. Instead the script added new baselines for the ignored values as well.

To run the bots and to pull the new baselines, I just ran:
./third_party/WebKit/Tools/Scripts/webkit-patch rebaseline-cl

Comment 1 by ericrk@chromium.org, Nov 29 2016

Owner: wkorman@chromium.org
wkorman@, do you know who the right owner for this is?

Comment 2 by wkorman@chromium.org, Nov 29 2016

Cc: wkorman@chromium.org
Owner: qyears...@chromium.org
Status: Assigned (was: Untriaged)
Passing to owner, this may be a dupe of  http://crbug.com/642980  but we need more test cases with data so owner should review the above.

Comment 3 by qyears...@chromium.org, Nov 30 2016

Mergedinto: 642980
Status: Duplicate (was: Assigned)
Looks like what happened here is that recently the builders are failing to find gsutil, so they're failing to upload the retry summary to GS, so the rebaseline-cl tool can't find the retry summary, so it's falling back to a default behavior where it doesn't consider what tests failed without the patch.

Log:
https://build.chromium.org/p/tryserver.blink/builders/mac10.10_blink_rel/builds/1347/steps/archive_retry_summary/logs/stdio
Log excerpt:
/b/rr/tmpZvojBV/rw/checkout/scripts/slave/gsutil: line 4: /b/rr/tmpZvojBV/rw/checkout/scripts/slave/../../../depot_tools/gsutil.py: No such file or directory

Duping into  bug 642980  since that issue has more context.
►

Sign in to add a comment