New issue
Advanced search Search tips

Issue 669653 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Dec 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 474273



Sign in to add a comment

rebaseline-cl should bail on missing results

Project Member Reported by fmalita@chromium.org, Nov 29 2016

Issue description

Chris reminded me to file this, but I no longer have the output - reporting from memory, sorry.

Ran into this issue with https://codereview.chromium.org/2513303002:

- some of the trybots failed to upload the results for whatever reason (e.g. https://storage.googleapis.com/chromium-layout-test-archives/mac10_11_retina_blink_rel/1321/layout-test-results/results.html: NoSuchKey)

- rebaseline-cl logged something ("could not download results for platform foo" IIRC), but proceeded with the rebaseline anyway

If would be nice if rebaseline-cl failed in this case - otherwise the problem is easy to overlook.
 
Blocking: 474273
Labels: -Pri-3 Pri-2
Agreed -- thanks for filing this.

There's another bug uncovered here, that for some reason the results for that run were archived at

  https://storage.googleapis.com/chromium-layout-test-archives/mac10_11_retina_blink_rel/1321/results.html

instead of the expected .../1321/layout-test-results/results.html.

Curiously, the results for the builds immediately before and after are in the expected place.

  https://storage.googleapis.com/chromium-layout-test-archives/mac10_11_retina_blink_rel/1320/layout-test-results/results.html
  https://storage.googleapis.com/chromium-layout-test-archives/mac10_11_retina_blink_rel/1322/layout-test-results/results.html

Anyway, I agree that this should be done, and plan to make this change.
Project Member

Comment 2 by bugdroid1@chromium.org, Dec 7 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ff4522901de3100f338fc1f606e301ede5e96c3d

commit ff4522901de3100f338fc1f606e301ede5e96c3d
Author: qyearsley <qyearsley@chromium.org>
Date: Wed Dec 07 16:45:32 2016

In webkit-patch rebaseline-cl, abort if results are missing.

Reason: In general, if layout test results for some builder cannot be found in GS, and they are not downloaded, then when baselines are deduped, some old (incorrect) baselines may be left behind for those platforms that we couldn't get results for.

As-is, rebaseline-cl sort of depends on having results for all support platforms whenever rebaselining in order to get correct results.

Main changes in this CL:

 1. Results are always fetched, even if tests are directly passed. This way the behavior is consistent regardless of whether the list of tests to rebaseline is explicitly specified or decided based on results.
 2. Fetching of the results is done in a method called _fetch_results(), which may raise an exception if any results are missing; this exception is caught in execute(), and an error message is printed.
 3. _builds_to_tests is removed, and part of _tests_to_rebaseline is moved to _fetch_results(). buildbot.fetch_results is used instead of buildbot.fetch_layout_test_results.

Later, we might consider adding a flag to force continuing with rebaselining even if some results are not available, but I'm not sure if this is necessary.

BUG= 669653 

Review-Url: https://codereview.chromium.org/2547153002
Cr-Commit-Position: refs/heads/master@{#436982}

[modify] https://crrev.com/ff4522901de3100f338fc1f606e301ede5e96c3d/third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline_cl.py
[modify] https://crrev.com/ff4522901de3100f338fc1f606e301ede5e96c3d/third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline_cl_unittest.py

Status: Fixed (was: Assigned)

Sign in to add a comment