New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 609871 link

Starred by 4 users

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature



Sign in to add a comment

Add a check to the w3c import script for long paths (which have limitations on Windows builders)

Project Member Reported by qyears...@chromium.org, May 6 2016

Issue description

In bug 609853, we saw a case where a builder failed because it was unable to create a file with a path of a certain length (Windows has a max file path length of 260 according to https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath).

The path that failed was "third_party/WebKit/LayoutTests/imported/web-platform-tests/custom-elements/pre-v1/instantiating-custom-elements/extensions-to-document-interface/create-element-interface-type-is-a-type-extension.html" (starting from the Chromium repo base, 199 characters).

In webkitpy/w3c/test_importer.py, we could add a check for long paths, potentially skip those paths, and print an error.
 
Cc: thakis@chromium.org brettw@chromium.org brucedaw...@chromium.org scottmg@chromium.org
Components: Build
We should also figure out if there are other parts of the build and test process that run up against this limit that we can also check for. Adding a few people who might know of them ...
Owner: qyears...@chromium.org
Status: Assigned (was: Available)
In bug 609853, kojii noted that web-platform-tests has a check for max path length of 150. Specifically, the check is here: https://github.com/w3c/wpt-tools/blob/master/lint/lint.py#L46

I wonder whether folks would be OK with decreasing that limit, just based on the rationale that we're running into Windows path length issues here.

Anyway, that can be separate from adding a check to our importer to skip long path lengths and give warnings, for now.
I think the whole Chromium repository should probably have a presubmit which prevents overly long paths ending up in the repository as a whole? (IE This problem isn't unique to w3c importing?)
Cc: tansell@chromium.org
See also  bug 612667  that tansell@ just filed that addresses my comment #1.

Comment 6 by tkent@chromium.org, May 20 2016

With ToT, I found update-w3c-deps skipped a reference file though it imported the corresponding test HTML.  

imported/wpt/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-002b-expected.html was skipped due to the long path length.
imported/wpt/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-002b.html was imported.

Dropping only -expected.html causes a test failure, of course.

I didn't think of that -- sounds like if we skip a reference, we should skip the corresponding test. Will make a follow-up CL.
Project Member

Comment 8 by bugdroid1@chromium.org, May 20 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2014d9ba913adaeae35027b2bed27a6f38fb278f

commit 2014d9ba913adaeae35027b2bed27a6f38fb278f
Author: qyearsley <qyearsley@chromium.org>
Date: Fri May 20 23:34:54 2016

W3C test importer: Skip importing files whose ref files would be too long.

BUG= 609871 

Review-Url: https://codereview.chromium.org/1997113002
Cr-Commit-Position: refs/heads/master@{#395202}

[modify] https://crrev.com/2014d9ba913adaeae35027b2bed27a6f38fb278f/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py

Status: Fixed (was: Assigned)
As it stands, skipping these tests by importer script is a silent act. This leads to head scratch moments for anyone trying to figure out why a wpt test in missing in Blink. Normally any test that is skipped has an entry in LayoutTests/W3CImportExpectations which is where one looks to find tests that are not imported. 

Base on my check, currently there are only 12 tests that hit this file name limit.

Couple of suggested ways to help (in my order of preference):

 1. Add a separate auto-generated expectations file that list these skipped files. Import script can generate this file and add it.

 2. Print these in the output of Tools/Scripts/update-w3c-deps. Right now you can see these sprinkled in the --verbose output but I think they should be printed out more prominently e.g., in normal output.

Project Member

Comment 11 by bugdroid1@chromium.org, Jan 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/706a4e2dceb1e19d8b74cff5a9cd8b04dc781b1d

commit 706a4e2dceb1e19d8b74cff5a9cd8b04dc781b1d
Author: majidvp <majidvp@chromium.org>
Date: Mon Jan 23 23:45:10 2017

Increase file path length limit for w3c test importer

Since the original limit was added the base path has been reduced by 15 chars.*
Increasing the limit by this amount allows us to import more tests without
breaking any existing windows bots.

* from 'imported/web-platform-tests' to 'external/wpt'

BUG= 609871 

Review-Url: https://codereview.chromium.org/2645323003
Cr-Commit-Position: refs/heads/master@{#445546}

[modify] https://crrev.com/706a4e2dceb1e19d8b74cff5a9cd8b04dc781b1d/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py

Sign in to add a comment