New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 729553 link

Starred by 3 users

Issue metadata

Status: Archived
Owner: ----
Closed: Jul 10
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug



Sign in to add a comment

Transport security state generator does not handle Windows style line-endings in the pins file.

Project Member Reported by mart...@martijnc.be, Jun 5 2017

Issue description

The transport security state generator build step will fail when the pins file uses Windows style line-endings.

ninja: Entering directory `out/Default'
[5/16] ACTION //net/http:generate_transport_security_state(//build/toolchain/win:x64)
FAILED: gen/net/http/transport_security_state_static.h
D:/depot_tools_10/python276_bin/python.exe ../../build/gn_run_binary.py transport_security_state_generator.exe ../../net/http/transport_security_state_static.json ../../net/http/transport_security_state_static.pins ../../net/http/transport_security_state_static.template gen/net/http/transport_security_state_static.h
[0605/151613.923:ERROR:input_file_parsers.cc(193)] Invalid name in pins file:
[0605/151613.923:ERROR:transport_security_state_generator.cc(213)] Error while parsing the input files.
transport_security_state_generator.exe failed with exit code 1
ninja: build stopped: subcommand failed.

The pins file from the repository uses correct Unix style line-endings but git converts them to Windows style line-endings if core.autocrlf is not set correctly.

The old get-the-code instructions[1] explicitly mentioned this setting but the new in-repo instructions do not [2]. Someone contacted me about this and linked to the in-repo docs.

[1] https://www.chromium.org/developers/how-tos/get-the-code-v2
[2] https://chromium.googlesource.com/chromium/src/+/master/docs/windows_build_instructions.md
 
Confirm the issue, same output occurred building Chromium Embedded branch 3112 (currently corresponding to chromium refs/tags/60.0.3112.10).
Cc: phajdan.jr@chromium.org
Note that depot_tools explicitly sets `git config --system core.autocrlf false` during Windows Git installation [1], which suggests that this should be the required/documented/expected value.

[1] https://cs.chromium.org/chromium/tools/depot_tools/bootstrap/win/git_bootstrap.py?rcl=76c6ea28e5f9437f9653c204eb63f9ec14cd7484&l=149
I have already another git installation with core.autocrlf set to true. It can be a conflict in the scenario depot_tools' git set crlf to false, but then reads it using another git invocation which happens to be the existing installation and this one returns true?

I don't know how much sense that makes, but I was performing the checkout (not the build) using the documented automate-git.py script of Chromium Embedded - which invokes depot_tools - and I ended up with core.crlf on true despite depot_tools git setting it to false.
@comment#3: `git config --system` and `git config --global` write to different locations on Windows. Using the Git installed with depot_tools the `--system` flag will write to "C:\Users\[User]\.gitconfig" and the `--global` flag will write to "depot_tools\git-2.10.0-64_bin\mingw64\etc\gitconfig". The priority read order lowest-to-highest is system, global, repository-specific. So, it would be quite easy to override or loose the system value set during depot_tools installation if you use or install a different version of Git.
Can you please confirm how can I sync the codebase correctly on my system? With limited internet services its hard to delete the sync because of just core.autocrlf messup.

Thanks.
Labels: TE-NeedsTriageHelp
As this issue is specific to Internals>Network>DomainSecurityPolicy , difficult to traige from TE end.

Comment 7 by est...@chromium.org, Jun 13 2017

Cc: mart...@martijnc.be est...@chromium.org
 Issue 732746  has been merged into this issue.
Components: Infra
Labels: Build-Tools
I'd like to hear from the infra team about whether it's okay to require the checkout to have correct `clrf` settings.

As an intermediate compromise, we could consider modifying the error to point out this issue on Windows. (We could even do so conditionally, based on a small sanity check for line endings.)

Comment 9 by mmoss@chromium.org, Jun 26 2017

Components: -Infra Infra>Git
Agree that a clearer error or solution would help. My build mysteriously broke today and it seems it's related to this.
Labels: -Pri-3 Pri-2
FYI anyone suffering a similar problem can use dos2unix on the two affected files (transport_security_state_static.pins and transport_security_state_static_unittest.pins) 
Project Member

Comment 13 by bugdroid1@chromium.org, Jul 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/89d553d1c6f6d386c52fc3ade38eb4fe2508ab7f

commit 89d553d1c6f6d386c52fc3ade38eb4fe2508ab7f
Author: martijn <martijn@martijnc.be>
Date: Sun Jul 09 21:44:30 2017

Add a .gitattributes file to force LF line-endings in .pins files.

This change forces the pins files to use LF line-endings and adds a check to
detect CRLF line-endings that outputs a more helpful error message.

BUG= 729553 

Review-Url: https://codereview.chromium.org/2941543002
Cr-Commit-Position: refs/heads/master@{#485165}

[modify] https://crrev.com/89d553d1c6f6d386c52fc3ade38eb4fe2508ab7f/.gitattributes
[modify] https://crrev.com/89d553d1c6f6d386c52fc3ade38eb4fe2508ab7f/net/tools/transport_security_state_generator/input_file_parsers.cc

Project Member

Comment 14 by sheriffbot@chromium.org, Jul 10

Status: Archived (was: Unconfirmed)
Issue has not been modified or commented on in the last 365 days, please re-open or file a new bug if this is still an issue.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
This was kind of fixed by enforcing LF endings in the pin file in .gitattributes. So it doesn't matter anymore whether the program chokes on CRLF or not.

Sign in to add a comment