New issue
Advanced search Search tips

Issue 758725 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

autoninja makes it harder to compile a single source file

Project Member Reported by sebmarchand@chromium.org, Aug 24 2017

Issue description

When using ninja you can run:

  ninja -C out\Debug ..\..\base\at_exit.cc^^

to compile at_exit.cc (the double caret is needed on Windows as it's an escape character).

The recommended setup for autoninja is to have a batch file called ninja.bat that calls autoninja.bat, which itself calls ninja.exe. Because of this I need to use *32* caret characters for this command to work :).

FYI I'm trying to fix the compile_current_file script[1], and because of this I need to make it use ninja.exe directly, see https://chromium-review.googlesource.com/c/chromium/src/+/634267

[1] https://cs.chromium.org/chromium/src/tools/sublime/compile_current_file.py
 
Description: Show this description
D'oh! No wonder that stopped working. Thanks for figuring it out.

Using ninja.exe directly is the simplest workaround for now. I'm not sure how to fix this cleanly. It would be nice to have a character that didn't have special meaning to the command processor.
Status: Started (was: Untriaged)
Project Member

Comment 4 by bugdroid1@chromium.org, Sep 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/ee3946be4e5237f443a5cc2aa2ec2fa75495539b

commit ee3946be4e5237f443a5cc2aa2ec2fa75495539b
Author: Bruce Dawson <brucedawson@chromium.org>
Date: Wed Sep 06 18:29:47 2017

Fix autoninja.bat to not swallow ^^ sequences

Ninja uses the '^' character to indicate that ninja should build the
targets that are generated from the specified file, rather than
building the specified file. On Windows '^^' is needed because '^' is
the line continuation character. However autoninja.bat complicates
things because the multiple levels of batch files successfully swallow
pairs of '^' characters.

By adding quotes around %* in autoninja.bat it becomes possible to
invoke autoninja.bat normally. That is, this works:

    autoninja -C out\debug_component ..\..\base\win\enum_variant.cc^^

It can be convenient to have a ninja.bat file which starts goma and lets
users keep typing the same build commands. However even with this fix
the previously recommended ninja.bat file requires four '^' characters.
If that is too much then the new recommended ninja.bat is to copy
autoninja.bat and modify as needed, perhaps like this:

    @echo off
    call python c:\goma\goma-win64\goma_ctl.py ensure_start >nul
    FOR /f "usebackq tokens=*" %%a in (`python c:\src\depot_tools\autoninja.py "%*"`) do echo %%a & %%a

BUG= 758725 

Change-Id: I648cf42675af2f946be7aa4033956b015d953829
Reviewed-on: https://chromium-review.googlesource.com/651826
Reviewed-by: Sébastien Marchand <sebmarchand@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

[modify] https://crrev.com/ee3946be4e5237f443a5cc2aa2ec2fa75495539b/autoninja.bat

Status: Fixed (was: Started)
Project Member

Comment 6 by bugdroid1@chromium.org, Sep 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/464a4779c5a28407026104fc9b13e139e8b9843c

commit 464a4779c5a28407026104fc9b13e139e8b9843c
Author: Bruce Dawson <brucedawson@chromium.org>
Date: Wed Sep 06 22:24:13 2017

Revert "Fix autoninja.bat to not swallow ^^ sequences"

This reverts commit ee3946be4e5237f443a5cc2aa2ec2fa75495539b.

Reason for revert: This broke autoninja. The arguments all come in as one and aren't recognized. I think I can fix but I'm reverting for now.

Original change's description:
> Fix autoninja.bat to not swallow ^^ sequences
> 
> Ninja uses the '^' character to indicate that ninja should build the
> targets that are generated from the specified file, rather than
> building the specified file. On Windows '^^' is needed because '^' is
> the line continuation character. However autoninja.bat complicates
> things because the multiple levels of batch files successfully swallow
> pairs of '^' characters.
> 
> By adding quotes around %* in autoninja.bat it becomes possible to
> invoke autoninja.bat normally. That is, this works:
> 
>     autoninja -C out\debug_component ..\..\base\win\enum_variant.cc^^
> 
> It can be convenient to have a ninja.bat file which starts goma and lets
> users keep typing the same build commands. However even with this fix
> the previously recommended ninja.bat file requires four '^' characters.
> If that is too much then the new recommended ninja.bat is to copy
> autoninja.bat and modify as needed, perhaps like this:
> 
>     @echo off
>     call python c:\goma\goma-win64\goma_ctl.py ensure_start >nul
>     FOR /f "usebackq tokens=*" %%a in (`python c:\src\depot_tools\autoninja.py "%*"`) do echo %%a & %%a
> 
> BUG= 758725 
> 
> Change-Id: I648cf42675af2f946be7aa4033956b015d953829
> Reviewed-on: https://chromium-review.googlesource.com/651826
> Reviewed-by: Sébastien Marchand <sebmarchand@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

TBR=dpranke@chromium.org,brucedawson@chromium.org,sebmarchand@chromium.org

Change-Id: I131b9ba00882acb5a2d009a2a444f186740d7394
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  758725 
Reviewed-on: https://chromium-review.googlesource.com/654117
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>

[modify] https://crrev.com/464a4779c5a28407026104fc9b13e139e8b9843c/autoninja.bat

Status: Started (was: Fixed)
New fix coming...
Project Member

Comment 8 by bugdroid1@chromium.org, Sep 29 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/f3b4f060e2a578420706275e858dc6a612dbaf8f

commit f3b4f060e2a578420706275e858dc6a612dbaf8f
Author: Bruce Dawson <brucedawson@chromium.org>
Date: Fri Sep 29 18:15:44 2017

Fix autoninja to allow compiling one source file

This is the second attempt at fixing autoninja to allow passing '^^' to
it to specify that ninja should build the outputs of the specified file
instead of building that file itself. The problem is that '^' is a
special character and when extra layers of indirection are added the
number of '^' characters needed grows exponentially in some poorly
understood way. The first fix attempt just quoted the arguments that
autoninja.bat passed to autoninja.py, but that meant they came in as
one argument. This fix expands on that by modifying autoninja.py to
understand how to deal with the monolithic argument. With this change
this once again works:

    autoninja -C out\debug_component ..\..\base\win\enum_variant.cc^^

It can be convenient to have a ninja.bat file which starts goma and lets
users keep typing the same build commands. However even with this fix
the previously recommended ninja.bat file must be invoked with four
'^' characters. If that is too much then the new recommended ninja.bat
is to copy autoninja.bat and modify as needed, perhaps like this:

    @echo off
    call python c:\goma\goma-win64\goma_ctl.py ensure_start >nul
    FOR /f "usebackq tokens=*" %%a in (`python c:\src\depot_tools\autoninja.py "%*"`) do echo %%a & %%a

BUG:  758725 
Change-Id: Ieee9cf343ee5f22e9988a1969cb7a7a90687666b
Reviewed-on: https://chromium-review.googlesource.com/656478
Reviewed-by: Sébastien Marchand <sebmarchand@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

[modify] https://crrev.com/f3b4f060e2a578420706275e858dc6a612dbaf8f/autoninja.bat
[modify] https://crrev.com/f3b4f060e2a578420706275e858dc6a612dbaf8f/autoninja.py

Status: Fixed (was: Started)
Project Member

Comment 10 by bugdroid1@chromium.org, Oct 5 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/41288f5ae4dd53e7d93e29340751e25b9cefe24c

commit 41288f5ae4dd53e7d93e29340751e25b9cefe24c
Author: depot-tools-roller@chromium.org <depot-tools-roller@chromium.org>
Date: Thu Oct 05 10:49:11 2017

Roll src/third_party/depot_tools/ b2e961b11..b5807979e (72 commits; 5 trivial rolls)

https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/b2e961b1171d..b5807979e8a9

$ git log b2e961b11..b5807979e --date=short --no-merges --format='%ad %ae %s'
2017-10-04 xiaoyin.l Use HTTPS links in CheckAuthorizedAuthor
2017-10-04 dnj [vpython] Roll CIPD version.
2017-10-04 phajdan.jr gclient config: add support for custom vars
2017-10-03 phajdan.jr gclient: make predefined variables native
2017-10-03 mmoss Allow gclient solution URLs to be |None|.
2017-08-28 agable Simplify git-cl-diff for Gerrit
2017-10-03 phajdan.jr gclient flatten: properly quote conditionals
2017-09-28 phajdan.jr gclient: add support for native boolean variables
2017-09-29 phajdan.jr gclient flatten: emit conditions for hooks
2017-09-29 siggi Update fetch config with new Syzygy location.
2017-09-29 agable Remove rietveld commit queue command line client
2017-09-29 agable Remove rietveld git cherry-pick-upload command
2017-09-29 borenet roll-dep: Add comments indicating that --no-log needs to keep working
2017-09-28 mmoss Expose new gclient.sync() arg in gclient.checkout().
2017-09-07 brucedawson Fix autoninja to allow compiling one source file
2017-09-29 agable Remove defunct Rietveld end-to-end tests
2017-09-28 dnj [depot_tools] Add "ensure_bootstrap" script.
2017-09-28 agable bot_update: use patch repo instead of project if it is mapped
2017-09-27 mmoss Add ability to pass extra flags to gclient.sync().
2017-09-27 dnj Roll "vpython".
2017-09-26 agrieve Fix CheckPatchFormatted() when using inherit-review-settings-ok
2017-09-26 mmoss Remove unused gclient.sync() 'with_branch_heads' arg.
2017-09-25 phajdan.jr gclient: evaluate variables before passing them to GN
2017-09-26 phajdan.jr gclient: expose target_os
2017-09-26 yyanagisawa git-cl upload: adjust code-review score on TBR.
2017-09-25 emso Adds progress argument option to fetch for git checkout
2017-09-18 rharrison Add support for PDFium to my_activity.py
2017-09-18 rmistry Add skia_buildbot config
2017-09-14 mmoss Add some differentiating details to gerrit step names.
2017-09-15 ehmaldonado WebRTC: Remove hack in bot_update.py.
2017-09-13 agable git-cl: Parse new change urls containing project paths
2017-09-11 ehmaldonado WebRTC: Update source-of-truth on 'fetch webrtc'
2017-09-13 iannucci Roll vpython.
2017-09-13 athom Add gsutil.vpython spec to recipe bundles
2017-09-13 ehmaldonado Revert "Fix checkout.py issues when p.patchlevel > 1."
2017-09-12 ehmaldonado Fix checkout.py issues when p.patchlevel > 1.
2017-09-12 iannucci Roll led and vpython
2017-09-12 phajdan.jr gclient flatten: improve reporting of DEPS files
2017-09-11 iannucci Add s390x support for cipd client bootstrap script.
2017-09-11 agable git-cl upload: Send mail when starting CQ
2017-09-11 bjaideep gclient: Add aix to host_os list
2017-09-11 agable Add GCE instructions to git cl creds check
2017-09-11 ehmaldonado WebRTC: Update source-of-truth for 'git-cl'
2017-09-07 dcheng Use dateutil.parser for parsing dates.
2017-09-11 tikuta Update ninja to v1.8.2
2017-09-07 dnj [gsutil] run through "vpython" (2)
2017-09-07 mbjorge Revert "[gsutil] run through "vpython""
2017-09-07 dnj [gsutil] run through "vpython"
2017-09-07 phajdan.jr Revert "update_depot_tools: make cipd_bin_setup failures fatal"
2017-09-06 phajdan.jr cipd: also support wget as fetch command
2017-09-06 phajdan.jr update_depot_tools: make cipd_bin_setup failures fatal
2017-09-05 phajdan.jr gclient: fetch arbitrary refs
2017-09-06 phajdan.jr Move downstream recipe trybots to production
2017-09-06 brucedawson Revert "Fix autoninja.bat to not swallow ^^ sequences"
2017-09-06 agable Revert "Capture ctrl-c in presubmit multiprocessing pool"
2017-09-05 brucedawson Fix autoninja.bat to not swallow ^^ sequences
2017-09-06 sebmarchand Add a script to make it easier to build a single source file with ninja
2017-09-05 agable git-cl set_close: Don't fail on branches with no issue
2017-09-05 agable Remove gerrit-over-ssh code from my_activity.py
2017-09-05 agable Capture ctrl-c in presubmit multiprocessing pool
2017-08-24 agable Don't send email from git-cl-description
2017-09-04 whesse Dart: Update the config for the "fetch dart" command.
2017-09-05 tikuta Add Nico and Scott to OWNERS for ninja related things
2017-09-02 dpranke Add a --batch mode to `git-cl owners`.
2017-09-04 tikuta Update ninja to v1.8.0.
2017-09-01 iannucci Update led tool to c9c1865b81113f02fd618259624170f59e2c832e.
2017-08-31 dnj [vpython] Update version.

Created with:
  roll-dep src/third_party/depot_tools
BUG= 771277 ,769369, 764087 , 741001 ,None,758725,741001


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


TBR=phajdan.jr@chromium.org

Change-Id: Icb9ca53560ec8ee470e60fea5ba7fc23a764fb34
Reviewed-on: https://chromium-review.googlesource.com/701321
Reviewed-by: depot-tools-roller . <depot-tools-roller@chromium.org>
Commit-Queue: depot-tools-roller . <depot-tools-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#506697}
[modify] https://crrev.com/41288f5ae4dd53e7d93e29340751e25b9cefe24c/DEPS

Sign in to add a comment