git ls-files does the same thing. Trying to find all the places that depot_tools uses the output of 'status' and 'ls-files' (not to mention all the times that recipes and other scripts do so) is going to be a lot of work.

I'm starting with gclient, git-cl, and presubmit. Everything else can come after that, by someone else.

https://chromium-review.googlesource.com/#/c/chromium/tools/depot_tools/+/815454

does

git config core.quotePath false

fix this?

Yes, hence the change above. I haven't landed it because of some recipes stuff, but it will hopefully land today or monday.

Thanks for fixing this.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/763f394a34933306409c1d15bfddc412e979f461

commit 763f394a34933306409c1d15bfddc412e979f461
Author: Benjamin C. Wiley Sittler <bsittler@chromium.org>
Date: Fri Dec 08 20:51:19 2017

Forms: remove accidental normalization-varying filename characters

These cause problems on macOS until git config is updated.

TBR=

Bug:  793347 ,  792302 
Change-Id: I54e53fdcd97ab42464ee84a1f216db4d393609bd
Reviewed-on: https://chromium-review.googlesource.com/817698
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Cr-Commit-Position: refs/heads/master@{#522867}
[delete] https://crrev.com/06d36e78907996e8778c06caa177f663920c2396/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-☺😂.txt
[modify] https://crrev.com/763f394a34933306409c1d15bfddc412e979f461/third_party/WebKit/LayoutTests/http/tests/local/fileapi/select-dragged-file-input-utf-8.html

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49

commit 9219d356885a93d9cbcaf4a2c7fc3f6e29caad49
Author: Aaron Gable <agable@chromium.org>
Date: Mon Dec 11 20:02:00 2017

Use core.quotePath=false when git is listing files

This prevents git from putting quotes around some file names
(those that have astral-plane characters) and not around others.

R=maruel

Bug:  792302 
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
Reviewed-on: https://chromium-review.googlesource.com/815454
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>

[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/recipes/recipe_modules/tryserver/examples/full.expected/with_rietveld_patch_new.json
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/tests/git_drover_test.py
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/gclient_scm.py
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/recipes/recipe_modules/tryserver/examples/full.expected/with_wrong_patch_new.json
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/recipes/recipe_modules/tryserver/api.py
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/git_drover.py
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/git_cl.py
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/scm.py
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/recipes/recipe_modules/tryserver/examples/full.expected/with_git_patch.json
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/presubmit_support.py
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/recipes/README.recipes.md
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/recipes/recipe_modules/tryserver/examples/full.expected/with_rietveld_patch.json
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/recipes/recipe_modules/tryserver/examples/full.expected/with_wrong_patch.json
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/recipes/recipe_modules/tryserver/examples/full.expected/with_git_patch_luci.json
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/tests/gclient_scm_test.py
[modify] https://crrev.com/9219d356885a93d9cbcaf4a2c7fc3f6e29caad49/tests/git_cl_test.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/6aaee3065cf218d33d3ea549fdbfdf4972722586

commit 6aaee3065cf218d33d3ea549fdbfdf4972722586
Author: Robbie Iannucci <iannucci@chromium.org>
Date: Tue Dec 12 02:15:26 2017

Revert "Use core.quotePath=false when git is listing files"

This reverts commit 9219d356885a93d9cbcaf4a2c7fc3f6e29caad49.

Reason for revert: unfortunately this says "core,quotePath" and since it includes recipe changes, we need something that the roller can munch on :(

Original change's description:
> Use core.quotePath=false when git is listing files
> 
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
> 
> R=​maruel
> 
> Bug:  792302 
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>

TBR=maruel@chromium.org,agable@chromium.org

Change-Id: I226388f19024403240a1443eb2b878b9293220e1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  792302 
Reviewed-on: https://chromium-review.googlesource.com/821671
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/recipes/recipe_modules/tryserver/examples/full.expected/with_rietveld_patch_new.json
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/tests/git_drover_test.py
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/gclient_scm.py
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/recipes/recipe_modules/tryserver/examples/full.expected/with_wrong_patch_new.json
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/recipes/recipe_modules/tryserver/api.py
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/git_drover.py
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/git_cl.py
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/scm.py
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/recipes/recipe_modules/tryserver/examples/full.expected/with_git_patch.json
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/presubmit_support.py
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/recipes/README.recipes.md
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/recipes/recipe_modules/tryserver/examples/full.expected/with_rietveld_patch.json
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/recipes/recipe_modules/tryserver/examples/full.expected/with_wrong_patch.json
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/recipes/recipe_modules/tryserver/examples/full.expected/with_git_patch_luci.json
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/tests/gclient_scm_test.py
[modify] https://crrev.com/6aaee3065cf218d33d3ea549fdbfdf4972722586/tests/git_cl_test.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/7817f025433cae38174365d8964333befd26d01b

commit 7817f025433cae38174365d8964333befd26d01b
Author: Aaron Gable <agable@chromium.org>
Date: Tue Dec 12 18:26:25 2017

Reland "Use core.quotePath=false when git is listing files"

This is a reland of 9219d356885a93d9cbcaf4a2c7fc3f6e29caad49

The original was reverted due to a typo (core,quotePath instead
of core.quotePath). This version is fixed.

Original change's description:
> Use core.quotePath=false when git is listing files
>
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
>
> R=maruel
>
> Bug:  792302 
>
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>

Bug:  792302 
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I28d2260948aaf63bd865888c2f60e4cdee9aea48
Reviewed-on: https://chromium-review.googlesource.com/822990
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/recipes/recipe_modules/tryserver/examples/full.expected/with_rietveld_patch_new.json
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/tests/git_drover_test.py
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/gclient_scm.py
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/recipes/recipe_modules/tryserver/examples/full.expected/with_wrong_patch_new.json
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/recipes/recipe_modules/tryserver/api.py
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/git_drover.py
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/git_cl.py
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/scm.py
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/recipes/recipe_modules/tryserver/examples/full.expected/with_git_patch.json
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/presubmit_support.py
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/recipes/README.recipes.md
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/recipes/recipe_modules/tryserver/examples/full.expected/with_rietveld_patch.json
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/recipes/recipe_modules/tryserver/examples/full.expected/with_wrong_patch.json
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/recipes/recipe_modules/tryserver/examples/full.expected/with_git_patch_luci.json
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/tests/gclient_scm_test.py
[modify] https://crrev.com/7817f025433cae38174365d8964333befd26d01b/tests/git_cl_test.py

The change above isn't enough. PRESUBMIT process the output of "git diff", and "git diff" names files differently (using \\302\\105\\whatever escape sequences) than "git status", "git ls-files", and other operations do.

Right now, if you modify one of the unicode-named files, then "git cl presubmit" thinks you haven't modified anything and quits early.

FWIW git diff does honor core.quotePath:

src$ git config core.quotePath true
src$ git diff 
diff --git "a/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~\342\200\276\302\245\342\211\210\302\244\357\275\245\343\203\273\342\200\242\342\210\231\302\267\342\230\274\342\230\205\346\230\237\360\237\214\237\346\230\237\342\230\205\342\230\274\302\267\342\210\231\342\200\242\343\203\273\357\275\245\302\244\342\211\210\302\245\342\200\276~XYZ.txt" "b/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~\342\200\276\302\245\342\211\210\302\244\357\275\245\343\203\273\342\200\242\342\210\231\302\267\342\230\274\342\230\205\346\230\237\360\237\214\237\346\230\237\342\230\205\342\230\274\302\267\342\210\231\342\200\242\343\203\273\357\275\245\302\244\342\211\210\302\245\342\200\276~XYZ.txt"
index 5471fb58cb30..70ae43f38bea 100644
--- "a/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~\342\200\276\302\245\342\211\210\302\244\357\275\245\343\203\273\342\200\242\342\210\231\302\267\342\230\274\342\230\205\346\230\237\360\237\214\237\346\230\237\342\230\205\342\230\274\302\267\342\210\231\342\200\242\343\203\273\357\275\245\302\244\342\211\210\302\245\342\200\276~XYZ.txt"
+++ "b/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~\342\200\276\302\245\342\211\210\302\244\357\275\245\343\203\273\342\200\242\342\210\231\302\267\342\230\274\342\230\205\346\230\237\360\237\214\237\346\230\237\342\230\205\342\230\274\302\267\342\210\231\342\200\242\343\203\273\357\275\245\302\244\342\211\210\302\245\342\200\276~XYZ.txt"
@@ -1 +1,2 @@
-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ
\ No newline at end of file
+wut
+ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ
src$ git config core.quotePath false
src$ git diff 
diff --git a/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ.txt b/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ.txt
index 5471fb58cb30..70ae43f38bea 100644
--- a/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ.txt
+++ b/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ.txt
@@ -1 +1,2 @@
-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ
\ No newline at end of file
+wut
+ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ

Huh, I tried that already. Trying to figure out why it didn't work for me.

I think it also honors locale (e.g. LANG=en_US.UTF-8 ) - what does 'locale charmap' show?

$ LC_ALL=C git diff 
diff --git a/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~<E2><80><BE><C2><A5><E2><89><88><C2><A4><EF>
<BD><A5><E3><83><BB><E2><80><A2><E2><88><99><C2><B7><E2><98><BC><E2><98><85><E6><98><9F><F0><9F><8C><9F><E6><98><9F><E2><98><85><E2><98><BC><C2><B7><E2><88><99><E2><80><A2><E3><83><BB><EF><BD><A5><C2><A4><E2><89><88><C2><A5><E2><80><BE>~XYZ.txt b/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~<E2><80><BE><C2><A5><E2><89><88><C2><A4><EF><BD><A5><E3><83><BB><E2><80><A2><E2><88><99><C2><B7><E2><98><BC><E2><98><85><E6><98><9F><F0><9F><8C><9F><E6><98><9F><E2><98><85><E2><98><BC><C2><B7><E2><88><99><E2><80><A2><E3><83><BB><EF><BD><A5><C2><A4><E2><89><88><C2><A5><E2><80><BE>~XYZ.txt
index 5471fb58cb30..70ae43f38bea 100644
--- a/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~<E2><80><BE><C2><A5><E2><89><88><C2><A4><EF><BD><A5>
<E3><83><BB><E2><80><A2><E2><88><99><C2><B7><E2><98><BC><E2><98><85><E6><98><9F><F0><9F><8C><9F><E6><98><9F><E2><98><85><E2><98><BC><C2><B7><E2><88><99><E2><80><A2><E3><83><BB><EF><BD><A5><C2><A4><E2><89><88><C2><A5><E2><80><BE>~XYZ.txt
+++ b/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~<E2><80><BE><C2><A5><E2><89><88><C2><A4><EF><BD><A5>
<E3><83><BB><E2><80><A2><E2><88><99><C2><B7><E2><98><BC><E2><98><85><E6><98><9F><F0><9F><8C><9F><E6><98><9F><E2><98><85><E2><98><BC><C2><B7><E2><88><99><E2><80><A2><E3><83><BB><EF><BD><A5><C2><A4><E2><89><88><C2><A5><E2><80><BE>~XYZ.txt
@@ -1 +1,2 @@
-ABC~<E2><80><BE><C2><A5><E2><89><88><C2><A4><EF><BD><A5><E3><83><BB><E2><80><A2><E2><88><99><C2><B7><E2><98><BC><E2><98><85><E6><98><9F><F0><9F><8C>
<9F><E6><98><9F><E2><98><85><E2><98><BC><C2><B7><E2><88><99><E2><80><A2><E3><83><BB><EF><BD><A5><C2><A4><E2><89><88><C2><A5><E2><80><BE>~XYZ
\ No newline at end of file
+wut
+ABC~<E2><80><BE><C2><A5><E2><89><88><C2><A4><EF><BD><A5><E3><83><BB><E2><80><A2><E2><88><99><C2><B7><E2><98><BC><E2><98><85><E6><98><9F><F0><9F><8C>
<9F><E6><98><9F><E2><98><85><E2><98><BC><C2><B7><E2><88><99><E2><80><A2><E3><83><BB><EF><BD><A5><C2><A4><E2><89><88><C2><A5><E2><80><BE>~XYZ

At least for me, LESSCHARSET (when present) also influences it.

$ LESSCHARSET=UTF-8 LC_ALL=C  git diff 
diff --git a/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ.txt b/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ.txt
index 5471fb58cb30..70ae43f38bea 100644
--- a/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ.txt
+++ b/third_party/WebKit/LayoutTests/http/tests/local/fileapi/resources/file-for-drag-to-send3-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ.txt
@@ -1 +1,2 @@
-ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ
\ No newline at end of file
+wut
+ABC~‾¥≈¤･・•∙·☼★星🌟星★☼·∙•・･¤≈¥‾~XYZ

Further fixes here: https://chromium-review.googlesource.com/#/c/chromium/tools/depot_tools/+/823131

I'm not sure why core.quotePath=false wasn't working for git diff for me in my manual tests; it definitely works from inside gclient.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/f4068aa3ea9440212295af8cdd07dc35fe9dddcd

commit f4068aa3ea9440212295af8cdd07dc35fe9dddcd
Author: Aaron Gable <agable@chromium.org>
Date: Tue Dec 12 23:28:19 2017

Use core.quotePath=false for more git diffs

This includes a minor refactor so that some gclient_scm methods
can all share the same core.quotePath specifier.

R=iannucci

Bug:  792302 
Change-Id: Iaadf190f5c0666787cf7c2ccda88d6dba9aace9b
Reviewed-on: https://chromium-review.googlesource.com/823131
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>

[modify] https://crrev.com/f4068aa3ea9440212295af8cdd07dc35fe9dddcd/checkout.py
[modify] https://crrev.com/f4068aa3ea9440212295af8cdd07dc35fe9dddcd/gclient_scm.py
[modify] https://crrev.com/f4068aa3ea9440212295af8cdd07dc35fe9dddcd/git_cl.py
[modify] https://crrev.com/f4068aa3ea9440212295af8cdd07dc35fe9dddcd/scm.py
[modify] https://crrev.com/f4068aa3ea9440212295af8cdd07dc35fe9dddcd/tests/gclient_scm_test.py
[modify] https://crrev.com/f4068aa3ea9440212295af8cdd07dc35fe9dddcd/git_upstream_diff.py

I believe this is fixed. At least, "git cl owners" works on my own test change which touches one of the unicode-named files.