cros-workon: rsync filter rules don't support .gitignore ! negation |
|||||
Issue descriptioncrrev.com/c/845474 updated cros-workon.eclass to set rsync --filter=":- .gitignore" to honor .gitignore on copying files locally. However rsync's supported set of filter patterns is not the same as that of git, so it may cause unexpected results. In fact, tfiga@ reported an issue about .gitignore using negative patterns, such as: gen*_pack.h !genX_pack.h
,
Feb 16 2018
fortunately, a cs/ suggests we basically never use this feature. however, i agree it would make sense to do so. going through the rsync documentation, i don't think the existing rules/formats make this feasible. i think the only way to pull this off would be to add support for the gitignore format directly to rsync and send that patch upstream. considering the prevalence of git, and that rsync has support for .cvsignore already, i don't think it'd be a tough sell. basically: - add a new --git-exclude command line option - extend the parser backend to support this format - extend the merge-file filter logic to have a new "G" keyword (like there is a "C" keyword already for CVS) then we'd be able to update cros-workon to use --filter=":G" and everything would be peachy.
,
Feb 19 2018
tfiga@ suggested another way to ask git to list ignored files and pass it to rsync. For example: $ git ls-files --others --ignored --exclude-standard --directory
,
Feb 19 2018
that might work if we did something like: ... --exclude-from=<(git ls-files --others --ignored --exclude-standard --directory) ...
,
Feb 19 2018
Or perhaps git ls-files --others --ignored --exclude-standard --directory | rsync [...] --exclude-from=- [...] as man seems to mention that you can give "-" to --exclude-from and the list will be read from stdin.
,
Feb 19 2018
process substitution is preferable as you can run multiple commands to read from, and you won't have a pipeline to deal with. both otherwise that seems like it'd work in this specific case.
,
Feb 19 2018
Fair enough. Thanks for explaining. Let me hack up a CL quickly. To be honest I didn't even know such thing existed. (Yes, it's embarrassing. ;))
,
Feb 19 2018
,
Feb 19 2018
,
Feb 19 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/3110e22b5b6ed23ef99b686372637301cbbf4129 commit 3110e22b5b6ed23ef99b686372637301cbbf4129 Author: Mike Frysinger <vapier@chromium.org> Date: Mon Feb 19 10:36:03 2018 gitignore: delete We don't keep any of these binary types in the tree, nor do we want to. So drop this gitignore rule that permits them. BUG= chromium:812979 TEST=precq passes Change-Id: I180ecd6cd18baf46d1faa31d44d16fd557845715 Reviewed-on: https://chromium-review.googlesource.com/924348 Commit-Ready: Mike Frysinger <vapier@chromium.org> Tested-by: Mike Frysinger <vapier@chromium.org> Reviewed-by: Shuhei Takahashi <nya@chromium.org> [delete] https://crrev.com/3b8e1aae5207561b910f1cd44d0415361cd6c151/.gitignore
,
Feb 20 2018
,
Feb 21 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/b6f21b6b6e9164cfdd1d37fa9290592ee762a3d2 commit b6f21b6b6e9164cfdd1d37fa9290592ee762a3d2 Author: Tomasz Figa <tfiga@chromium.org> Date: Wed Feb 21 12:27:06 2018 cros-workon: Use git ls-files for excluding files from copying The --filter=":- <file>" switch of rsync, used currently, does not fully handle .gitignore syntax and would behave incorrectly at least in following two example cases: a) negative matches gen*_pack.h !genX_pack.h anything matching the first pattern, except anything matching the second one too, should be ignored, but rsync reads the exclamation mark as a part of file name pattern. b) files checked in, but present in .gitignore - even though it's questionable whether this is a correct repository setup, the affected files are there in the checkout and so should be also included in the source tree used for compilation. Use git ls-files to query the list of ignored files instead and use the output with --exclude-from rsync switch. This ensures that file exclusion works exactly the same way as in git. BUG= chromium:812979 BUG= chromium:812982 TEST=Create following directory stucture in src/third_party/arc-mesa x/y/ignored x/y/not_ignored y/ignored y/not_ignored y/.gitignore and put following contents in y/.gitignore ignored emerge-soraka arc-mesa copies all files except y/ignored. Change-Id: I459a066c41f8effb2f7d655f324ab7af645bcd2e Reviewed-on: https://chromium-review.googlesource.com/925222 Commit-Ready: Tomasz Figa <tfiga@chromium.org> Tested-by: Tomasz Figa <tfiga@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/b6f21b6b6e9164cfdd1d37fa9290592ee762a3d2/eclass/cros-workon.eclass
,
Feb 21 2018
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by tfiga@chromium.org
, Feb 16 2018