CIPD should have a "resolved versions file" with hashes of all packages in an ensure file |
||
Issue description
Currently almost all ensure files use cipd tags to specify package versions, e.g. "git_revision:2dd3271a53e061ec44da...".
When cipd installs packages from such ensure file it uses ResolveVersion(...) backend RPC to convert the tag into a concrete package instance (identifies by SHA256 of the package file content). It then fetches the package file, calculates its SHA256, compares it to the expect one, and proceeds with the installation.
The problem with this approach is that the caller places full trust into CIPD backend server, since the server can return whatever it wants in ResolveVersion(...).
In many situations this is not OK. We want the trust to be rooted in a git repository that contains the ensure file. It means we need to resolve all versions into SHA256 hashes in advance, commit this hashes in the repo, and tell CIPD client to use this hashes exclusively when installing the ensure file.
The concrete proposal is to introduce a new directive into the ensure file:
$ResolvedVersions path_relative_to_the_ensure_file
And introduce new subcommand:
"cipd ensure-resolve <ensure-file> [-check]"
The 'ensure-resolve' subcommand would look at all $VerifiedPlatforms in the ensure file (e.g. "linux-amd64", "mac-amd64", ...) and use them to generate all needed permutations of ${platform}, ${os}, ${arch} vars and substitute them into package name templates.
The output of this process is a list of pairs (package name, version). It will resolve each version into a concrete SHA256 instance ID (using ResolveVersion call), and put them into the file specified by $ResolvedVersions.
The format of the file is:
"""
# Autogenerated from <ensure_file_name>. Do not edit.
<pkg name>
<version>
<instance ID>
<pkg name>
<version>
<instance ID>
...
"""
(Leading and trailing spaces are insignificant. An empty one or more new lines between entries is significant, to allow to extend this format with more per-entry stuff later if needed. Entries are sorted alphabetically by (pkg name, version) key).
E.g.
"""
infra/tools/cipd/linux-amd64
git_revision:82ac2c5d0ab63d39f4f4bb83de992f9d0525cfb5
oMVjfwD4oa7kihzUk-bxpMachNcYgoPg4X3k4M3YRfgC
infra/tools/cipd/mac-amd64
git_revision:82ac2c5d0ab63d39f4f4bb83de992f9d0525cfb5
alsdkjalsdjkaoa7kihzUk-bxpMachNcYgoPadasdasC
"""
(This is bikeshadeable. I propose this format because it avoid overlong lines and presumably would result in nice-ish diffs when committing updates. We can also stop inventing DSLs and use e.g. YAML or JSON, though they are more verbose).
'cipd ensure-resolve -check' will return 0 iif the current resolved file is up-to-date (for presubmit tests).
If $ResolvedVersions is specified in the ensure file, 'cipd ensure' will exclusively use resolved versions from it, never contacting the backend. If some (package, version) pair is missing, it produces an error.
----
An alternative is to stuff the resolved versions into the existing ensure file. It has downsides:
1. The ensure file becomes mixed manual/autogenerated, which complicates life.
2. The file will become much fatter.
3. gclient doesn't even use a committed ensure file (but it still can use external resolved file, by referencing it from an ensure file it generates).
----
Some other issues:
1. We'll be potentially disclosing some information (SHA256) about packages that aren't accessible to the caller (e.g. if a git reader ACL is more permissive than CIPD reader ACL). Probably not a big deal.
2. Currently it is possible to retroactively add a platform to "old" ensure file. With committed resolved version, we'll need to know all possible values of ${platform} at the time of the commit. Also probably not a big deal.
3. If some repos reference a moving ref in an ensure file (e.g. 'latest'), they'll have to setup an auto-roller.
4. We may potential want to add support for this scheme into Buildbucket task templates and Puppet/Salt. This may not be trivial...
,
Aug 3
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/581fbd2989842b16b6ce80d83f99a3da8ef5fadb commit 581fbd2989842b16b6ce80d83f99a3da8ef5fadb Author: Vadim Shtayura <vadimsh@chromium.org> Date: Fri Aug 03 00:07:58 2018 [common/sync] Add promise.Map type. It's a map of promises, for when we need to wait for keyed things. Will be used in a following CL that refactors cipd.Verifier to be simpler and more parallel. R=iannucci@chromium.org, nodir@chromium.org BUG= 870166 Change-Id: Iac563f74a3f0d663bf37a7d25b3b311fb7604dc3 Reviewed-on: https://chromium-review.googlesource.com/1161488 Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [add] https://crrev.com/581fbd2989842b16b6ce80d83f99a3da8ef5fadb/common/sync/promise/map.go [add] https://crrev.com/581fbd2989842b16b6ce80d83f99a3da8ef5fadb/common/sync/promise/map_test.go
,
Aug 3
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/cafd7ffba42b2d09bde8a32fbf796efb0e167e89 commit cafd7ffba42b2d09bde8a32fbf796efb0e167e89 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Fri Aug 03 23:33:04 2018 [cipd] Rename cipd.Verifier to cipd.Resolver, simplify and speed it up a bit. Its API is reduced to only 2 functions: * ResolvePackage * ResolveEnsureFile 'ensure' package API has also been shrunk a bit. The version resolution is now also done in parallel (even in 'cipd ensure'). R=iannucci@chromium.org BUG= 870166 Change-Id: I3733ba1372669fb10db3ab38b9ab8ad91a758d54 Reviewed-on: https://chromium-review.googlesource.com/1161503 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/cafd7ffba42b2d09bde8a32fbf796efb0e167e89/cipd/client/cipd/ensure/bad_test.go [modify] https://crrev.com/cafd7ffba42b2d09bde8a32fbf796efb0e167e89/cipd/client/cipd/ensure/file.go [modify] https://crrev.com/cafd7ffba42b2d09bde8a32fbf796efb0e167e89/cipd/client/cipd/ensure/good_test.go [modify] https://crrev.com/cafd7ffba42b2d09bde8a32fbf796efb0e167e89/cipd/client/cipd/ensure/package_def.go [add] https://crrev.com/cafd7ffba42b2d09bde8a32fbf796efb0e167e89/cipd/client/cipd/resolver.go [add] https://crrev.com/cafd7ffba42b2d09bde8a32fbf796efb0e167e89/cipd/client/cipd/resolver_test.go [modify] https://crrev.com/cafd7ffba42b2d09bde8a32fbf796efb0e167e89/cipd/client/cipd/template/template.go [delete] https://crrev.com/0c7a0735965f3fe40fe6e3b3bae0ed7d26f2cc48/cipd/client/cipd/verify.go [modify] https://crrev.com/cafd7ffba42b2d09bde8a32fbf796efb0e167e89/cipd/client/cli/main.go [modify] https://crrev.com/cafd7ffba42b2d09bde8a32fbf796efb0e167e89/vpython/cipd/cipd.go
,
Aug 3
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/cec6132b5f4b5b11b0e41a968f2a93e051edcedc commit cec6132b5f4b5b11b0e41a968f2a93e051edcedc Author: Vadim Shtayura <vadimsh@chromium.org> Date: Fri Aug 03 23:38:09 2018 [cipd] Move version resolution for all platforms to cipd.Resolver. It is going to be reused for 'ensure-file-resolve' subcommand. It also now has a test. R=iannucci@chromium.org BUG= 870166 Change-Id: I7eb2675e12349c73ddea41720159f953b91f7098 Reviewed-on: https://chromium-review.googlesource.com/1161607 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/cec6132b5f4b5b11b0e41a968f2a93e051edcedc/cipd/client/cipd/resolver.go [modify] https://crrev.com/cec6132b5f4b5b11b0e41a968f2a93e051edcedc/cipd/client/cipd/resolver_test.go [modify] https://crrev.com/cec6132b5f4b5b11b0e41a968f2a93e051edcedc/cipd/client/cli/main.go
,
Aug 7
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/f4a480f422006069db64576c92f3d60a25c08eb9 commit f4a480f422006069db64576c92f3d60a25c08eb9 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Tue Aug 07 00:13:34 2018 [cipd] Fix flaky resolver test. It depended on the order of map iteration. TBR=iannucci@chromium.org BUG= 870166 Change-Id: I486929b83d32fdebb1264864e63a25df70bb827e Reviewed-on: https://chromium-review.googlesource.com/1164537 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/f4a480f422006069db64576c92f3d60a25c08eb9/cipd/client/cipd/ensure/file.go
,
Aug 7
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/3bbcb4491334b200a745e219a5b0f639da7733d2 commit 3bbcb4491334b200a745e219a5b0f639da7733d2 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Tue Aug 07 00:44:14 2018 [cipd] Add $ResolvedVersions parser and serializer + doc. Such file holds a mapping "(pkg name, version) -> instance ID" and it is serialized as a list of triples separated by a new line: """ <package name> <version> <resolved instance ID> <package name> <version> <resolved instance ID> """ R=iannucci@chromium.org BUG= 870166 Change-Id: I946ec00682848b83321c4945b5c6d30f07f22900 Reviewed-on: https://chromium-review.googlesource.com/1162978 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/3bbcb4491334b200a745e219a5b0f639da7733d2/cipd/client/cipd/ensure/doc.go [modify] https://crrev.com/3bbcb4491334b200a745e219a5b0f639da7733d2/cipd/client/cipd/ensure/file.go [modify] https://crrev.com/3bbcb4491334b200a745e219a5b0f639da7733d2/cipd/client/cipd/ensure/file_test.go [modify] https://crrev.com/3bbcb4491334b200a745e219a5b0f639da7733d2/cipd/client/cipd/ensure/good_test.go [modify] https://crrev.com/3bbcb4491334b200a745e219a5b0f639da7733d2/cipd/client/cipd/ensure/item_parsers.go [add] https://crrev.com/3bbcb4491334b200a745e219a5b0f639da7733d2/cipd/client/cipd/ensure/loading.go [add] https://crrev.com/3bbcb4491334b200a745e219a5b0f639da7733d2/cipd/client/cipd/ensure/versions_file.go [add] https://crrev.com/3bbcb4491334b200a745e219a5b0f639da7733d2/cipd/client/cipd/ensure/versions_file_test.go [modify] https://crrev.com/3bbcb4491334b200a745e219a5b0f639da7733d2/cipd/client/cli/main.go
,
Aug 9
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/1aacd0d8fea5896c1930dca95f529300be53217a commit 1aacd0d8fea5896c1930dca95f529300be53217a Author: Vadim Shtayura <vadimsh@chromium.org> Date: Thu Aug 09 02:59:28 2018 [cipd] Dedup -ensure-file and -root flags definitions. They are copy-pasted too much. Soon to be added 'ensure-file-resolve' would have copy-pasted -ensure-file one more time. R=iannucci@chromium.org BUG= 870166 Change-Id: Id09995f0f7e17fa9222f17a1cfdd69ca898df5fe Reviewed-on: https://chromium-review.googlesource.com/1164623 Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/1aacd0d8fea5896c1930dca95f529300be53217a/cipd/client/cli/main.go
,
Aug 9
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/60017af8df79eb85d041b3c9c79c904ba14605e2 commit 60017af8df79eb85d041b3c9c79c904ba14605e2 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Thu Aug 09 20:27:16 2018 [cipd] Implement 'ensure-file-resolve' subcommand. It generates the file specified by $ResolvedVersions directive in the ensure file. As -json-output it returns same structured pin map as 'ensure-file-verify', for consistency. 'ensure-file-verify' now also checks that the resolve versions file (if configured) is up-to-date. R=iannucci@chromium.org BUG= 870166 Change-Id: I68a713586776b0dbabe6e403809785d61f43bd69 Reviewed-on: https://chromium-review.googlesource.com/1164632 Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/60017af8df79eb85d041b3c9c79c904ba14605e2/cipd/client/cipd/ensure/versions_file.go [modify] https://crrev.com/60017af8df79eb85d041b3c9c79c904ba14605e2/cipd/client/cipd/ensure/versions_file_test.go [modify] https://crrev.com/60017af8df79eb85d041b3c9c79c904ba14605e2/cipd/client/cipd/resolver.go [modify] https://crrev.com/60017af8df79eb85d041b3c9c79c904ba14605e2/cipd/client/cipd/resolver_test.go [modify] https://crrev.com/60017af8df79eb85d041b3c9c79c904ba14605e2/cipd/client/cli/main.go
,
Aug 9
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/98d6adae8e3f2f9b8bb070ac17df4d2b5af269d3 commit 98d6adae8e3f2f9b8bb070ac17df4d2b5af269d3 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Thu Aug 09 20:33:52 2018 [cipd] Teach 'cipd ensure' to actually use $ResolvedVersions. R=iannucci@chromium.org BUG= 870166 Change-Id: I50edfcbea6a3ab65ea206ab2dacef7946351fdf5 Reviewed-on: https://chromium-review.googlesource.com/1166444 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/98d6adae8e3f2f9b8bb070ac17df4d2b5af269d3/cipd/client/cipd/client.go [modify] https://crrev.com/98d6adae8e3f2f9b8bb070ac17df4d2b5af269d3/cipd/client/cipd/ensure/versions_file.go [modify] https://crrev.com/98d6adae8e3f2f9b8bb070ac17df4d2b5af269d3/cipd/client/cipd/ensure/versions_file_test.go [modify] https://crrev.com/98d6adae8e3f2f9b8bb070ac17df4d2b5af269d3/cipd/client/cli/main.go
,
Aug 10
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/3108b3dcbfff4c0c7bed6b4c1a8f4ca0d10ed2ca commit 3108b3dcbfff4c0c7bed6b4c1a8f4ca0d10ed2ca Author: Vadim Shtayura <vadimsh@chromium.org> Date: Fri Aug 10 20:08:30 2018 [cipd] Add ClientDigestsFile struct + serializer and parser. It holds list of tuples (platform, hash algo, hex digest) with locked digests of the CIPD client binary. R=iannucci@chromium.org BUG= 870166 Change-Id: Ia8ed7c19d2ccd5b8d77c9dbf5912c045c890c4b2 Reviewed-on: https://chromium-review.googlesource.com/1170433 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [add] https://crrev.com/3108b3dcbfff4c0c7bed6b4c1a8f4ca0d10ed2ca/cipd/client/cipd/digests/client_digests_file.go [add] https://crrev.com/3108b3dcbfff4c0c7bed6b4c1a8f4ca0d10ed2ca/cipd/client/cipd/digests/client_digests_file_test.go
,
Aug 10
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/7ba8da7abb17461d84464bdea1fb41020938a8de commit 7ba8da7abb17461d84464bdea1fb41020938a8de Author: Vadim Shtayura <vadimsh@chromium.org> Date: Fri Aug 10 20:14:13 2018 [cipd] Add DescribeClient method to cipd.Client. Will be used to generate *.digests file with locked hashes of the CIPD client binary. R=iannucci@chromium.org BUG= 870166 Change-Id: I67fd7920045ac589a861daf644c96a358a194a3d Reviewed-on: https://chromium-review.googlesource.com/1170435 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/7ba8da7abb17461d84464bdea1fb41020938a8de/cipd/client/cipd/client.go [modify] https://crrev.com/7ba8da7abb17461d84464bdea1fb41020938a8de/cipd/client/cipd/client_test.go [modify] https://crrev.com/7ba8da7abb17461d84464bdea1fb41020938a8de/cipd/client/cipd/json_structs.go
,
Aug 10
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/37e98b172e4d0feac4e184c927a448bee136a7fb commit 37e98b172e4d0feac4e184c927a448bee136a7fb Author: Vadim Shtayura <vadimsh@chromium.org> Date: Fri Aug 10 20:19:43 2018 [cipd] Implement generation and reading of *.digests file. It is involved during self-update procedure. It holds the expected hashes of the client binary (per platform). It is supposed to be committed side by side with scripts that do CIPD client bootstrap, so that they can check the hash of the file they download. Three new flavors of 'selfupdate' call: * 'cipd selfupdate -version-file <path>': reads the version to update to from the given file, and reads expected client digests from <path>.digests, then fetches the new client and verifies the hash is know (this last part is still TODO). * 'cipd selfupdate -version-file <path> -generate-digests': resolves the client version, fetches digests of the client binary and writes them into <path>.digests. * 'cipd selfupdate -version-file <path> -check-digests': returns 0 iff the digests file is up-to-date (i.e. regenerating it now is noop). R=iannucci@chromium.org BUG= 870166 Change-Id: Id73b2326f88562e5e0b91088bde91293fd745148 Reviewed-on: https://chromium-review.googlesource.com/1170436 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/37e98b172e4d0feac4e184c927a448bee136a7fb/cipd/client/cipd/client.go [modify] https://crrev.com/37e98b172e4d0feac4e184c927a448bee136a7fb/cipd/client/cipd/client_test.go [modify] https://crrev.com/37e98b172e4d0feac4e184c927a448bee136a7fb/cipd/client/cli/main.go
,
Aug 13
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/747f83c5facfcff93bbd0af42c6e207a3cdf45af commit 747f83c5facfcff93bbd0af42c6e207a3cdf45af Author: Vadim Shtayura <vadimsh@chromium.org> Date: Mon Aug 13 21:24:46 2018 [cipd] Verify CIPD client binary hash against *.digests when self-updating. R=iannucci@chromium.org, nodir@chromium.org BUG= 870166 Change-Id: Ib91862300b683ce16208d8e1282605c265375de6 Reviewed-on: https://chromium-review.googlesource.com/1171662 Reviewed-by: Nodir Turakulov <nodir@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/747f83c5facfcff93bbd0af42c6e207a3cdf45af/cipd/client/cipd/client.go [modify] https://crrev.com/747f83c5facfcff93bbd0af42c6e207a3cdf45af/cipd/client/cipd/client_test.go [modify] https://crrev.com/747f83c5facfcff93bbd0af42c6e207a3cdf45af/cipd/client/cipd/install_client.go [modify] https://crrev.com/747f83c5facfcff93bbd0af42c6e207a3cdf45af/cipd/client/cipd/install_client_windows.go [modify] https://crrev.com/747f83c5facfcff93bbd0af42c6e207a3cdf45af/cipd/client/cipd/json_structs.go [modify] https://crrev.com/747f83c5facfcff93bbd0af42c6e207a3cdf45af/cipd/client/cipd/platform/platform.go [modify] https://crrev.com/747f83c5facfcff93bbd0af42c6e207a3cdf45af/cipd/client/cli/main.go
,
Aug 13
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/88e481ab9969f106176458dcf8a8525b1349b5dc commit 88e481ab9969f106176458dcf8a8525b1349b5dc Author: Vadim Shtayura <vadimsh@chromium.org> Date: Mon Aug 13 21:51:16 2018 Roll infra/go/src/go.chromium.org/luci/ 0b67d5886..747f83c5f (8 commits) https://chromium.googlesource.com/infra/luci/luci-go/+log/0b67d5886e5f..747f83c5facf $ git log 0b67d5886..747f83c5f --date=short --no-merges --format='%ad %ae %s' 2018-08-13 vadimsh [cipd] Verify CIPD client binary hash against *.digests when self-updating. 2018-08-11 vadimsh [cipd] Stop escaping '&' in JSON output. 2018-08-10 vadimsh [cipd] Make JSON replies in the handlers have spaces after ':'. 2018-08-10 tikuta [bq] Upload data to bigquery concurrently 2018-08-10 vadimsh [cipd] Implement generation and reading of *.digests file. 2018-08-10 vadimsh [cipd] Add DescribeClient method to cipd.Client. 2018-08-10 vadimsh [cipd] Add ClientDigestsFile struct + serializer and parser. 2018-08-10 tandrii remove cq_name from cq.cfg Created with: roll-dep infra/go/src/go.chromium.org/luci TBR=nodir@chromium.org BUG= 870166 Change-Id: I31e5741a11135ef4e8c2fdf1917c667bf0d6fc77 Reviewed-on: https://chromium-review.googlesource.com/1173416 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/88e481ab9969f106176458dcf8a8525b1349b5dc/DEPS
,
Aug 14
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/930e1ac5dd787bb24fc8d579a0446afc3ac69cbf commit 930e1ac5dd787bb24fc8d579a0446afc3ac69cbf Author: Vadim Shtayura <vadimsh@chromium.org> Date: Tue Aug 14 18:48:25 2018 [cipd] Move *.digests generation and checks into selfupdate-roll subcommand. Also teach it to write to the version file, so to roll a pinned client all we have to do is to run: cipd selfupdate-roll -version-file <path> -version <new-version> The primary reason we do it at once is that in depot_tools (and probably other places where the client is pinned), modifying <version-file> manually makes 'cipd' invocation try to self-update to this new version next time the client runs. So when running 'cipd selfupdate ... -generate-digests' to regenerate *.digests we hit the self-update code path with OLD *.digests file, and the self-update rightfully fails. R=nodir@chromium.org, iannucci@chromium.org BUG= 870166 Change-Id: I18a57a12fd6b34d45f62ea52036d31575d36f091 Reviewed-on: https://chromium-review.googlesource.com/1173728 Reviewed-by: Nodir Turakulov <nodir@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/930e1ac5dd787bb24fc8d579a0446afc3ac69cbf/cipd/client/cipd/client.go [modify] https://crrev.com/930e1ac5dd787bb24fc8d579a0446afc3ac69cbf/cipd/client/cipd/digests/client_digests_file.go [modify] https://crrev.com/930e1ac5dd787bb24fc8d579a0446afc3ac69cbf/cipd/client/cipd/digests/client_digests_file_test.go [modify] https://crrev.com/930e1ac5dd787bb24fc8d579a0446afc3ac69cbf/cipd/client/cli/main.go
,
Aug 14
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/ea6c07cfcb596be6b63a1e6deb95bba79524b0c8 commit ea6c07cfcb596be6b63a1e6deb95bba79524b0c8 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Tue Aug 14 19:10:23 2018 Roll infra/go/src/go.chromium.org/luci/ 747f83c5f..930e1ac5d (2 commits) https://chromium.googlesource.com/infra/luci/luci-go/+log/747f83c5facf..930e1ac5dd78 $ git log 747f83c5f..930e1ac5d --date=short --no-merges --format='%ad %ae %s' 2018-08-14 vadimsh [cipd] Move *.digests generation and checks into selfupdate-roll subcommand. 2018-08-13 vadimsh [cipd] Add a page with basic information about package instances. Created with: roll-dep infra/go/src/go.chromium.org/luci TBR=nodir@chromium.org BUG= 870166 Change-Id: I4c4b367eeefaf5067aab888f10abd309c9be5d80 Reviewed-on: https://chromium-review.googlesource.com/1174966 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/ea6c07cfcb596be6b63a1e6deb95bba79524b0c8/DEPS
,
Aug 14
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/eebc3d823220a0c3124e96ad4d577d32bbbeab6e commit eebc3d823220a0c3124e96ad4d577d32bbbeab6e Author: Vadim Shtayura <vadimsh@chromium.org> Date: Tue Aug 14 20:39:46 2018 [cipd] Check CIPD client hash against pinned SHA256 during updates. Linux and OSX only for now. This also rolls CIPD client to a version that supports pinned hashes (v2.2.5). CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since it makes no sense when pinning hashes of the binaries at specific version on the specific backend. Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output consistently. R=iannucci@chromium.org, nodir@chromium.org BUG= 870166 Change-Id: I9e61f9f8fbdcf10985c52828b2bfbec64b4234f0 Reviewed-on: https://chromium-review.googlesource.com/1171957 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/eebc3d823220a0c3124e96ad4d577d32bbbeab6e/cipd [modify] https://crrev.com/eebc3d823220a0c3124e96ad4d577d32bbbeab6e/cipd_client_version [add] https://crrev.com/eebc3d823220a0c3124e96ad4d577d32bbbeab6e/cipd_client_version.digests [modify] https://crrev.com/eebc3d823220a0c3124e96ad4d577d32bbbeab6e/tests/cipd_bootstrap_test.py
,
Aug 15
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/50f490e369c9ee4b070932e9da02ad2c1b7b2a41 commit 50f490e369c9ee4b070932e9da02ad2c1b7b2a41 Author: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Aug 15 04:20:18 2018 Roll src/third_party/depot_tools 1164342495fd..52c721180793 (3 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/1164342495fd..52c721180793 git log 1164342495fd..52c721180793 --date=short --no-merges --format='%ad %ae %s' 2018-08-14 tandrii@chromium.org Set explicit latest OWNERS for bot_update recipe_module. 2018-08-14 jbudorick@chromium.org bot_update: pull revisions from provided cfg if present. 2018-08-14 vadimsh@chromium.org [cipd] Check CIPD client hash against pinned SHA256 during updates. Created with: gclient setdep -r src/third_party/depot_tools@52c721180793 The AutoRoll server is located here: https://depot-tools-chromium-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG=chromium:873186, chromium:870166 TBR=agable@chromium.org Change-Id: I2ffab4553b1e83615deebd280c90efcb6c1bfb8f Reviewed-on: https://chromium-review.googlesource.com/1175291 Reviewed-by: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#583165} [modify] https://crrev.com/50f490e369c9ee4b070932e9da02ad2c1b7b2a41/DEPS
,
Aug 15
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/5e5c2173baec63edee49a7491468c594c98e4a7a commit 5e5c2173baec63edee49a7491468c594c98e4a7a Author: Vadim Shtayura <vadimsh@chromium.org> Date: Wed Aug 15 21:11:10 2018 Revert "[cipd] Check CIPD client hash against pinned SHA256 during updates." This reverts commit eebc3d823220a0c3124e96ad4d577d32bbbeab6e. Reason for revert: crbug.com/874586 Original change's description: > [cipd] Check CIPD client hash against pinned SHA256 during updates. > > Linux and OSX only for now. This also rolls CIPD client to a version that > supports pinned hashes (v2.2.5). > > CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since > it makes no sense when pinning hashes of the binaries at specific version on > the specific backend. > > Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output > consistently. > > R=​iannucci@chromium.org, nodir@chromium.org > BUG= 870166 > > Change-Id: I9e61f9f8fbdcf10985c52828b2bfbec64b4234f0 > Reviewed-on: https://chromium-review.googlesource.com/1171957 > Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> > Reviewed-by: Nodir Turakulov <nodir@chromium.org> TBR=iannucci@chromium.org,vadimsh@chromium.org,nodir@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 870166 Change-Id: I9aa8e7a7f07520aa69d366c76e4dbccae345bc00 Reviewed-on: https://chromium-review.googlesource.com/1175294 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/5e5c2173baec63edee49a7491468c594c98e4a7a/cipd [modify] https://crrev.com/5e5c2173baec63edee49a7491468c594c98e4a7a/cipd_client_version [delete] https://crrev.com/0065711927a4c79a7edc5ad27bcdc7a69d9425da/cipd_client_version.digests [modify] https://crrev.com/5e5c2173baec63edee49a7491468c594c98e4a7a/tests/cipd_bootstrap_test.py
,
Aug 16
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/3d429cf5131a10f0e6ced89e8809820398ece78b commit 3d429cf5131a10f0e6ced89e8809820398ece78b Author: Vadim Shtayura <vadimsh@chromium.org> Date: Thu Aug 16 00:15:08 2018 [cipd] Check CIPD client hash against pinned SHA256 during updates. Linux and OSX only for now. This also rolls CIPD client to a version that supports pinned hashes (v2.2.5). CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since it makes no sense when pinning hashes of the binaries at specific version on the specific backend. Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output consistently. R=iannucci@chromium.org, nodir@chromium.org BUG= 870166 , 874586 Change-Id: Iac67fbb6b5d07dcd81d44536737b03b146f1ad14 Reviewed-on: https://chromium-review.googlesource.com/1176727 Reviewed-by: Nodir Turakulov <nodir@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/3d429cf5131a10f0e6ced89e8809820398ece78b/cipd [modify] https://crrev.com/3d429cf5131a10f0e6ced89e8809820398ece78b/cipd_client_version [add] https://crrev.com/3d429cf5131a10f0e6ced89e8809820398ece78b/cipd_client_version.digests [modify] https://crrev.com/3d429cf5131a10f0e6ced89e8809820398ece78b/tests/cipd_bootstrap_test.py
,
Aug 16
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/58c3efcabfb467bd546392bd590969c55cc5b430 commit 58c3efcabfb467bd546392bd590969c55cc5b430 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Thu Aug 16 00:27:47 2018 [cipd] Make 'ensure-file-resolve' work. It had a stupid post-refactoring mistake that caused it to always load $ResolvedVersions file, even in 'ensure-file-resolve' where this file may not exist yet. TBR=nodir@chromium.org BUG= 870166 Change-Id: Idd398f4c837fd9f2d0b80f71a9b4ebce7dea051a Reviewed-on: https://chromium-review.googlesource.com/1176785 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/58c3efcabfb467bd546392bd590969c55cc5b430/cipd/client/cipd/client.go [modify] https://crrev.com/58c3efcabfb467bd546392bd590969c55cc5b430/cipd/client/cli/main.go
,
Aug 16
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1e345a1f7494bee312c7137476c048c784d10dd9 commit 1e345a1f7494bee312c7137476c048c784d10dd9 Author: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Thu Aug 16 02:15:21 2018 Roll src/third_party/depot_tools ba883cb5ed14..421bc3f84d1b (5 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/ba883cb5ed14..421bc3f84d1b git log ba883cb5ed14..421bc3f84d1b --date=short --no-merges --format='%ad %ae %s' 2018-08-16 tandrii@chromium.org Deploy git 2.18 to prod win depot_tools users. 2018-08-16 tandrii@chromium.org Register expect_tests in gclient config. 2018-08-15 rsesek@chromium.org Clarify OWNERS error if using TBR. 2018-08-15 vadimsh@chromium.org Revert "[cipd] Check CIPD client hash against pinned SHA256 during updates." 2018-08-15 jbudorick@chromium.org Grant everyone owners on README.recipes.md. Created with: gclient setdep -r src/third_party/depot_tools@421bc3f84d1b The AutoRoll server is located here: https://depot-tools-chromium-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG= chromium:851505 , chromium:870166 TBR=agable@chromium.org Change-Id: I110a7cde5025f046174287878daebf69c48a7286 Reviewed-on: https://chromium-review.googlesource.com/1176682 Reviewed-by: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#583501} [modify] https://crrev.com/1e345a1f7494bee312c7137476c048c784d10dd9/DEPS
,
Aug 16
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0f024cadff1959e9f00e12fd7a68418a0806b5f1 commit 0f024cadff1959e9f00e12fd7a68418a0806b5f1 Author: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Thu Aug 16 04:01:05 2018 Roll src/third_party/depot_tools 421bc3f84d1b..3d429cf5131a (1 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/421bc3f84d1b..3d429cf5131a git log 421bc3f84d1b..3d429cf5131a --date=short --no-merges --format='%ad %ae %s' 2018-08-16 vadimsh@chromium.org [cipd] Check CIPD client hash against pinned SHA256 during updates. Created with: gclient setdep -r src/third_party/depot_tools@3d429cf5131a The AutoRoll server is located here: https://depot-tools-chromium-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG= chromium:870166 ,chromium:874586 TBR=agable@chromium.org Change-Id: I0955bdd2622d27a2731bc6e690c5d003340b2a93 Reviewed-on: https://chromium-review.googlesource.com/1176982 Reviewed-by: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#583530} [modify] https://crrev.com/0f024cadff1959e9f00e12fd7a68418a0806b5f1/DEPS
,
Aug 16
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/95fb6dc810348da9dd261a4f7ac2d66c5743da86 commit 95fb6dc810348da9dd261a4f7ac2d66c5743da86 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Thu Aug 16 19:01:27 2018 [cipd] Check CIPD client hash during updates on Windows. This is Windows counterpart to https://chromium-review.googlesource.com/1171957 Also cleanup code style in cipd.ps1 script. It appears the common convention for global variables in PowerShell is $GlobalVar and for locals is $localVar. R=nodir@chromium.org, iannucci@chromium.org BUG= 870166 Change-Id: Ib8ffbad85497db31f7f5d44fdca6ed843e56f220 Reviewed-on: https://chromium-review.googlesource.com/1175248 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/95fb6dc810348da9dd261a4f7ac2d66c5743da86/cipd.ps1 [modify] https://crrev.com/95fb6dc810348da9dd261a4f7ac2d66c5743da86/cipd.bat
,
Aug 16
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b070be8f1fae659d4d6f1948a42c10baf7639d6c commit b070be8f1fae659d4d6f1948a42c10baf7639d6c Author: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Thu Aug 16 22:46:24 2018 Roll src/third_party/depot_tools efb38bb3d729..95fb6dc81034 (1 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/efb38bb3d729..95fb6dc81034 git log efb38bb3d729..95fb6dc81034 --date=short --no-merges --format='%ad %ae %s' 2018-08-16 vadimsh@chromium.org [cipd] Check CIPD client hash during updates on Windows. Created with: gclient setdep -r src/third_party/depot_tools@95fb6dc81034 The AutoRoll server is located here: https://depot-tools-chromium-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG= chromium:870166 TBR=agable@chromium.org Change-Id: Ie559e2b70ec74632248a8cefe5d7646d471fd1bf Reviewed-on: https://chromium-review.googlesource.com/1178542 Reviewed-by: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#583867} [modify] https://crrev.com/b070be8f1fae659d4d6f1948a42c10baf7639d6c/DEPS
,
Sep 14
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/2174136d2520b2f030addb8466373f13f5badd39 commit 2174136d2520b2f030addb8466373f13f5badd39 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Fri Sep 14 22:17:02 2018 [cipd] Add presubmit support for checking *.digests file. Not enabling it yet, since PRESUBMIT.py uses presubmit_canned_checks.py from depot_tools at HEAD, not from a CL (thus enabling this check in this CL will make it fail the presubmit). R=nodir@chromium.org BUG= 870166 Change-Id: I97802ec37f1f7513dfca3950f7f38a5c51ab0350 Reviewed-on: https://chromium-review.googlesource.com/1227432 Reviewed-by: Nodir Turakulov <nodir@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/2174136d2520b2f030addb8466373f13f5badd39/presubmit_canned_checks.py [modify] https://crrev.com/2174136d2520b2f030addb8466373f13f5badd39/tests/presubmit_unittest.py [modify] https://crrev.com/2174136d2520b2f030addb8466373f13f5badd39/tests/cipd_bootstrap_test.py
,
Sep 15
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/54ad986c611b372ea9e32d19ea11d3ae32e9fbec commit 54ad986c611b372ea9e32d19ea11d3ae32e9fbec Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Date: Sat Sep 15 02:37:04 2018 Roll src/third_party/depot_tools 53f9d8726afd..2174136d2520 (2 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/53f9d8726afd..2174136d2520 git log 53f9d8726afd..2174136d2520 --date=short --no-merges --format='%ad %ae %s' 2018-09-14 vadimsh@chromium.org [cipd] Add presubmit support for checking *.digests file. 2018-09-14 vadimsh@chromium.org [cipd] More careful error handling during CIPD bootstrap on Windows. Created with: gclient setdep -r src/third_party/depot_tools@2174136d2520 The AutoRoll server is located here: https://autoroll.skia.org/r/depot-tools-chromium-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG= chromium:870166 , chromium:853722 TBR=agable@chromium.org Change-Id: I07c6783b636aaf2b3f6ab2f2fe0bb2d120c2ea8d Reviewed-on: https://chromium-review.googlesource.com/1227433 Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#591567} [modify] https://crrev.com/54ad986c611b372ea9e32d19ea11d3ae32e9fbec/DEPS
,
Sep 17
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/9b19f31256adcd78f0f0a9952a7789ea9afa5643 commit 9b19f31256adcd78f0f0a9952a7789ea9afa5643 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Mon Sep 17 20:58:55 2018 [cipd] Handle stale *.digests file in self-update more gracefully. Before this CL self-update unconditionally trusted what's in *.digests file. This led to very confusing behavior when manually rolling the client version, but forgetting to update *.digests file: 1. Bootstrap a client. 2. Manually modify cipd_client_version (skipping 'cipd selfupdate-roll'). 3. Run the client self-update. Since the client binary hash still matches what's in *.digests, it assumes it is up-to-date. And it happily records its version (in .versions/.cipd_client.cipd_version) to match new version from cipd_client_version file. 4. The client is left in a semi-broken state: it claims it is new, while in fact it is old. To fix this, self-update now always double checks that the hash in the *.digests matches what is known to the backend. The mismatch indicates that either the *.digests file is stale (most likely), or the backend is lying (very unlikely). Due to caching of client refs in the tag cache, this causes 0 new RPCs on a happy path (when the client is up-to-date). On the unhappy path, DescribeClient RPC now happens slightly earlier. R=nodir@chromium.org, iannucci@chromium.org, tandrii@chromium.org BUG= 870166 Change-Id: I43c614a8e231bcdabed4d6012569b50231105944 Reviewed-on: https://chromium-review.googlesource.com/1226169 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/9b19f31256adcd78f0f0a9952a7789ea9afa5643/cipd/client/cipd/client.go [modify] https://crrev.com/9b19f31256adcd78f0f0a9952a7789ea9afa5643/cipd/client/cipd/client_test.go [modify] https://crrev.com/9b19f31256adcd78f0f0a9952a7789ea9afa5643/cipd/client/cipd/digests/client_digests_file.go [modify] https://crrev.com/9b19f31256adcd78f0f0a9952a7789ea9afa5643/cipd/client/cipd/digests/client_digests_file_test.go
,
Sep 17
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/ae858010cab65d348d4c6e5c3338c75d2a72cdb9 commit ae858010cab65d348d4c6e5c3338c75d2a72cdb9 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Mon Sep 17 23:53:51 2018
,
Sep 17
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/1ae855609dfb5bb1534897b5eacce869f8c63e1d commit 1ae855609dfb5bb1534897b5eacce869f8c63e1d Author: Vadim Shtayura <vadimsh@chromium.org> Date: Mon Sep 17 23:56:06 2018 [cipd] Bump CIPD client version to 2.2.8. This brings support for $ResolvedVersions and 'selfupdate-roll'. It also makes SHA256 the default hash algo. TBR=nodir@chromium.org, tandrii@chromium.org, iannucci@chromium.org, maruel@chromium.org BUG= 870166 , 821194 Change-Id: Iba8d8f9cc68290a7bb46bf265c6a443e109db58a Reviewed-on: https://chromium-review.googlesource.com/1229553 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/1ae855609dfb5bb1534897b5eacce869f8c63e1d/scripts/slave/cipd_bootstrap_v2.py
,
Sep 18
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/f9b4845975ffaf4b7bfe7b0b6ae0198b8a729bbd commit f9b4845975ffaf4b7bfe7b0b6ae0198b8a729bbd Author: Vadim Shtayura <vadimsh@chromium.org> Date: Tue Sep 18 00:22:26 2018 [cipd] Bump CIPD client version to 2.2.8, add presubmit check for *.digests. The new version uses SHA256 for uploads by default. It also fully supports $ResolvedVersions and 'selfupdate-roll'. The roll was generated by: $ ./cipd selfupdate-roll \ -version-file cipd_client_version \ -version git_revision:fb963f0f43e265a65fb7f1f202e17ea23e947063 BUG=821194, 870166 R=nodir@chromium.org, iannucci@chromium.org Change-Id: If7c23c43ff2c1bf50c9c50f4e9e17a23e84859b9 Reviewed-on: https://chromium-review.googlesource.com/1227431 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/f9b4845975ffaf4b7bfe7b0b6ae0198b8a729bbd/cipd_client_version [modify] https://crrev.com/f9b4845975ffaf4b7bfe7b0b6ae0198b8a729bbd/PRESUBMIT.py [modify] https://crrev.com/f9b4845975ffaf4b7bfe7b0b6ae0198b8a729bbd/cipd_client_version.digests
,
Sep 18
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c70b7d42ac883c61c8897e0ce8b8d42c8083d0a1 commit c70b7d42ac883c61c8897e0ce8b8d42c8083d0a1 Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Date: Tue Sep 18 03:12:42 2018 Roll src/third_party/depot_tools ea4301e5c5d2..f9b4845975ff (1 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/ea4301e5c5d2..f9b4845975ff git log ea4301e5c5d2..f9b4845975ff --date=short --no-merges --format='%ad %ae %s' 2018-09-18 vadimsh@chromium.org [cipd] Bump CIPD client version to 2.2.8, add presubmit check for *.digests. Created with: gclient setdep -r src/third_party/depot_tools@f9b4845975ff The AutoRoll server is located here: https://autoroll.skia.org/r/depot-tools-chromium-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG=chromium:821194, chromium:870166 TBR=agable@chromium.org Change-Id: I87d0b91a9894ec790a266a0ff346538b7a0dbc65 Reviewed-on: https://chromium-review.googlesource.com/1229503 Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#591928} [modify] https://crrev.com/c70b7d42ac883c61c8897e0ce8b8d42c8083d0a1/DEPS
,
Sep 18
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/01710d041f2cdbdbbae1c1e145eb1e23ff998e07 commit 01710d041f2cdbdbbae1c1e145eb1e23ff998e07 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Tue Sep 18 17:00:43 2018 [cipd] Pin hashes of CIPD packages. Together with already committed cipd_client_version.digests file, this cryptographically binds contents of CIPD packages used by depot_tools with depot_tool's git revision (assuming the CIPD client pinned by cipd_client_version.digests is trusted too, which can presumably be verified when it is being pinned). This holds true even if the CIPD backend is compromised. The worst that can happen is a denial of service (e.g. if the backend refuses to serve packages at all). If a bad backend tries to serve a malicious (unexpected) CIPD client, 'cipd' bootstrap script (and its powershell counterpart) will detect a mismatch between SHA256 of the fetched binary and what's specified in cipd_client_version.digests, and will refuse to run the untrusted binary. Similarly, if the bad backend tries to serve some other unexpected package (in place of a package specified in cipd_manifest.txt), the CIPD client (already verified and trusted as this point) will detect a mismatch between what was fetched and what's pinned in cipd_manifest.versions, and will refuse to install untrusted files. cipd_manifest.versions was generated from cipd_manifest.txt by: $ cipd ensure-file-resolve -ensure-file cipd_manifest.txt This will have to be rerun each time cipd_manifest.txt is updated. There's a presubmit check that verifies *.versions file is up-to-date (it's part of 'cipd ensure-file-verify'). BUG= 870166 R=nodir@chromium.org, iannucci@chromium.org, tandrii@chromium.org Change-Id: I25314adf0a9b05c69cd16e75aff01dbc79c87aa5 Reviewed-on: https://chromium-review.googlesource.com/1227435 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> [add] https://crrev.com/01710d041f2cdbdbbae1c1e145eb1e23ff998e07/cipd_manifest.versions [modify] https://crrev.com/01710d041f2cdbdbbae1c1e145eb1e23ff998e07/cipd_manifest.txt
,
Sep 18
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/37812790f2c5b3025e3cffc92c880ff3bb9a7f66 commit 37812790f2c5b3025e3cffc92c880ff3bb9a7f66 Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Date: Tue Sep 18 19:32:01 2018 Roll src/third_party/depot_tools 804797362e1d..1aa405fd859a (5 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/804797362e1d..1aa405fd859a git log 804797362e1d..1aa405fd859a --date=short --no-merges --format='%ad %ae %s' 2018-09-18 sergiyb@chromium.org Add an option to disable adding CC emails automatically 2018-09-18 nodir@google.com [gclient] Fix typo in get_gerrit_patch_root 2018-09-18 nodir@google.com [bot_update] Add gclient.get_gerrit_patch_root 2018-09-18 vadimsh@chromium.org [cipd] Pin hashes of CIPD packages. 2018-09-18 borenet@chromium.org Revert "gerrit_util: Support OAuth2 bearer tokens in CookieAuthenticator" Created with: gclient setdep -r src/third_party/depot_tools@1aa405fd859a The AutoRoll server is located here: https://autoroll.skia.org/r/depot-tools-chromium-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG= chromium:878303 , chromium:694348 , chromium:694348 , chromium:870166 TBR=agable@chromium.org Change-Id: I439b74e4283ec9962446dd2b14a101175f7fbb4d Reviewed-on: https://chromium-review.googlesource.com/1231496 Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#592145} [modify] https://crrev.com/37812790f2c5b3025e3cffc92c880ff3bb9a7f66/DEPS
,
Sep 18
This has been fully deployed to depot_tools: https://chromium.googlesource.com/chromium/tools/depot_tools/+/3d429cf5131a10f0e6ced89e8809820398ece78b https://chromium.googlesource.com/chromium/tools/depot_tools/+/95fb6dc810348da9dd261a4f7ac2d66c5743da86 https://chromium.googlesource.com/chromium/tools/depot_tools/+/01710d041f2cdbdbbae1c1e145eb1e23ff998e07 Other repos that bootstrap CIPD client should do something similar (I consider this outside the scope of this bug). |
||
►
Sign in to add a comment |
||
Comment 1 by vadimsh@chromium.org
, Aug 2Status: Assigned (was: Untriaged)