New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 866912 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Jul 25
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug



Sign in to add a comment

Cannot gclient sync for src revisions between b8736d85 and b04e4978e (Apr 5, 2018 -> July 24 2018)

Project Member Reported by agrieve@chromium.org, Jul 24

Issue description

Yesterday, we ran:
https://cs.chromium.org/chromium/src/tools/android/roll/android_deps/fetch_all.py

But it didn't really work as expected :(.

It ended up causing many packages to have 2 "instances" with the same version tag:

$ cipd describe chromium/third_party/android_deps/repository/android_arch_core_common --version "version:1.0.0-cr0"
Error: ambiguity when resolving the tag, more than one instance has it.

We worked around the issue by creating a new version tag for all affected packages, and updating DEPS to point at them:
https://chromium-review.googlesource.com/c/chromium/src/+/1147496

However, it's still the case that if you sync you source tree to a point in time where the DEPS file points to -cr0, you will be unable to recreate the source tree at that revision :(.

Some investigation of cipd commands:

$ cipd instances chromium/third_party/android_deps/repository/android_arch_core_common
Instance ID                              │ Timestamp             │ Uploader                  │ Refs
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────
b9a07c5edaa4eee44e035a83280997e77e2594f5 │ Jul 23 14:46 EDT 2018 │ wnwen@google.com1479d64932c283bbf866b8d2eff475fa22e4b776 │ Feb 20 09:12 EST 2018 │ dgn@google.com2a49c129e6a9995f6b56033848645e7ad4e7cd3a │ Jan 08 09:23 EST 2018 │ dgn@google.com            │

$ cipd search chromium/third_party/android_deps/repository/android_arch_core_common -tag "version:1.0.0-cr0"
Instances:
  chromium/third_party/android_deps/repository/android_arch_core_common:b9a07c5edaa4eee44e035a83280997e77e2594f5
  chromium/third_party/android_deps/repository/android_arch_core_common:1479d64932c283bbf866b8d2eff475fa22e4b776

$ cipd search chromium/third_party/android_deps/repository/android_arch_core_common -tag "version:1.0.0-cr1"
Instances:
  chromium/third_party/android_deps/repository/android_arch_core_common:b9a07c5edaa4eee44e035a83280997e77e2594f5

# Used this to show that "common-1.0.0.jar" within them is the same
$ cipd pkg-fetch chromium/third_party/android_deps/repository/android_arch_core_common -version b9a07c5edaa4eee44e035a83280997e77e2594f5 -out cr0.zip
$ cipd pkg-fetch chromium/third_party/android_deps/repository/android_arch_core_common -version 1479d64932c283bbf866b8d2eff475fa22e4b776 -out cr1.zip 



Although this has now been worked-around, and the fix has been patched into many release branches (bug 866773)... We should really try to:
1) Fix people's ability to sync back in time between b8736d85 and b04e4978e, and
2) Try to prevent this from happening again


For 1), hopefully there's a command to do it? I see a set-ref and set-tag, but I'm not confident in my understanding between "instances" vs "refs" vs "tags" vs "version".

For 2), ideas:
 * Update CIPD to not allow this (I don't understand the underlying design enough to know if this makes sense)
 * Add to //docs/cipd.md a note that you should always search for existing tags before uploading a new one
   * Also add to fetch_all.py a call to "cipd search" to prevent it uploading a second instance.


Assigning to jbudorick to hopefully figure out part 1) here.
 
Summary: Cannot gclient sync for src revisions between b8736d85 and b04e4978e (Apr 5, 2018 -> July 24 2018) (was: Cannot gclient sync for src revisions between b8736d85 and b04e4978e (Aor 5, 2018 -> July 24 2018))
Another idea for 2) is that we should use instanceId rather than version tags in DEPS.
1) The CIPD server provides the ability to detach tags (https://codesearch.chromium.org/chromium/infra/go/src/go.chromium.org/luci/cipd/api/cipd/v1/repo.proto?rcl=e4e246622188306bd65d623be7f370abc052adef&l=280), but it doesn't look like the client uses it at the moment. Investigating more.

2) I think we should start w/ updating fetch-all.
Labels: -Restrict-View-Google
Un-RVGing; we can fork a separate bug if there are internal-specific details.
This is a known pitfall :( Re preventing it: Issue 866949.
Re 1): Cleared a bunch of tags thanks to vadim's help w/ issuing RPCs to the CIPD server. There should no longer be duplicate tags for anything that was updated in https://chromium-review.googlesource.com/c/chromium/src/+/1147496.
Thanks John and Vadim! I will fix fetch_all to check for existing tags. Happy to help with preventing this in the future.
Labels: OS-Android
Status: Fixed (was: Assigned)
Closing this out, as (1) is fixed and (2) is now tracked in https://bugs.chromium.org/p/chromium/issues/detail?id=867454.

Sign in to add a comment