New issue
Advanced search Search tips

Issue 795152 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocked on:
issue 859181
issue 795473



Sign in to add a comment

Replace dup-it with a simple service running on sentinel

Project Member Reported by pprabhu@chromium.org, Dec 15 2017

Issue description

dup-it copies over source tarballs from gentoo mirror to our own mirror.

It's quite simple:
- Only gsutil cp:// from one place to one place.
- Never overwrite existing files.

It consists of two parts:
An appengine app: https://dupit-hr.appspot.com/
This app was designed to support (changing) source / targets to mirror.

And a worker running on a GCE VM in the same project: https://pantheon.corp.google.com/compute/instancesDetail/zones/us-east4-b/instances/dupit-hr-worker-beefy?project=dupit-hr&organizationId=433637338589&graph=GCE_CPU&duration=P7D

This worker polls the appengine app for work and every day is instructed to copy over the files once.

We don't actually get good feedback about how many files were copied etc.

----

It'll be easier to simply make this a service on sentinel and have the usual metrics via ts_mon.
 
Cc: dgarr...@chromium.org
Don: since you got some context in this by chance, I'll send the CLs your way.
Blockedon: 795473
Considering using golang for this.
First, I need to pull in tsmon in golang.

Comment 3 by vapier@chromium.org, Jun 29 2018

Cc: jclinton@chromium.org
Components: -Infra>Client>ChromeOS Infra>Client>ChromeOS>Build Infra>Client>ChromeOS>CI
i'm assuming Prathmesh has been sucked up and away from this

Jason: where do you think this project falls CI/build wise ?
Hrm. It's a fuzzy line. I'd say leave it in both components and let's see who gets to it first. We can discuss formal owner in a few months when we have more headcount.
Blockedon: 859181
Owner: ----
Status: Available (was: Assigned)
Yeah, I don't have cycles for this.
I'll fix issue 859181 and dump the current state of the world on that bug.
Cc: pprabhu@chromium.org
pprabhu@: we def don't expect you to own this anymore given the restructuring that's happened of late.  but it would help immensely if you could brain dump all the diff pieces for how this service is managed and we can take it from there.
Jason: is this something the CI would own ?  it's currently python running on AE, but we don't care about the actual implementation.
Yes, CI will own this long-term. I just don't know when. Let's get that dump of the current state of things, though.
Components: -Infra>Client>ChromeOS>Build

Sign in to add a comment