New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 787546 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Dec 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Task

Blocking:
issue 776430



Sign in to add a comment

Make infra_libs available on vpython

Project Member Reported by katthomas@chromium.org, Nov 21 2017

Issue description

Steps:
1. Teach dockerbuild how to build a wheel for infra_libs (infra.git:packages/infra_libs) 
2. Build a wheel for infra_libs with dockerbuild
3. Turn that wheel into a cipd package
4. Automate this process by adding steps 2 and 3 to the infra.continuous builder
5. Write a presubmit check that will not allow you to submit a change to packages/infra_libs without bumping the version number.

Does this sound correct?
 

Comment 1 by d...@chromium.org, Nov 21 2017

I'd add:

0. Add a 'setup.py' to infra_libs so that it can make itself into a wheel.

"dockerbuild"'s wheel building is really just a wrapper around the standard Python setuptools suite, basically just running something like: "python setup.py bdist_wheel". 

There is a lot of information about how to configure a library to build wheels: https://docs.python.org/2/distributing/index.html

Once you get that working, (1) through (3) should be cake. (4) is something that is useful in general, since *none* of dockerbuild wheels are automated through CI ATM, but may be outside of the scope of this bug.
infra_libs already has a setup.py: https://chromium.googlesource.com/infra/infra/+/master/packages/infra_libs/setup.py (though I'm not sure if it's sufficient)
`python setup.py bdist_wheel` *seems* to work in infra_libs, so that's convenient!
indeed!
Cc: -jparent@chromium.org yyanagisawa@chromium.org
1-3 are done, so infra_libs is available on cipd now, meaning BigQueryHelper is available. 

I don't think 5 is necessary now because we're using infra rev instead of version number. 

Is 4 actually necessary?

Comment 7 by no...@chromium.org, Dec 12 2017

> 5. Write a presubmit check that will not allow you to submit a change to packages/infra_libs without bumping the version number.

yeah, i don't quite understand this, but maybe I am missing something.

(4) would be consistent with the rest of software we deploy via CIPD. Normally ones lands a CL to change a program and some time CIPD package with the change are available automatically.
Currently looking into the best way to accomplish (4). Currently, the infra_continuous recipe uses the build.py script in infra/build with the packages definitions in infra/build/packages. As far as I can tell, we don't have continuous building of cipd packages with dockerbuild yet. If anyone has any ideas, feel free to chime in. 
Status: Fixed (was: Assigned)
Given that 4 involves running dockerbuild on a builder, perhaps we're ok with building this package manually for now? This is the norm for cipd packages built with dockerbuild. 

Given that, I'm going to mark this is fixed! Feel free to re-open if there are concerns. 

Comment 10 by d...@chromium.org, Dec 18 2017

> Given that 4 involves running dockerbuild on a builder

FWIW, this is something that should be extremely safe. It's really easy to add Docker to builders:
https://chrome-internal.googlesource.com/infra/puppet/+/be919455d01617ac7653d92bae657a45a0c0a0a6/puppetm/opt/puppet/conf/nodes.yaml#864

Once added, Docker runs hermetically, so a recipe that runs "dockerbuild" on a builder should just work.

OTOH there is probably a bug or two to fix just b/c of the expansive nature of the script. YMMV.

Sign in to add a comment