Deploy ts_mon and cloud trace golang libraries for lucifer |
||||||||||
Issue descriptionI was exploring building the golang tsmon library to be used from lucifer and any other golang binaries we may want to deploy in the lab. The library source is here: https://github.com/luci/luci-go/tree/master/common/tsmon (Note that the home for this is go.chromium.org/luci not github.com/luci/luci-go) We're currently building lucifer binaries by using the chromeos packaging system and building for host in the chroot: https://cs.corp.google.com/chromeos_public/src/third_party/chromiumos-overlay/chromeos-base/lucifer/lucifer-9999.ebuild I was thinking we'd add dev-go/luci/luci-0.0.1.ebuild to build / package the bits of luci that we may want, following: https://www.chromium.org/chromium-os/developer-guide/go-in-chromium-os#TOC-Recommendations-for-local-development-and-builds
,
Dec 16 2017
,
Dec 16 2017
Challenge #1: There is already a mirror of those repos at https://chromium.googlesource.com/external/github.com/luci/luci-go/ (we need to mirror all go repos we want to build) But this is 4 months stale. I suspect this was a mirror that chrome-infra used to maintain for their own purposes that they've since abandoned.
,
Dec 16 2017
Challenge #2:
Third-party dependencies that we'd have to add via other ebuilds:
pprabhu@pprabhu:src$ go list -f '{{ join .Deps "\n" }}' go.chromium.org/luci/common/tsmon | xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}' | grep -v go.chromium.org/luci
cloud.google.com/go/compute/metadata
--> [available] https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/master/dev-go/gcp
github.com/golang/protobuf/jsonpb
github.com/golang/protobuf/proto
github.com/golang/protobuf/ptypes/struct
--> [available] https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/master/dev-go/protobuf
github.com/mitchellh/go-homedir
--> [NEED TO ADD]
golang.org/x/net/context
golang.org/x/net/context/ctxhttp
--> [available] https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/master/dev-go/net
golang.org/x/oauth2
golang.org/x/oauth2/google
golang.org/x/oauth2/internal
golang.org/x/oauth2/jws
golang.org/x/oauth2/jwt
--> [available] https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/master/dev-go/oauth2
google.golang.org/api/googleapi
google.golang.org/api/googleapi/internal/uritemplates
[NEED TO ADD]
google.golang.org/grpc/credentials
--> [available] https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/master/dev-go/grpc
,
Dec 16 2017
+akeshet, independent of how I ended up here. It's only a matter of time before we *need* tsmon in golang world for lucifer.
,
Dec 16 2017
,
Dec 16 2017
There's always the nuclear option, vendor everything. Golang has tools around this now, and based on my previous research, go dep + vendoring is roughly the official answer to Go dependency management. It sounds bad, but it's actually not, now that I think about it. We're going to be shipping statically linked binaries at the end of the day, and static type checking makes upgrading vendored dependencies much less painful.
,
Dec 16 2017
I've used dep as well as virtualgo (not impressed by the latter so much), and I agree that dep works quite well. But looking at #4, there isn't much we need at the moment beyond what cros already has, so there's benefit in sticking with the cros "blessed" way to build binaries. For example, if you want to use dep for vendoring, we'd have to add support for dep to cros-go.eclass The biggest problem I see with that is cros likes to keep all upstream sources mirrored on chromium.googlesource.com, while dep will pull sources from upstream directly Or are you saying vendor everything the old way, by copying under vendor/ ?
,
Dec 18 2017
,
Jan 18 2018
,
Feb 16 2018
,
Feb 16 2018
I know that Allen has made some progress in building cloud trace for us. Alen can you summarize the current state of affairs here. That way, I can perhaps parallel process ts_mon while you do cloud trace. Inheriting priority from blocked bug.
,
Feb 16 2018
,
Feb 16 2018
Ebuilds for cloud trace are pending, need to work around a Go toolchain peculiarity https://bugs.chromium.org/p/chromium/issues/detail?id=811542
,
Mar 5 2018
Cloud trace libs should be available now, need to verify
,
Mar 30 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/86ebd86c5adf5dfe0e73672b936be9da443cb63e commit 86ebd86c5adf5dfe0e73672b936be9da443cb63e Author: Allen Li <ayatane@chromium.org> Date: Fri Mar 30 02:51:27 2018 luci-*: New packages go-homedir is a dependency for tsmon oauth2 needs to be upreved for RetrieveError which was added later. BUG= chromium:795473 TEST=None Change-Id: I702a8b7fc5f9ac3f2352acedb8452c85d9d5100d Reviewed-on: https://chromium-review.googlesource.com/985137 Commit-Ready: Allen Li <ayatane@chromium.org> Tested-by: Allen Li <ayatane@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> Reviewed-by: Rahul Chaudhry <rahulchaudhry@chromium.org> [add] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/luci-tsmon/luci-tsmon-0.0.1-r1.ebuild [add] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/luci-tsmon/luci-tsmon-0.0.1.ebuild [add] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/homedir/homedir-0.0.1.ebuild [add] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/luci-chromeinfra/luci-chromeinfra-0.0.1-r1.ebuild [add] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/luci-chromeinfra/luci-chromeinfra-0.0.1.ebuild [rename] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/oauth2/oauth2-0.0.1-r3.ebuild [modify] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/chromeos-base/lucifer/lucifer-9999.ebuild [modify] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/oauth2/oauth2-0.0.1.ebuild [add] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/luci-common/luci-common-0.0.1.ebuild [add] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/luci-auth/luci-auth-0.0.1-r1.ebuild [add] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/luci-common/luci-common-0.0.1-r1.ebuild [add] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/luci-auth/luci-auth-0.0.1.ebuild [add] https://crrev.com/86ebd86c5adf5dfe0e73672b936be9da443cb63e/dev-go/homedir/homedir-0.0.1-r1.ebuild
,
Apr 4 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/ca6cad32ad9bed7ef6212b016145060e85ff3aac commit ca6cad32ad9bed7ef6212b016145060e85ff3aac Author: Allen Li <ayatane@chromium.org> Date: Wed Apr 04 04:41:02 2018 luci-tsmon: Include metric package This package is needed for the actual metrics API. BUG= chromium:795473 TEST=None Change-Id: Ia81e984ffdd180fe65a638964d9826acd8a0dae3 Reviewed-on: https://chromium-review.googlesource.com/994381 Commit-Ready: Allen Li <ayatane@chromium.org> Tested-by: Allen Li <ayatane@chromium.org> Reviewed-by: Rahul Chaudhry <rahulchaudhry@chromium.org> [rename] https://crrev.com/ca6cad32ad9bed7ef6212b016145060e85ff3aac/dev-go/luci-tsmon/luci-tsmon-0.0.1-r2.ebuild [modify] https://crrev.com/ca6cad32ad9bed7ef6212b016145060e85ff3aac/dev-go/luci-tsmon/luci-tsmon-0.0.1.ebuild
,
Apr 12 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/infra/lucifer/+/b3754c10d2e4f2fde72b72dd7b05807598d09e5c commit b3754c10d2e4f2fde72b72dd7b05807598d09e5c Author: Allen Li <ayatane@google.com> Date: Thu Apr 12 23:29:08 2018 Add tsmon BUG= chromium:795473 TEST=None CQ-DEPEND=CL:994381 Change-Id: If74ce3511fa56be96e8c01d9c9f9dea97036236b Reviewed-on: https://chromium-review.googlesource.com/984717 Commit-Ready: Allen Li <ayatane@chromium.org> Tested-by: Allen Li <ayatane@chromium.org> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> [modify] https://crrev.com/b3754c10d2e4f2fde72b72dd7b05807598d09e5c/src/lucifer/Gopkg.lock [modify] https://crrev.com/b3754c10d2e4f2fde72b72dd7b05807598d09e5c/src/lucifer/cmd/lucifer_run_job/main.go [modify] https://crrev.com/b3754c10d2e4f2fde72b72dd7b05807598d09e5c/src/lucifer/Gopkg.toml
,
Apr 12 2018
|
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by pprabhu@chromium.org
, Dec 16 2017