New issue
Advanced search Search tips

Issue 628321 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

Embed LogDog in Kitchen and get it running on Swarming.

Project Member Reported by d...@chromium.org, Jul 14 2016

Issue description

Embed LogDog into the Kitchen tool and get it running on Swarming.

We will use the prefix space:
swarm/<swarming-host>/<swarming-task-id>
 
Project Member

Comment 1 by bugdroid1@chromium.org, Jul 15 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/922db6e6030fd6e5de50ca36da3570840800e3f5

commit 922db6e6030fd6e5de50ca36da3570840800e3f5
Author: dnj <dnj@chromium.org>
Date: Fri Jul 15 17:29:32 2016

Kitchen: Generate LogDog prefix from Swarming.

Also bump luci-go:
14b7d6b Milo: Frontpage overall view for buildbot
9e2fdc2 logdog: make MakeStreamName more versatile.
fcc9be2 milo: buildbucket builer view
9fcbaf0 Bypass cloud endpoints proxy server on buildbucket cmd line client.
07de3c4 luci-cron: Add 'Abort' button to the job page.
72d9f14 luci-cron: Keep Bootstrap and jQuery closer to the cron source code.

BUG= chromium:628321 
TEST=None

Review-Url: https://codereview.chromium.org/2146403002

[modify] https://crrev.com/922db6e6030fd6e5de50ca36da3570840800e3f5/DEPS
[modify] https://crrev.com/922db6e6030fd6e5de50ca36da3570840800e3f5/go/src/infra/tools/kitchen/cook.go
[modify] https://crrev.com/922db6e6030fd6e5de50ca36da3570840800e3f5/go/src/infra/tools/kitchen/cook_logdog.go
[add] https://crrev.com/922db6e6030fd6e5de50ca36da3570840800e3f5/go/src/infra/tools/kitchen/cook_logdog_test.go

Project Member

Comment 2 by bugdroid1@chromium.org, Jul 18 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/51da592b407fc30601445f532855205309403910

commit 51da592b407fc30601445f532855205309403910
Author: dnj <dnj@chromium.org>
Date: Mon Jul 18 10:03:32 2016

Add infraenv, setup creds and tee for Kitchen.

Add "infra/libs/infraenv", a package that exposes Infra-specific
envrionment functionality. Currently it exports the LogDog service
account credential path.

Leverage "infraenv" in Kitchen to load the LogDog service account
credential for non-GCE instance systems; GCE systems will use their
native metadata token to authenticate to LogDog.

Enable STDOUT/STDERR teeing in Kitchen/LogDog integration so we can run
it in a hybrid mode, compatible both with Milo/Swarming and Milo/LogDog.

BUG= chromium:628321 
TEST=None

Review-Url: https://codereview.chromium.org/2154953002

[add] https://crrev.com/51da592b407fc30601445f532855205309403910/go/src/infra/libs/infraenv/creds.go
[add] https://crrev.com/51da592b407fc30601445f532855205309403910/go/src/infra/libs/infraenv/doc.go
[add] https://crrev.com/51da592b407fc30601445f532855205309403910/go/src/infra/libs/infraenv/infraenv.infra_testing
[add] https://crrev.com/51da592b407fc30601445f532855205309403910/go/src/infra/libs/infraenv/paths_posix.go
[add] https://crrev.com/51da592b407fc30601445f532855205309403910/go/src/infra/libs/infraenv/paths_windows.go
[modify] https://crrev.com/51da592b407fc30601445f532855205309403910/go/src/infra/tools/kitchen/cook_logdog.go
[modify] https://crrev.com/51da592b407fc30601445f532855205309403910/go/src/infra/tools/kitchen/cook_logdog_test.go

Project Member

Comment 3 by bugdroid1@chromium.org, Jul 18 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config.git/+/6c0cdf0df9e44c22cfde34a828a5afe2a95360ab

commit 6c0cdf0df9e44c22cfde34a828a5afe2a95360ab
Author: Dan Jacques <dnj@chromium.org>
Date: Mon Jul 18 16:52:37 2016

Project Member

Comment 4 by bugdroid1@chromium.org, Jul 18 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config.git/+/f6821bc3d91a20ebbb1357b805e6763b6182176c

commit f6821bc3d91a20ebbb1357b805e6763b6182176c
Author: Dan Jacques <dnj@chromium.org>
Date: Mon Jul 18 17:02:17 2016

Project Member

Comment 6 by bugdroid1@chromium.org, Jul 18 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config.git/+/978975841bf192c341a8f06b43d4b5ae13605422

commit 978975841bf192c341a8f06b43d4b5ae13605422
Author: Nodir Turakulov <nodir@google.com>
Date: Mon Jul 18 19:26:23 2016

Project Member

Comment 7 by bugdroid1@chromium.org, Jul 18 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config.git/+/6af0366974f5921513cee94e22e920978b45c05e

commit 6af0366974f5921513cee94e22e920978b45c05e
Author: Nodir Turakulov <nodir@google.com>
Date: Mon Jul 18 19:33:19 2016

Project Member

Comment 8 by bugdroid1@chromium.org, Jul 18 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/ac277ae13b4b040d504162c4dc6f6bd08de5149e

commit ac277ae13b4b040d504162c4dc6f6bd08de5149e
Author: dnj <dnj@chromium.org>
Date: Mon Jul 18 19:45:04 2016

LogDog: Actually use default max buffer age.

The documentation for the MaxBufferAge parameter suggests that if it's
zero, a default value will be used. However, this was not actually
implemented. Historically this has not been a problem, since the only
user always supplied a non-zero value, but now that Butler is being
used elsewhere as a library, let's fix this.

TBR=hinoka@chromium.org
BUG= chromium:628321 
TEST=None

Review-Url: https://codereview.chromium.org/2153383003

[modify] https://crrev.com/ac277ae13b4b040d504162c4dc6f6bd08de5149e/client/logdog/butler/butler.go

Project Member

Comment 9 by bugdroid1@chromium.org, Jul 18 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/741f18ada910fae7f78ac4ed52e17c828dd36fa3

commit 741f18ada910fae7f78ac4ed52e17c828dd36fa3
Author: dnj <dnj@chromium.org>
Date: Mon Jul 18 19:56:04 2016

LogDog/Kitchen: Misc fixes.

- Supply the proper Prefix parameter to the Butler instance.
- Will no longer strip annotations when teeing.
- Allow a user to manually supply the JSON service account file on
  command-line (useful for debugging).

BUG= chromium:628321 
TEST=local
  - Ran locally w/ new flag, observed it correctly sending parameters
    and streaming!

R=nodir@chromium.org

Review-Url: https://codereview.chromium.org/2157083002

[modify] https://crrev.com/741f18ada910fae7f78ac4ed52e17c828dd36fa3/go/src/infra/tools/kitchen/cook_logdog.go

Comment 10 by no...@chromium.org, Jul 20 2016

Labels: M-54 luci
make this bug included in go/luci-current-blockers
Project Member

Comment 11 by bugdroid1@chromium.org, Jul 21 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config.git/+/e0b1317ca2ecc4fd08c388fdebc13a0cd4f012ad

commit e0b1317ca2ecc4fd08c388fdebc13a0cd4f012ad
Author: Dan Jacques <dnj@chromium.org>
Date: Thu Jul 21 17:36:31 2016

Project Member

Comment 12 by bugdroid1@chromium.org, Jul 21 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config.git/+/e0b1317ca2ecc4fd08c388fdebc13a0cd4f012ad

commit e0b1317ca2ecc4fd08c388fdebc13a0cd4f012ad
Author: Dan Jacques <dnj@chromium.org>
Date: Thu Jul 21 17:36:31 2016

Dan/Nodir: I'm having this problem with BoringSSL CQ build: https://chromium-swarm.appspot.com/user/task/303a22b2ea746710 . Does this look like a reasonable fix https://boringssl-review.googlesource.com/#/c/8910/? 
Well, that CL had certainly fixed my problem, but please LMK if I should update it somehow.

Comment 15 by d...@chromium.org, Jul 25 2016

Wow, awesome, I'm really happy that it was that easy to add a LogDog project!

Please understand that the ACLs that you have added there are public-read. If this is appropriate for BoringSSL, then everything LGTM!
Yep, I get that they are public, so is BoringSSl. I was a bit confused by the use of "-dev" prefix in CL.

Comment 17 by d...@chromium.org, Jul 25 2016

Oh I didn't see that, you should actually remove that and change to "luci-logdog-chromium-writers" for prod.
OK, I'll do that, but then why was -dev used in infra/infra? :)

Comment 19 by d...@chromium.org, Jul 25 2016

ummm seems wrong too, I think Nodir maybe just copy/paste when he installed that, will fix.

Comment 21 by d...@chromium.org, Sep 1 2016

Status: Fixed (was: Started)
I believe this is done.

Sign in to add a comment