New issue
Advanced search Search tips

Issue 659291 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Nov 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 651987



Sign in to add a comment

Support runtime generation of logdog stream URLs in the logdog client library

Project Member Reported by jbudorick@chromium.org, Oct 25 2016

Issue description

^
 
Project Member

Comment 1 by bugdroid1@chromium.org, Oct 27 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/ca2642967ad3c8b5b16068dc03e126c81231a341

commit ca2642967ad3c8b5b16068dc03e126c81231a341
Author: dnj <dnj@chromium.org>
Date: Thu Oct 27 16:11:03 2016

Butler: export Coordinator host in environment.

Export the configured Coordinator host as an environment variable, if
present. Update the bootstrap client to pull this value out of the
environment as well.

BUG= chromium:659291 
TEST=local
  - Ran Butler locally w/ debug, observed host in environment.

Review-Url: https://codereview.chromium.org/2456673003

[modify] https://crrev.com/ca2642967ad3c8b5b16068dc03e126c81231a341/logdog/client/butler/bootstrap/env.go
[modify] https://crrev.com/ca2642967ad3c8b5b16068dc03e126c81231a341/logdog/client/butlerlib/bootstrap/bootstrap.go
[modify] https://crrev.com/ca2642967ad3c8b5b16068dc03e126c81231a341/logdog/client/butlerlib/bootstrap/bootstrap_test.go
[modify] https://crrev.com/ca2642967ad3c8b5b16068dc03e126c81231a341/logdog/client/butlerlib/bootstrap/environment.go
[modify] https://crrev.com/ca2642967ad3c8b5b16068dc03e126c81231a341/logdog/client/cmd/logdog_butler/main.go
[modify] https://crrev.com/ca2642967ad3c8b5b16068dc03e126c81231a341/logdog/client/cmd/logdog_butler/output_logdog.go
[modify] https://crrev.com/ca2642967ad3c8b5b16068dc03e126c81231a341/logdog/client/cmd/logdog_butler/subcommand_run.go
[modify] https://crrev.com/ca2642967ad3c8b5b16068dc03e126c81231a341/logdog/client/cmd/logdog_butler/subcommand_serve.go
[modify] https://crrev.com/ca2642967ad3c8b5b16068dc03e126c81231a341/logdog/client/cmd/logdog_butler/subcommand_stream.go

Project Member

Comment 2 by bugdroid1@chromium.org, Oct 28 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/16e6635d9d47ce93ee0bc0cd952e53abbd95a686

commit 16e6635d9d47ce93ee0bc0cd952e53abbd95a686
Author: dnj <dnj@chromium.org>
Date: Fri Oct 28 07:04:20 2016

LogDog: Update client/bootstrap to generate URLs.

Add the ability for a populated Bootstrap to generate viewer URLs.

Update streamclient.Stream to be able to emit its stream properties. This is
useful in general, notably by the Bootstrap's URL generation functions.

This change involves modifying streamproto.Properties to remove the false
assumption that they are safe to copy/modify. To this end, places where
this assumption was made now properly Clone() the Properties to take
possession of them, and Properties are now passed around as a pointer to
make this clear.

BUG= chromium:659291 
TEST=unit

Review-Url: https://codereview.chromium.org/2456953003

[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butler/bundler/bundler.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butler/butler.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butler/butler_test.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butler/streamserver/handshake_test.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/bootstrap/bootstrap.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/bootstrap/bootstrap_test.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/streamclient/client.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/streamclient/client_namedPipe_posix.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/streamclient/client_namedPipe_windows.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/streamclient/client_test.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/streamclient/local.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/streamclient/registry.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/streamclient/stream.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/streamclient/stream_test.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/streamproto/properties.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/butlerlib/streamproto/properties_test.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/cmd/logdog_annotee/filesystemClient.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/client/cmd/logdog_butler/stream.go
[add] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/common/viewer/url.go
[add] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/logdog/common/viewer/url_test.go
[modify] https://crrev.com/16e6635d9d47ce93ee0bc0cd952e53abbd95a686/milo/appengine/swarming/memoryClient.go

Project Member

Comment 3 by bugdroid1@chromium.org, Nov 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-py.git/+/a32ab2fad3bda41efae1607ec66be78f48a9d0b9

commit a32ab2fad3bda41efae1607ec66be78f48a9d0b9
Author: dnj <dnj@chromium.org>
Date: Wed Nov 02 00:26:39 2016

Update LogDog client library to generate URLs.

Update the LogDog client library to generate LogDog stream viewer URLs.
This includes:
- Capturing the LogDog Coordinator host from the enviornment via
  bootstrap.
- Implementing full stream paths and URL generation.
- Adjusting the StreamClient implementations to retain the prefix and
  coordinator host values.
- Wrapping text/binary StreamClient file descriptors in a container to
  expose per-stream higher-level functionality.

Tests have also been updated to assert the new functionality.

TBR=nodir@chromium.org
BUG= chromium:659291 
TEST=unit

Review-Url: https://codereview.chromium.org/2453273002

[modify] https://crrev.com/a32ab2fad3bda41efae1607ec66be78f48a9d0b9/client/libs/logdog/bootstrap.py
[modify] https://crrev.com/a32ab2fad3bda41efae1607ec66be78f48a9d0b9/client/libs/logdog/stream.py
[modify] https://crrev.com/a32ab2fad3bda41efae1607ec66be78f48a9d0b9/client/libs/logdog/streamname.py
[modify] https://crrev.com/a32ab2fad3bda41efae1607ec66be78f48a9d0b9/client/libs/logdog/tests/bootstrap_test.py
[modify] https://crrev.com/a32ab2fad3bda41efae1607ec66be78f48a9d0b9/client/libs/logdog/tests/stream_test.py
[modify] https://crrev.com/a32ab2fad3bda41efae1607ec66be78f48a9d0b9/client/libs/logdog/tests/streamname_test.py

Project Member

Comment 4 by bugdroid1@chromium.org, Nov 10 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bc0614e467b9538c2930b93a5c058c8b66c93cac

commit bc0614e467b9538c2930b93a5c058c8b66c93cac
Author: jbudorick <jbudorick@chromium.org>
Date: Thu Nov 10 21:42:30 2016

Roll src/tools/swarming_client/ 380e32662..ebc8dab6f (25 commits).

https://chromium.googlesource.com/external/swarming.client.git/+log/380e32662312..ebc8dab6f8b8

$ git log 380e32662..ebc8dab6f --date=short --no-merges --format='%ad %ae %s'
2016-11-01 dnj Update LogDog client library to generate URLs.
2016-11-01 bpastene Roll infra_libs to 564aaf7480f24c90687df79d9cef910cc342a54d
2016-10-31 aludwin Don't recreate a directory if it already exists
2016-10-31 vadimsh Fix how --auth-disabled works, in particular on headless bots.
2016-10-28 iannucci Fix secret-bytes-path flag to use dashes instead of underscores.
2016-10-28 iannucci Add secret_bytes functionality to swarming.
2016-10-28 aludwin Add --output option to Swarming client and pass it through to the bot
2016-10-27 aludwin Return specific files, not just those in $(ISOLATED_OUTDIR)
2016-10-27 maruel Update stale links to code.google.com.
2016-10-27 iannucci Fix auth_server_test.py
2016-10-26 nodir swarming: implement named caches
2016-10-26 nodir run_isolated: fix --leak-temp-dir
2016-10-26 aludwin Pass args in file from task_runner to run_isolated
2016-10-20 iannucci Alter bot code to use new LUCI_CONTEXT library.
2016-10-19 nodir run_isolated: rename cache to isolate_cache
2016-10-19 iannucci Add luci_context library.
2016-10-18 nodir LRUDict: add timestamps
2016-10-13 leilei Fix encode issue.
2016-10-12 nodir run_isolated: rename cache to isolate_cache
2016-10-12 smut Don't assume existence of dimensions when querying for Swarming bots
2016-09-14 ryanmartens Adds LUCI context auth service access token.
2016-09-13 vadimsh Add --service-account option to 'trigger' and 'run'.
2016-09-12 ryanmartens Adds auth option `luci_context_json`.
2016-09-09 vadimsh Implement local HTTP server that serves task OAuth access tokens.
2016-09-07 ryanmartens Refactors authentictors into separate file.

BUG= 659291 
R=maruel@chromium.org,dnj@chromium.org

Review-Url: https://codereview.chromium.org/2482633002
Cr-Commit-Position: refs/heads/master@{#431365}

[modify] https://crrev.com/bc0614e467b9538c2930b93a5c058c8b66c93cac/DEPS
[modify] https://crrev.com/bc0614e467b9538c2930b93a5c058c8b66c93cac/build/secondary/tools/swarming_client/isolate.pydeps

Comment 5 by d...@chromium.org, Nov 30 2016

Status: Fixed (was: Untriaged)

Sign in to add a comment