Switch all masters to LogDog-only mode. |
||
Issue descriptionThis is a top-level bug for switching all BuildBot master logging over to LogDog-only mode. The advantages of this are a unified logging system and significant relief to the I/O and CPU of each master process. Current state: - LogDog-only mode is mature. It has been successfully running on several masters for months without major issue. - The public waterfall template UI and Milo UI was recently updated to make LogDog-only mode not show junk links: https://screenshot.googleplex.com/eZXKEfdnXZu.png - Several masters are already switched: https://chromium.googlesource.com/chromium/tools/build/+/b2df1caa8ced6d94b8c91decfe53bc000c48fa7b/scripts/slave/logdog-params.pyl The remainder of the switch will be a combination of opportunity and need, based on master load and suitability.
,
Mar 8 2017
One thing I really miss with LogDog is a quick way to download logs from a URL using my favorite command like HTTP tool (IE wget/curl). With LogDog only mode getting https://bugs.chromium.org/p/chromium/issues/detail?id=698429 fixed seems like it might be more important?
,
Mar 8 2017
Higher up in that bug are the commands to get the logs, i.e.: mkdir $LOGDOG_CIPD_ROOT cd $LOGDOG_CIPD_ROOT /path/to/depot_tools/cipd init /path/to/depot_tools/cipd install infra/tools/luci/logdog/logdog/linux-amd64 At this point, you will have the "logdog" binary available, and can run it: ./logdog cat <path> <path> can be one of two things: 1) A <project>/<path> tuple, like the one you would see the viewer: https://screenshot.googleplex.com/wfmB4a05Dip e.g.: ./logdog cat chromium/bb/chromium.fyi/CrWin7Goma_dbg_/32153/+/recipes/steps/taskkill/0/stdout 2) A LogDog URL, like the one you would see in a build property: https://screenshot.googleplex.com/pQL9CJGRLvD ./logdog cat logdog://luci-logdog.appspot.com/chromium/bb/chromium.fyi/CrWin7Goma_dbg_/32153/+/recipes/annotations We'll make this less manual but no promises (yet) to have straight HTTP programmatic access.
,
Mar 8 2017
Following those instructions is a real step backwards -- it's gone from using an existing tools everyone is familiar with (curl <url>), to having to learn multiple steps and about things like cipd, logdog CLI, project/path tuples and other random stuff.
,
Mar 8 2017
Yup, we do know we want to make something better here.
,
Mar 8 2017
that said, I don't know how many people really use curl to fetch things vs. "save as"
,
Mar 8 2017
The plan is to have "logdog" CLI install into "depot_tools", so every developer will have it on their path. At that point, it's a one-liner. We are still working on a harness to deploy and maintain binaries in "depot_tools", though, so this isn't in place yet.
,
Mar 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/217e731a76cd32ede129a95c93a6002bd88935fc commit 217e731a76cd32ede129a95c93a6002bd88935fc Author: Ryan Tseng <hinoka@google.com> Date: Wed Mar 08 02:39:38 2017 Add logdog to infra's cipd deployment So that "cit logdog" works. BUG= 698968 Change-Id: If20d3b22816d4b5ce500f2248a0e21308f938bf4 Reviewed-on: https://chromium-review.googlesource.com/450969 Commit-Queue: Ryan Tseng <hinoka@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/217e731a76cd32ede129a95c93a6002bd88935fc/bootstrap/install_cipd_packages.py [add] https://crrev.com/217e731a76cd32ede129a95c93a6002bd88935fc/bootstrap/cipd/cipd_windows_amd64.txt [modify] https://crrev.com/217e731a76cd32ede129a95c93a6002bd88935fc/bootstrap/cipd/cipd_mac_amd64.txt [modify] https://crrev.com/217e731a76cd32ede129a95c93a6002bd88935fc/bootstrap/cipd/cipd_linux_amd64.txt
,
Mar 8 2017
Alright, you can try running this instead: cit logdog cat logdog://luci-logdog.appspot.com/chromium/bb/chromium.fyi/CrWin7Goma_dbg_/32153/+/recipes/annotations It'll spend a couple minutes doing basically a full gclient sync, but after that should be fast.
,
Mar 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/c1c68b5a49e8a9ddc42193acb95d9d86fca469c3 commit c1c68b5a49e8a9ddc42193acb95d9d86fca469c3 Author: Dirk Pranke <dpranke@chromium.org> Date: Thu Mar 09 18:55:08 2017 Make the remaining Chromium builders LogDog-only. This CL switches the builders on the following masters to LogDog-only (i.e., the buildbot logs on the master will now just be stubs): - chromium.android - chromium.android.fyi - chromium.chrome - chromium.chromedriver - chromium.chromiumos - chromium.goma - chromium.lkgr - chromium.swarm - tryserver.blink - tryserver.chromium.android - tryserver.chromium.angle - tryserver.chromium.mac - tryserver.chromium.win After this CL lands, all of the public Chromium builders will be LogDog-only. R=dnj@chromium.org BUG= 698968 Change-Id: I8137e843df7927d89f69f378835d78bda97f1e49 Reviewed-on: https://chromium-review.googlesource.com/452677 Commit-Queue: Dirk Pranke <dpranke@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/c1c68b5a49e8a9ddc42193acb95d9d86fca469c3/scripts/slave/logdog-params.pyl
,
Mar 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/c1c68b5a49e8a9ddc42193acb95d9d86fca469c3 commit c1c68b5a49e8a9ddc42193acb95d9d86fca469c3 Author: Dirk Pranke <dpranke@chromium.org> Date: Thu Mar 09 18:55:08 2017 Make the remaining Chromium builders LogDog-only. This CL switches the builders on the following masters to LogDog-only (i.e., the buildbot logs on the master will now just be stubs): - chromium.android - chromium.android.fyi - chromium.chrome - chromium.chromedriver - chromium.chromiumos - chromium.goma - chromium.lkgr - chromium.swarm - tryserver.blink - tryserver.chromium.android - tryserver.chromium.angle - tryserver.chromium.mac - tryserver.chromium.win After this CL lands, all of the public Chromium builders will be LogDog-only. R=dnj@chromium.org BUG= 698968 Change-Id: I8137e843df7927d89f69f378835d78bda97f1e49 Reviewed-on: https://chromium-review.googlesource.com/452677 Commit-Queue: Dirk Pranke <dpranke@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/c1c68b5a49e8a9ddc42193acb95d9d86fca469c3/scripts/slave/logdog-params.pyl
,
Sep 26
pretty sure this is done |
||
►
Sign in to add a comment |
||
Comment 1 by estaab@chromium.org
, Mar 7 2017Status: Assigned (was: Untriaged)