Long LogDog logs are slow. |
||||||
Issue descriptionWe are getting reports that logdog log streaming speeds for completed builds are unacceptably slow for really long build logs. A concrete example is here: https://luci-logdog.appspot.com/v/?s=chromeos/buildbucket/cr-buildbucket.appspot.com/8955839056303699808/+/steps/VMTest__attempt_2_/0/stdout
,
Feb 6 2018
,
Feb 6 2018
Downloading is easy: $ logdog cat chromeos/bb/chromeos/../+/....
,
Feb 6 2018
Where does the logdog command come from? It doesn't seem to be depot_tools.
,
Feb 6 2018
Do you have documentation for this? Google finds other products. ihf@ql:~$ logdog logdog: command not found
,
Feb 6 2018
Looks like it's in "infra/infra": https://chromium.googlesource.com/infra/infra/+/master/bootstrap/cipd/cipd_linux_amd64.txt#23 Probably should be part of "depot_tools"; TBH I thought it was.
,
Feb 6 2018
Can we make this process simpler for the average developer?
,
Feb 6 2018
(Suggestion in #6 would be making it simpler for the average developer)
,
Feb 6 2018
Agreed. Further alternatives... * Add a download link to the logdog header bar. * Add an additional URL argument that means "display as simple text" ie: "?format=text" Just thinking. Who owns these updates now?
,
Feb 6 2018
Both of those are great ideas. Would you mind filing bugs for each and assigning LogDog component? LUCI team still owns LogDog :)
,
Feb 6 2018
,
Feb 6 2018
Filed. Thanks!
,
Feb 21 2018
FYI, "cit logdog" works with just depot_tools in PATH. We'll try to get logdog improvements added to the schedule for next quarter.
,
Mar 6 2018
One of our users would like Milo to present a link that downloads the log as a plain file, as mentioned in the above comment. They also think that it is the logdog streaming api that makes it slow, so that when the log has been finished and saved to a single cloud storage object, they want the link to download directly from that location instead, when getting the raw data. I don't see a separate issue as having been filed with this request yet. This is the main Milo issue that affects our users' productivity, so I think its priority should be higher than 3.
,
Apr 5 2018
I agree that this should be higher priority than 3. Bumping it to 2.
,
Jul 16
i believe this was fixed by issue 698429 thanks to hinoka specifically, https://luci-logdog.appspot.com/v/?s=chromeos/buildbucket/cr-buildbucket.appspot.com/8955839056303699808/+/steps/VMTest__attempt_2_/0/stdout# load data <1.5s and Chrome renders it <5s for me. please reopen if it is still slow
,
Jul 16
wrong link: https://luci-logdog.appspot.com/logs/chromeos/buildbucket/cr-buildbucket.appspot.com/8955839056303699808/+/steps/VMTest__attempt_2_/0/stdout and here is the new viewer for the "long log" in #2: https://logs.chromium.org/logs/chromeos/bb/chromeos/betty-vmtest-informational/152/+/recipes/steps/VMTest/0/stdout now the bottleneck is Chrome. If authenticated, you can use curl to load that log, though. Sometimes like # luci-auth is in depot_tools luci-auth login # do the login curl -H "Authorization: Bearer `luci-auth token`" 'https://logs.chromium.org/logs/chromeos/bb/chromeos/betty-vmtest-informational/152/+/recipes/steps/VMTest/0/stdout' > mylog |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by ihf@chromium.org
, Feb 6 2018