New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 889947 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Oct 16
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 3
Type: Bug-Regression



Sign in to add a comment

Regression: compile output now has timestamps

Project Member Reported by thakis@chromium.org, Sep 27

Issue description

This makes the output much larger, and makes the build output more noisy and way slower to download, and is 100% redundant with the ninja trace that goma uploads and which has an about:tracing style output.

(See also lengthy discussion in https://codereview.chromium.org/483853002/ and https://bugs.chromium.org/p/chromium/issues/detail?id=727451#c14 onward.
 
We can add a switch/checkbox (saved via cookie) to turn off the timestamp gutter.  wdyt?
It should default off for compile output, and probably for test output. That output is already large and takes a long time to download.
Components: Infra>Platform>LogDog
So if I understand correctly, there are two concerns:

1. Clutter
2. Speed

Visiting speed:

Compare no timestamps:
https://3501-b4019a5-dot-luci-logdog.appspot.com/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8934275041283648992/+/steps/compile__with_patch_/0/stdout

With timestamps:
https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8934275041283648992/+/steps/compile__with_patch_/0/stdout

No timestamps:   108KB, load time ~700ms
With timestamps: 236KB, load time 300ms - 1200ms (varies widely).
Most of the time is spent in transport.  I believe the bottleneck is most likely fetching entries out of bigtable, which is the same regardless of whether or not the timestamps are there.
In this case, I think any slowness attributed to timestamps is mostly eaten by variance in bigtable (or whatever backend0 loading latency.
We might see a more pronounced effect on a larger log stream, though.

So if we have a meaningful switch or option to turn off timestamps for speed reason, it can't be on the client side, it would have to be a URL param or cookie.

Visiting clutter:
The timestamps do add a fair amount of visual clutter to the log view.  Currently the gutter is 100px wide, and set to the same color as the log lines.  If we make the gutter smaller (60px is probably still safe), and fade out the text, I think it would be less clutter.
Original: https://screenshot.googleplex.com/iojUHTePms1
Faded:    https://screenshot.googleplex.com/Adj26rAHdKs
Assuming fixed bandwidth, half the page size should translate into half the load time, no?
Can we make this an opt-in thing for whatever view this was requested for?
re #4: it is only if we assume the backend could serve logs at the full speed of the network pipe.  This isn't quite true, since we're bottlenecked on the rendering speed of the backend.  Even without timestamps, the load speed is about 1.2Mb/s, which is less than my 1Gb/s connection.

user option is probably the way to go, for this.
Looks like the times are also colored: https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8934205803885461280/+/steps/compile__with_patch_/0/stdout

This is actively misleading for compile output since a bunch of tasks run in parallel and printed when completed. So if something takes a long time but completes right after some fast task, it shows up as fast. That's why we have that about:tracing view of the build already, which displays parallelism.
If there's going to be an user-side option, then let's please default to off.
hinoka@ and I discussed this today. We should default to timestamps being off (as you say), but being to enable them will be a nice feature generically for logs, even if for some logs it'll either be redundant or actively confusing and hence shouldn't be shown.
Also adding in discission with iannucci@:

Half of the feature requests for logdog are to add some sort of enhancements to the logs, for example linkification, subset of html tags, color highlighting for error severity, etc.  The proposal is to reduce modes have two modes for logdog: lite and featured.  Lite will be like the old plain-text mode, and logs are stripped of all features, and the emphasis is on minimum amount of features in exchange for speed.  Featured mode will include all features, including timestamps and the other FRs mentioned above.  The setting will be stored as a cookie (instead of user-settings, which adds complexity), and we can default to lite mode.
This is in addition to the raw mode currently available.
Sounds great, thanks!
Cc: -hinoka@chromium.org
Owner: hinoka@chromium.org
Status: Assigned (was: Untriaged)
This has been around for close to two weeks now. Is it difficult to revert? What's the hold-up?
Status: Fixed (was: Assigned)
Timestamps are now hidden behind a link.  Compared to the original implementation, the log in #3 went from 108KB to 95KB.
👍 Thanks!

Sign in to add a comment