New issue
Advanced search Search tips

Issue 901549 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Nov 19
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 1
Type: Bug



Sign in to add a comment

LogDog: Fix archivist and turn it back on

Project Member Reported by hinoka@chromium.org, Nov 2

Issue description

The archivist is currently turned off due to  crbug.com/900148 

In about 10 days, this will lead to data loss.

Tracking bug to fix the underlying issue and turn the archivist back on.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Nov 7

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/a3fc31c5e4f551fa1b7e60904ce0b1df72cfd249

commit a3fc31c5e4f551fa1b7e60904ce0b1df72cfd249
Author: Ryan Tseng <hinoka@google.com>
Date: Wed Nov 07 00:42:26 2018

[logdog] Do not schedule optimistic archival

Previously, the behiavor of logdog was to schedule (up to) 2 archival tasks.
For example if ArchiveDelayMax was set to 48hr, and SettleDelay set to 5min:
RegisterPrefix: Schedule archival in 48hrs (pessimistic archival)
TerminateStream: Schedule archival in 5 min, replace pessimistic archival (optimistic archival)

This CL removes the optimistic archival, so all stream are archived 48hrs
after registration, regardless of when they were terminated.

This result of this is that all archival tasks received by the archivist
are assumed to be final (stored in bigtable).
In the unlikely event any stream is not final, we are not willing
to wait any longer, and we forcefully terminate the stream anyways.

Bug:  901549 
Change-Id: I431232a0237ed71a8d19b53770b8ff21983a5a10
Reviewed-on: https://chromium-review.googlesource.com/c/1315469
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/a3fc31c5e4f551fa1b7e60904ce0b1df72cfd249/logdog/appengine/coordinator/endpoints/services/terminateStream.go
[modify] https://crrev.com/a3fc31c5e4f551fa1b7e60904ce0b1df72cfd249/logdog/appengine/coordinator/endpoints/services/terminateStream_test.go

Project Member

Comment 2 by bugdroid1@chromium.org, Nov 7

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/41343b1b13bbeedb8dc52a4b3001a988708133b2

commit 41343b1b13bbeedb8dc52a4b3001a988708133b2
Author: Ryan Tseng <hinoka@google.com>
Date: Wed Nov 07 18:05:47 2018

Project Member

Comment 3 by bugdroid1@chromium.org, Nov 8

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/0a1f27f24977ff46c367d85941ef3b49eb2585e2

commit 0a1f27f24977ff46c367d85941ef3b49eb2585e2
Author: Ryan Tseng <hinoka@google.com>
Date: Thu Nov 08 19:18:39 2018

[gs] Update condition for isNotFoundError

At some point the error returned from the library changed to ErrObjectNotExist

Bug:  901549 
Change-Id: Ic6a6204b5d81f07cebc65843b168de83af159951
Reviewed-on: https://chromium-review.googlesource.com/c/1327462
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/0a1f27f24977ff46c367d85941ef3b49eb2585e2/common/gcloud/gs/gs.go

Status: Fixed (was: Started)
Archivist is back on.

Unfortunately we did lose some logs (most logs between Oct 2nd - Oct 22nd, and some on Oct 31st) in the process

Sign in to add a comment