New issue
Advanced search Search tips

Issue 848817 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Status of failed isolator doesn't seem to propagate correctly

Project Member Reported by vadimsh@chromium.org, Jun 1 2018

Issue description

Labels: Tricium
Labels: -Pri-3 Pri-2
Status: Available (was: Untriaged)
The above case:
https://chromium-review.googlesource.com/c/infra/luci/luci-py/+/1080733/7
https://tricium-dev.appspot.com/run/5648750718484480


Another possibly related case:

At least once, the swarming tasks completed successfully but Tricium didn't update its state:
https://chromium-review.googlesource.com/c/playground/gerrit-tricium/+/1101311
https://tricium-dev.appspot.com/run/5069141895544832

This could be a separate case; also possibly related; I'm seeing "Create Pub/Sub subscription: tricium-dev@appspot.gserviceaccount.com failed to create worker-completion" errors in the App Engine activity log.
Components: Infra>Platform>Tricium
Components: -Infra>CodeAnalysis
Labels: -Tricium
Summary: Status of failed isolator doesn't seem to propagate correctly (was: Status of failed analyzers doesn't seem to propagate correctly )
Similar case:
https://tricium-prod.appspot.com/run/5735072447594496
https://chromium-swarm.appspot.com/task?id=3f06b150a25f0910

In both this and the original case, GitFileIsolator failed with something like:

2018/06/01 00:03:24 Command failed: exit status 1, stderr: cp: cannot stat '/b/s/w/itp49t0Z/git-file-isolator736926028/appengine/components/components/prpc/discovery/test_support': No such file or directory
2018/07/31 01:07:35 Command failed: exit status 1, stderr: cp: cannot stat '/b/s/w/itZO7gN9/git-file-isolator159455758/overlay-nocturne/chromeos-base/chromeos-bsp-nocturne/chromeos-bsp-nocturne-0.0.1-r7.ebuild': No such file or directory

After GitFileIsolator failed, it was marked as failed, but the subsequent analyzers were never run.
What should be done instead: if an isolator fails, subsequent analyzers should be aborted.
Project Member

Comment 6 by bugdroid1@chromium.org, Aug 1

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/2bf923ab8f3042d567d372f554f215948787ee76

commit 2bf923ab8f3042d567d372f554f215948787ee76
Author: Quinten Yearsley <qyearsley@chromium.org>
Date: Wed Aug 01 20:50:15 2018

[tricium GitFileIsolator] Skip symlinks

Sometimes the GitFileIsolator fails to copy files,
in particular when the file is a symlink. This change
is supposed to make it so that only regular files
(not symlinks) are copied.

Bug: 848817
Change-Id: I0ca3fa838c6cf0a1f95f17ddad9f52268f372405
Reviewed-on: https://chromium-review.googlesource.com/1159111
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>

[modify] https://crrev.com/2bf923ab8f3042d567d372f554f215948787ee76/go/src/infra/tricium/functions/git-file-isolator/isolator.go

Owner: qyears...@chromium.org
Status: Started (was: Available)
Project Member

Comment 8 by bugdroid1@chromium.org, Aug 2

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/9cb61755e7302eb8ce06d2458d2cfa829a117ee3

commit 9cb61755e7302eb8ce06d2458d2cfa829a117ee3
Author: Quinten Yearsley <qyearsley@chromium.org>
Date: Thu Aug 02 18:11:56 2018

[tricium GitFileIsolator] Follow-up fix and test update

Before submitting https://crrev.com/c//1159111, I didn't
run the isolator with the test input, which would have caught
the bug (trying to stat relative path, which was incorrect).

This CL fixes that and updates the test to a test case that
Vadim pointed out a while ago. After running this I see that
it behaves as expected, skipping symlinks and writing output
which doesn't include the symlinks.

Bug: 848817
Change-Id: I1919fb4ca85928082fc40d0fa997228ea748d9de
Reviewed-on: https://chromium-review.googlesource.com/1161017
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>

[modify] https://crrev.com/9cb61755e7302eb8ce06d2458d2cfa829a117ee3/go/src/infra/tricium/functions/git-file-isolator/test/tricium/data/git_file_details.json
[modify] https://crrev.com/9cb61755e7302eb8ce06d2458d2cfa829a117ee3/go/src/infra/tricium/functions/git-file-isolator/isolator.go

Owner: ----
Status: Available (was: Started)
Note, I think that this isn't fixed; the above CLs just fixed GitFileIsolator but not failure propogation. Next step: Clarify exactly what is wrong and find other example to reproduce this.

Sign in to add a comment