New issue
Advanced search Search tips

Issue 699331 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: 15
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocked on:
issue 699326
issue 699327
issue 699336
issue 715848



Sign in to add a comment

Bots should manage $TMPDIR, $TMP, $TEMP to achieve better hermeticity

Project Member Reported by iannu...@google.com, Mar 8 2017

Issue description

Currently:
  * builds leak files into these directories
  * builds that crash or leak handles sometimes leak locks between multiple builds on the same machine. Since all builds currently share the same tempdir, this means that leaked lockfiles or simply having files opened on windows causes issues between builds
  * no clean way to 'clean' the tempdir; currently we use a big hammer that does a 'try to delete everything and hope for the best' pass on temp

The solution here is to have the builders set unique values for the various temp environment variables before running the task. Additionally, when the task starts, they'll attempt to delete any existing temporary directories that were leftovers.
 
Blockedon: 699327
Blockedon: 699326
Also blocked on b/36039166
Blockedon: 699336
It looks like there's some other interesting stuff going on with client.art too (besides the path length stuff): https://luci-milo.appspot.com/buildbot/client.art/hammerhead-concurrent-collector/2705

It looks like that master runs multiple builds on the same slaves simultaneously (!!).
Labels: -Pri-3 Pri-2
Looks like some cumulative knowledge from the various blockers:
  * On windows, place temp dir in C:\ to avoid cross-volume issues
  * On linux, drop pid files/hold flocks in temp dirs to avoid issues like the ones seen on the art builder (where multiple slaves run on the same host at the same time).

Comment 8 by iannu...@google.com, Apr 27 2017

Blockedon: 715848

Comment 9 by efoo@chromium.org, May 4 2017

EstimatedDays: 15
Labels: LUCI-M0-Backlog REQBY-LUCI-M4-OpenBeta
Status: Available (was: Untriaged)
Project Member

Comment 11 by sheriffbot@chromium.org, Jul 25 2017

Labels: Hotlist-Google

Comment 12 by efoo@chromium.org, Aug 14 2017

Components: -Infra>Platform Infra>Platform>Buildbot
Labels: -REQBY-LUCI-M4-OpenBeta -LUCI-M0-Backlog

Comment 13 by no...@chromium.org, Jun 16 2018

Labels: -Pri-2 Pri-3
pri-3 because luci

Sign in to add a comment