New issue
Advanced search Search tips

Issue 842802 link

Starred by 4 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocked on:
issue 846383
issue 846712



Sign in to add a comment

isolate go defaults to writable checkout instead of read-only

Project Member Reported by mar...@chromium.org, May 14 2018

Issue description

When refactoring with mb, we lost track of the read_only flag that was set to 1. It still default to 0, which means that for the last few years the files have been mapped as read-write.

AI:
- Try to enable read-only mapping.
- Spot all the failures, fix them.
- Redo until the roll stick in.

It'll likely take a while. I'll do it at the mb.py level but we should eventually change the default too.
 

Comment 2 by mar...@chromium.org, May 14 2018

Forked off  issue 841878 .

The default is true on python code base, but not the Go one. That's why it silently regressed. We cannot easily just turn it on and hope for it work, it doesn't. Even the python code had regression creep in, e.g. the internal failure in https://chromium-swarm.appspot.com/task?id=3d78d4a3d9d2f210 is due to a mishandling of symlink that happens only if read_only:1 is set.

Comment 3 by mar...@chromium.org, May 14 2018

Here's two examples of a regression of a task that writes to input files:
https://chromium-swarm.appspot.com/task?id=3d78d4aa5abe6210
https://chromium-swarm.appspot.com/task?id=3d78d50f380f5710
Oh, I see. I thought are we talking about the code on the client (i.e., archive/exparchive) or code on the bot? I thought the bot was still python?

Comment 5 by mar...@chromium.org, May 14 2018

run_isolated.py defaults to read_only:0 sadly if read_only is not specified. And yes it should default to 1 but that cannot be switched wholesale now.
got it.

Comment 7 by mar...@chromium.org, May 14 2018

Issue 813413 becomes kinda important. :/
Project Member

Comment 8 by bugdroid1@chromium.org, May 14 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-py.git/+/a2c9980f9c3ae27a7b44fd5e1b04ac469e70636f

commit a2c9980f9c3ae27a7b44fd5e1b04ac469e70636f
Author: Marc-Antoine Ruel <maruel@chromium.org>
Date: Mon May 14 20:25:16 2018

[client] fix isolateserver client file mode handling with tarred filed

It was a bit too relax. Tighten it down a lot. Update test to confirm that
tarred files are correctly written as read-only.

Bug: 842802
Change-Id: I95c9c9ace7aa0a7f6c69ce8edb3ae400a9e5ffe8
Reviewed-on: https://chromium-review.googlesource.com/1057878
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>

[modify] https://crrev.com/a2c9980f9c3ae27a7b44fd5e1b04ac469e70636f/client/isolateserver.py
[modify] https://crrev.com/a2c9980f9c3ae27a7b44fd5e1b04ac469e70636f/client/tests/isolateserver_test.py

Project Member

Comment 9 by bugdroid1@chromium.org, May 15 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-py.git/+/7e767c526988faa3d684ec74f47b49c3b022a948

commit 7e767c526988faa3d684ec74f47b49c3b022a948
Author: Marc-Antoine Ruel <maruel@chromium.org>
Date: Tue May 15 01:42:07 2018

[client] stop mocking the file system in some unit test

Test only change.

I wanted to confirm a suspicion with file mode bug with links, so removed the
mock of file I/O and look at file system. Sadly I didn't reproduce the bug but
this makes the test more useful.

Bug: 842802
Change-Id: I5a389af9a0b1fc2fae3b6a876f6762c9067362fa
Reviewed-on: https://chromium-review.googlesource.com/1058467
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>

[modify] https://crrev.com/7e767c526988faa3d684ec74f47b49c3b022a948/client/tests/isolateserver_test.py

Cc: bpastene@chromium.org
FYI, tests I'm currently adding to the chromeos-amd64-generic-rel trybot rely heavily on being able to modify parts of their input isolate. It's partly a consequence of a funky toolchain cache the tests use, which isn't entirely under our control (mainly owned by cros folks). Depending on how we do it, breaking that dependency would be non-trivial. As it stands, adding more tests to that bot probably takes a higher priority than fixing the cache-modifications... Sorry :(

But once the bot's in a good place, I can try to work on making those tests less broken.

(discovered this issues btw 'cause I'm continuously on the watch for failures on that bot)
Blockedon: 846383
Blockedon: 846712
FWIW, I think I've resolved the chromeos issue I mentioned in #10. So those tests should be compatible w/ a read-only isolate cache now... (I think).

Sign in to add a comment