In the very worst case, if a ChromeOS build fails badly and fails to reboot, there can bind mounts, loopback mounts and other weirdness inside that cache directory.

We have reliable safeguards to prevent those leaks from happening, but it's possible to leak them in the worst case (ie: if the build ends but leaves a process running inside our chroot).

I've been thinking about it yesterday and I'm not sure how to fix this in a clean way.

Would have it been a docker container, it would have been easy to mount the path by forcing the uid but with chroot this is not possible AFAIK. My knowledge in this area is very limited; I just know how to start docker run --user foo:bar and I feel like god.

In the current context, I feel that either we should grant run_isolated.py "root-like" delete capability, or leverage another mechanism I'm unaware of (capabilities?). I'd prefer to avoid running bot_main.py directly as root.

re leaks: we work around by rebooting after task failure and assume successful tasks clean up properly. This worked generally well enough for our purpose.

RE #3, an option could be to have a directory (/b/swarming/trash) that has world-write, and have a root-owned setuid script that purges everything in that directory. Swarming can just move files into that directory at end of build and run the setuid script to purge them.

I've talked to dgarrett@ about running in a container to contain mounts and stuff. We're OK rebooting in between runs for now. Container won't solve root-owned files though, b/c we'd have to bind-mount to main system anyway.

I meant CAP_FOWNER.

But better: if CAP_SYS_CHROOT was used, maybe root wouldn't be necessary at all?

Ref: http://man7.org/linux/man-pages/man7/capabilities.7.html

Marking available. I think Dan put out a design for solving this but I won't assign to him explicitly since I don't know if/when someone is implementing it.

It is still my belief that this could be easily fixed by I mention in comment #5. Using capabilities instead of passwordless sudo would just fix the problem permanently.

That said,  I'll add a hack in file_path.make_tree_writeable() to try to "sudo find . -type d -exec chmod a+rwX {} +" because of the case here where the bots are passwordless sudoers. That's much simpler and straightforward.

https://github.com/luci/luci-py/blob/master/client/utils/file_path.py#L1015

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-py.git/+/274f6429d25c5243eb8043afff854f884b20cf18

commit 274f6429d25c5243eb8043afff854f884b20cf18
Author: maruel <maruel@chromium.org>
Date: Thu Jun 22 17:22:27 2017

Try 'sudo chmod' in make_tree_deleteable() as last effort

- Some tasks on Swarming may create root owned files on passwordless sudoer
  account. This CL makes rmtree() try running 'sudo chmod' on linux in case the
  current account happens to be passwordless sudoer. If it fails once, do not
  try again.
- Make rmtree() usable even if fix_encoding() wasn't called so the test case
  above can be used.

Because correctly testing this code requires passwordless sudo, it
has to be manually tested.

The 3 manual test cases to confirm the code works as expected:

1) Verify make_tree_deleteable() works with passwordless sudo:
    mkdir -p x/y && sudo chown -R root:root x && sudo chmod -R a-rwX x
    sudo ls -la x x/y
    python -c 'import os; from utils import file_path; \
      file_path.make_tree_deleteable(os.path.abspath(u"x"))'
    ls -la x x/y
    rm -rf x
  Expected:
    x and x/y are fully deletable without sudo in the last rm -rf.

2) Verify rmtree() works with passwordless sudo:
    mkdir -p x/y && sudo chown -R root:root x && sudo chmod -R a-rwX x
    sudo ls -la x x/y
    python -c 'import os; from utils import file_path; \
      file_path.rmtree(os.path.abspath(u"x"))'
    ls -la x
  Expected:
    x and x/y were removed
    A logging warning was printed

3) Verify that a sudo password prompt fails the rmtree():
    mkdir -p x/y && sudo chown -R root:root x && sudo chmod -R a-rwX x
    sudo ls -la x x/y
    sudo -K
    python -c 'import os; from utils import file_path; \
      file_path.rmtree(os.path.abspath(u"x"))'
    ls -la x
  Expected:
    removal failed

R=vadimsh@chromium.org
BUG= 723809 

Review-Url: https://codereview.chromium.org/2949253002

[modify] https://crrev.com/274f6429d25c5243eb8043afff854f884b20cf18/client/utils/file_path.py

As I'm going to vacation now, I'll let Don deploy to chromeos-proxy. Vadim can help with this if you are unfamiliar but I think it's going to be a good exercise.

Huh? Are there docs somewhere?

https://github.com/luci/luci-py/tree/master/appengine/#pushing

I'm going to put this off until tomorrow. Pushing seems to require the AppEngine SDK, which I don't have.

I think I'm following the new directions correctly, but get this error:

luci-py$pwd
/usr/local/google/home/dgarrett/sand/luci-py/appengine/swarming


luci-py$./tools/gae upload --sdk-path ~/bin/google-cloud-sdk/platform/google_appengine chrome-swarming
Usage: gae upload [options] [module_id module_id ...]

gae: error: No such module: chrome-swarming

You forgot -A before chrome-swarming.

Thanks!

Of course, that gets us here:

You do not have permission to modify this app (app_id=u's~chrome-swarming').

Oh, this is for chrome-swarming, not chromeos-proxy...

chrome-swarming is owned by Chrome infra, I'll update it.

Updated.

Thanks!