run ./appengine/swarming/tools/os_utilities.py from luci-py.

If you can get the right checkouts on the machines, puppet manages the cron/schtasks jobs that generate the info pages.  I can help you get that going once you rip out the relevant bits of the script and get a working standalone copy, or if you decide to put the whole checkout on the bots that's fine too.

Once you have a solution, assign to me and I'll get it going.

If I understand correctly, one possible solution would be to write a script that
 - Clones luci-py from github to a known location on-bot that does not interfere with the bot's operation (38M as of last check)
 - Updates said checkout if already existed,
 - Runs os_utilities.py

The script would have to be self-contained, hosted at a url visible to the bots (github/luci-py itself?) and its only dependencies would be python and git. (Needs to be python so that it can run on all platforms)

Does that sound reasonable?

I think that should work.

Roberto, please, don't git clone on the bots. I recommend you create a one-time CIPD package and tell friedman@ its name and version hash.

For examples, see https://cs.chromium.org/chromium/infra/build/README.md?sq=package:chromium&dr=C and ask vadimsh@ or me if you have any questions.

CIPD packs are build & uploaded here:
 https://uberchromegw.corp.google.com/i/chromium.infra/console

for example https://uberchromegw.corp.google.com/i/chromium.infra/builders/infra-continuous-trusty-64/builds/9670

and you'll need "git_revision:....." tag.

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/2d17c3f264c0cc52ea8059e2f8b18a9d4b6c0362

commit 2d17c3f264c0cc52ea8059e2f8b18a9d4b6c0362
Author: Roberto Carrillo <robertocn@google.com>
Date: Fri Nov 03 21:38:04 2017

Add luci-py CIPD package definition.

This package is meant to provide the files necessary to run
luci/appengine/swarming/tools/os_utilities.py directly on the bots
without having to do a git checkout.

R=tandrii,vadimsh
BUG= 777494 

Change-Id: Ib52582f4854e46736f6a6a3793df746872f96f77
Reviewed-on: https://chromium-review.googlesource.com/752544
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>

[add] https://crrev.com/2d17c3f264c0cc52ea8059e2f8b18a9d4b6c0362/build/packages/luci-py.yaml

A cipd package is available:

Packages:
  infra/luci-py:91ecc9cf5a56bba80d12f8e1ba39339c2bbb9b3e

From this package, we need to run luci/appengine/swarming/tools/os_utilities.py and show its output on the build

Correction, the path should be appengine/swarming/tools/os_utilities.py after vadim's change lands: https://chromium-review.googlesource.com/c/infra/infra/+/754052

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/b96c3ec5a9d220185045356ab0fb2d86fd68ee56

commit b96c3ec5a9d220185045356ab0fb2d86fd68ee56
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Sun Nov 05 23:24:28 2017

Keep luci-py checkout at the root of infra/luci-py CIPD package.

Previously it was under 'luci/' directory.

R=robertocn@chromium.org
BUG= 777494 

Change-Id: I69946626746a8d482237bb522c9a89427f004b92
Reviewed-on: https://chromium-review.googlesource.com/754052
Reviewed-by: Roberto Carrillo <robertocn@chromium.org>
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>

[modify] https://crrev.com/b96c3ec5a9d220185045356ab0fb2d86fd68ee56/build/packages/luci-py.yaml

Is this on all the bots now?  Please let me know when this is at a place where I can test it.

No. Elliott, can you please help add this package to puppet config s.t. it is deployed to all buildbot slaves?

Yes, but what's the package called?

Package: "infra/luci-py" (see also CL above).

friendly ping.

It's in code review...

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/puppet/+/fe0daba30ce7e90dcead274237236d9c76d835b6

commit fe0daba30ce7e90dcead274237236d9c76d835b6
Author: Elliott Friedman <friedman@google.com>
Date: Fri Dec 08 00:53:07 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/puppet/+/1dcf9f369d08c6d7beb1786b73fe466530e04f56

commit 1dcf9f369d08c6d7beb1786b73fe466530e04f56
Author: Elliott Friedman <friedman@google.com>
Date: Fri Dec 08 01:28:40 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/puppet/+/5f1bbb990ccfaa6a2f500c0d39f2276cd74a19eb

commit 5f1bbb990ccfaa6a2f500c0d39f2276cd74a19eb
Author: Elliott Friedman <friedman@google.com>
Date: Fri Dec 08 20:27:47 2017

I think this is the issue on windows:
chrome-bot@VM1032-M4 ~
$ ls -l C:/chrome-infra/luci-py/appengine/swarming/tools/os_utilities.py
-rwxr-xr-x 1 chrome-bot Domain Users 1377 Dec  8 00:59 C:/chrome-infra/luci-py/appengine/swarming/tools/os_utilities.py

chrome-bot@VM1032-M4 ~
$ /cygdrive/c/infra-system/bin/python C:/chrome-infra/luci-py/appengine/swarming/tools/os_utilities.py
Traceback (most recent call last):
  File "C:/chrome-infra/luci-py/appengine/swarming/tools/os_utilities.py", line 50, in <module>
    sys.exit(main())
  File "C:/chrome-infra/luci-py/appengine/swarming/tools/os_utilities.py", line 21, in main
    test_env_bot.setup_test_env()
  File "C:\chrome-infra\luci-py\appengine\swarming\swarming_bot\test_env_bot.py", line 38, in setup_test_env
    init_symlinks(BOT_DIR)
  File "C:\chrome-infra\luci-py\appengine\swarming\swarming_bot\test_env_bot.py", line 27, in init_symlinks
    with open(path) as f:
IOError: [Errno 13] Permission denied: 'C:\\chrome-infra\\luci-py\\appengine\\swarming\\swarming_bot\\adb'

chrome-bot@VM1032-M4 ~
$ ls -l C:\\chrome-infra\\luci-py\\appengine\\swarming\\swarming_bot\\adb
lrwxrwxrwx 1 chrome-bot Domain Users 32 Dec  8 00:59 C:\chrome-infra\luci-py\appengine\swarming\swarming_bot\adb -> ../../third_party/python-adb/adb

chrome-bot@VM1032-M4 ~
$ ls -l C:/chrome-infra/luci-py/appengine/third_party/python-adb/adb
total 100
-rw-r--r-- 1 chrome-bot Domain Users     0 Dec  8 00:59 __init__.py
-rw-r--r-- 1 chrome-bot Domain Users  8220 Dec  8 00:59 adb_commands.py
-rwxr-xr-x 1 chrome-bot Domain Users  1997 Dec  8 00:59 adb_debug.py
-rw-r--r-- 1 chrome-bot Domain Users 19957 Dec  8 00:59 adb_protocol.py
-rw-r--r-- 1 chrome-bot Domain Users 14853 Dec  8 00:59 common.py
-rw-r--r-- 1 chrome-bot Domain Users  4814 Dec  8 00:59 common_cli.py
drwxr-xr-x 1 chrome-bot Domain Users     0 Dec  8 00:59 contrib
-rw-r--r-- 1 chrome-bot Domain Users 12524 Dec  8 00:59 fastboot.py
-rwxr-xr-x 1 chrome-bot Domain Users  1825 Dec  8 00:59 fastboot_debug.py
-rw-r--r-- 1 chrome-bot Domain Users  7969 Dec  8 00:59 filesync_protocol.py
-rw-r--r-- 1 chrome-bot Domain Users  1041 Dec  8 00:59 sign_m2crypto.py
-rw-r--r-- 1 chrome-bot Domain Users  2516 Dec  8 00:59 sign_pythonrsa.py
-rw-r--r-- 1 chrome-bot Domain Users  2225 Dec  8 00:59 usb_exceptions.py

Err, are symlinks now checked out as symlinks? The code assumes they are not.

Friendly ping.

Its a symlink to a directory, not a binary...

robertocn, do you still need this?

Not anymore, thanks

I think what we needed, we got. Wed don't need to invest more into this.