skylab: skylab_inventory_manager choking on corrupted old git checkout |
||
Issue descriptionOn pprabhu-skylab-drone-2.cbf, skylab_inventory_manager is crashing, failing to make progress. The checkout layout is: chromeos-test@pprabhu-skylab-drone-2:/opt/infra-data/skylab_inventory$ ls -lt | head total 132 lrwxrwxrwx 1 chromeos-test eng 86 May 17 17:00 latest -> /opt/infra-data/skylab_inventory/skylab_inventory_67e80def-bf44-45ee-b65f-469ad52bc991 drwxr-xr-x 6 chromeos-test eng 4096 May 17 17:00 skylab_inventory_a362a742-3c9f-41ba-ac31-ae1871f26f07 drwxr-xr-x 3 chromeos-test eng 4096 May 17 16:59 skylab_inventory_67e80def-bf44-45ee-b65f-469ad52bc991 drwxr-xr-x 6 chromeos-test eng 4096 May 17 16:58 skylab_inventory_986f2293-fae3-4461-b59a-7a7687a9160f And chromeos-test@pprabhu-skylab-drone-2:/opt/infra-data/skylab_inventory$ ls skylab_inventory_67e80def-bf44-45ee-b65f-469ad52bc991 chromeos-test@pprabhu-skylab-drone-2:/opt/infra-data/skylab_inventory$ So somehow, 'latest' ended up pointing to a directory that is empty. That's bug of its own, but skylab_inventory_manager is failing to fix this condition because: chromeos-test@pprabhu-skylab-drone-2:/opt/infra-data/skylab_inventory$ /usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/bin/skylab_inventory_manager \ > --datadir /opt/infra-data/skylab_inventory \ > --remote https://chrome-internal.googlesource.com/chromeos/infra_internal/skylab_inventory/ \ > --wait-seconds 30 \ > --start-jitter-seconds 3 2018-05-18 09:10:29,898 NOTI| ts_mon was set up. 2018-05-18 09:10:29,899 INFO| Start jitter: Sleeping for 0 seconds... 2018-05-18 09:10:29,899 INFO| Jitter done. Starting work. 2018-05-18 09:10:29,899 INFO| ### Starting update ### 2018-05-18 09:10:29,899 DEBU| Running command: git rev-parse HEAD fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this: 'git <command> [<revision>...] -- [<file>...]' Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 288, in <module> main() File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 67, in main _sync_forever(opts) File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 77, in _sync_forever _sync_once(opts.datadir, opts.remote) File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 96, in _sync_once _create_latest_checkout(managed_dir, remote, gitc) File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 112, in _create_latest_checkout old_head = gitc.head(current_checkout) File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 272, in head return _run(['git', 'rev-parse', 'HEAD'], checkout_dir) File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 284, in _run return subprocess.check_output(cmd, cwd=cwd) File "/usr/lib/python2.7/subprocess.py", line 573, in check_output raise CalledProcessError(retcode, cmd, output=output) subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128
,
May 20 2018
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/71e72160a7bf4d0b39fa33a31f1da3b5ea0c6bc8 commit 71e72160a7bf4d0b39fa33a31f1da3b5ea0c6bc8 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Sun May 20 19:07:25 2018
,
May 22 2018
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/0d2fd03dfb5be36e04689f582cdd3e8e19138863 commit 0d2fd03dfb5be36e04689f582cdd3e8e19138863 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Tue May 22 20:16:18 2018
,
Aug 17
|
||
►
Sign in to add a comment |
||
Comment 1 by pprabhu@chromium.org
, May 18 2018