New issue
Advanced search Search tips

Issue 844478 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 17
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

skylab: skylab_inventory_manager choking on corrupted old git checkout

Project Member Reported by pprabhu@chromium.org, May 18 2018

Issue description

On pprabhu-skylab-drone-2.cbf, skylab_inventory_manager is crashing, failing to make progress.

The checkout layout is:

chromeos-test@pprabhu-skylab-drone-2:/opt/infra-data/skylab_inventory$ ls -lt | head
total 132
lrwxrwxrwx 1 chromeos-test eng   86 May 17 17:00 latest -> /opt/infra-data/skylab_inventory/skylab_inventory_67e80def-bf44-45ee-b65f-469ad52bc991
drwxr-xr-x 6 chromeos-test eng 4096 May 17 17:00 skylab_inventory_a362a742-3c9f-41ba-ac31-ae1871f26f07
drwxr-xr-x 3 chromeos-test eng 4096 May 17 16:59 skylab_inventory_67e80def-bf44-45ee-b65f-469ad52bc991
drwxr-xr-x 6 chromeos-test eng 4096 May 17 16:58 skylab_inventory_986f2293-fae3-4461-b59a-7a7687a9160f

And 
chromeos-test@pprabhu-skylab-drone-2:/opt/infra-data/skylab_inventory$ ls skylab_inventory_67e80def-bf44-45ee-b65f-469ad52bc991
chromeos-test@pprabhu-skylab-drone-2:/opt/infra-data/skylab_inventory$

So somehow, 'latest' ended up pointing to a directory that is empty. That's bug of its own, but skylab_inventory_manager is failing to fix this condition because:


chromeos-test@pprabhu-skylab-drone-2:/opt/infra-data/skylab_inventory$ /usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/bin/skylab_inventory_manager \
>     --datadir /opt/infra-data/skylab_inventory \
>     --remote https://chrome-internal.googlesource.com/chromeos/infra_internal/skylab_inventory/ \
>     --wait-seconds 30 \
>     --start-jitter-seconds 3
2018-05-18 09:10:29,898 NOTI| ts_mon was set up.
2018-05-18 09:10:29,899 INFO| Start jitter: Sleeping for 0 seconds...
2018-05-18 09:10:29,899 INFO| Jitter done. Starting work.
2018-05-18 09:10:29,899 INFO| ### Starting update ###
2018-05-18 09:10:29,899 DEBU| Running command: git rev-parse HEAD
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 288, in <module>
    main()
  File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 67, in main
    _sync_forever(opts)
  File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 77, in _sync_forever
    _sync_once(opts.datadir, opts.remote)
  File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 96, in _sync_once
    _create_latest_checkout(managed_dir, remote, gitc)
  File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 112, in _create_latest_checkout
    old_head = gitc.head(current_checkout)
  File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 272, in head
    return _run(['git', 'rev-parse', 'HEAD'], checkout_dir)
  File "/usr/local/google/home/chromeos-test/chromiumos/chromeos-admin/venv/skylab_inventory_manager/manager.py", line 284, in _run
    return subprocess.check_output(cmd, cwd=cwd)
  File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128
 
Project Member

Comment 2 by bugdroid1@chromium.org, May 20 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/71e72160a7bf4d0b39fa33a31f1da3b5ea0c6bc8

commit 71e72160a7bf4d0b39fa33a31f1da3b5ea0c6bc8
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Sun May 20 19:07:25 2018

Project Member

Comment 3 by bugdroid1@chromium.org, May 22 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/0d2fd03dfb5be36e04689f582cdd3e8e19138863

commit 0d2fd03dfb5be36e04689f582cdd3e8e19138863
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue May 22 20:16:18 2018

Status: Fixed (was: Started)

Sign in to add a comment