New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 826488 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2018
Cc:
Components:
EstimatedDays: 1
NextAction: ----
OS: ----
Pri: 1
Type: Task

Blocking:
issue 826487
issue 826497
issue 826500



Sign in to add a comment

Bootstrap an always-updated checkout of the skylab_inventory project on lab servers

Project Member Reported by pprabhu@chromium.org, Mar 27 2018

Issue description

This checkout will be used by
- swarming bot for skylab-drone
- inventory migration jobs in AutotestInfra to inject the inventory information into autotest db.

repo to setup: https://chrome-internal.googlesource.com/chromeos/infra_internal/skylab_inventory/
 
Blocking: 826487
EstimatedDays: 1
Labels: -Type-Bug Type-Task
Blocking: 826497
Blocking: 826500
Created a new external project to house all the protos / tools related to skylab inventory. Only the data will remain in the internal repo:

https://chromium-review.googlesource.com/admin/repos/chromiumos/infra/skylab_inventory
Project Member

Comment 6 by bugdroid1@chromium.org, Mar 27 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/manifest-internal/+/6947f70600870214d998a3e207bd1e5fe0474bee

commit 6947f70600870214d998a3e207bd1e5fe0474bee
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Mar 27 21:40:16 2018

Project Member

Comment 7 by bugdroid1@chromium.org, Mar 27 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/infra/skylab_inventory/+/dc1bfb3e7933ab61fd5cf356f41308b677899af6

commit dc1bfb3e7933ab61fd5cf356f41308b677899af6
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Mar 27 21:39:58 2018

Initial import from infra_internal/skylab_inventory.

Directory structure:
- protos/ contains the inventory schema definition.
- venv/ contains various scripts to query data in the schema specified
  above.
  - These scripts are currently broken because they expect the data to
    be in the same repo as the tools. Will be fixed in a future CL.
- venv/skylab_inventory/protos/ contain protoc generated python bindings
  for the inventory schema
- bin/ entry points to the scripts above.

BUG= chromium:826488 
TEST=None

Change-Id: I845a9dec6e761f67e3f360789d8dc58b49319158
Reviewed-on: https://chromium-review.googlesource.com/982644
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>

[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/scripts/print_dut_state.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/protos/server_pb2.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/bin/print_dut_by_id
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/scripts/print_dut_host_info.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/protos/lab.proto
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/scripts/list_duts_for_drone.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/protos/device_pb2.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/scripts/print_dut_dimensions.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/bin/print_dut_host_info
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/bin/print_dut_dimensions
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/.gitignore
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/protos/timestamp_pb2.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/protos/lab_pb2.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/bin/python_venv
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/requirements.txt
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/bin/print_servers
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/protos/connection_pb2.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/protos/stable_versions.proto
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/protos/stable_versions_pb2.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/bin/print_dut_state
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/examples/print_dut_by_id.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/loader.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/__init__.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/protos/connection.proto
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/README.md
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/examples/print_servers.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/protos/timestamp.proto
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/bin/list_duts_for_drone
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/protos/server.proto
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/examples/__init__.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/scripts/__init__.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/venv/skylab_inventory/protos/__init__.py
[add] https://crrev.com/dc1bfb3e7933ab61fd5cf356f41308b677899af6/protos/device.proto

Project Member

Comment 8 by bugdroid1@chromium.org, Mar 27 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/manifest/+/fa1565cadbe414c67bc14a9603a8650b5809f869

commit fa1565cadbe414c67bc14a9603a8650b5809f869
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Mar 27 22:09:40 2018

Add public skylab-inventory project.

BUG= chromium:826488 
TEST='repo sync -t copied.xml'

Change-Id: Ie505902976461086338f274dc4ff6af30ce60797
Reviewed-on: https://chromium-review.googlesource.com/982636
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>

[modify] https://crrev.com/fa1565cadbe414c67bc14a9603a8650b5809f869/full.xml

Project Member

Comment 9 by bugdroid1@chromium.org, Mar 27 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/infra/skylab_inventory/+/aac77eae11ec8e433c49f761af1ae205cf7177da

commit aac77eae11ec8e433c49f761af1ae205cf7177da
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Mar 27 22:10:46 2018

Explicitly provide inventory data directory to scripts.

Now that inventory data is separated out from schema definition / tools,
updates the tools to take in the path to the data checkout explicitly.

BUG= chromium:826488 
TEST=All the tools work again!

Change-Id: Id17251668763114d2686fac8573102642f3c2bdb
Reviewed-on: https://chromium-review.googlesource.com/982680
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>

[modify] https://crrev.com/aac77eae11ec8e433c49f761af1ae205cf7177da/venv/skylab_inventory/scripts/print_dut_state.py
[modify] https://crrev.com/aac77eae11ec8e433c49f761af1ae205cf7177da/venv/skylab_inventory/examples/print_servers.py
[modify] https://crrev.com/aac77eae11ec8e433c49f761af1ae205cf7177da/bin/print_servers
[modify] https://crrev.com/aac77eae11ec8e433c49f761af1ae205cf7177da/venv/skylab_inventory/scripts/print_dut_dimensions.py
[modify] https://crrev.com/aac77eae11ec8e433c49f761af1ae205cf7177da/venv/skylab_inventory/scripts/list_duts_for_drone.py
[modify] https://crrev.com/aac77eae11ec8e433c49f761af1ae205cf7177da/venv/skylab_inventory/scripts/print_dut_host_info.py
[modify] https://crrev.com/aac77eae11ec8e433c49f761af1ae205cf7177da/venv/skylab_inventory/examples/print_dut_by_id.py
[modify] https://crrev.com/aac77eae11ec8e433c49f761af1ae205cf7177da/venv/skylab_inventory/loader.py

TODO: add a presubmit to protect compiled proto bindings.
https://chrome-internal-review.googlesource.com/c/chromeos/chromeos-admin/+/596932/

This stack deploys a new service to our servers to manage an up-to-date checkout of the inventory project on our servers.

Project Member

Comment 13 by bugdroid1@chromium.org, Mar 29 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/a1fa0a907753f69f39c8a94b681861db9d856cb0

commit a1fa0a907753f69f39c8a94b681861db9d856cb0
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Thu Mar 29 00:40:49 2018

Project Member

Comment 14 by bugdroid1@chromium.org, Mar 29 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/3b775a77c32b9844e309e710bb023293178f033b

commit 3b775a77c32b9844e309e710bb023293178f033b
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Thu Mar 29 00:40:49 2018

Project Member

Comment 15 by bugdroid1@chromium.org, Mar 29 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/c60cb9d34d9f394b5e455c962625d8d0c9f2ba41

commit c60cb9d34d9f394b5e455c962625d8d0c9f2ba41
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Thu Mar 29 00:40:50 2018

Project Member

Comment 16 by bugdroid1@chromium.org, Mar 29 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/infra/skylab_inventory/+/398020453412c5f6963de25631d14cc9471519e7

commit 398020453412c5f6963de25631d14cc9471519e7
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Thu Mar 29 00:51:45 2018

Add presubmit check to validate py bindings against proto definitions.

BUG= chromium:826488 
TEST=manual, presubmit.

Change-Id: I9ac9f7311d6bfa71f5a947f2eed7b20150dbf033
Reviewed-on: https://chromium-review.googlesource.com/985148
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Trybot-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>

[add] https://crrev.com/398020453412c5f6963de25631d14cc9471519e7/bin/check_proto_bindings
[add] https://crrev.com/398020453412c5f6963de25631d14cc9471519e7/PRESUBMIT.cfg

Project Member

Comment 17 by bugdroid1@chromium.org, Mar 29 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/ed5db36cebd65d86e0aecf4bfe834d2577982e02

commit ed5db36cebd65d86e0aecf4bfe834d2577982e02
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Thu Mar 29 21:28:32 2018

Project Member

Comment 18 by bugdroid1@chromium.org, Mar 29 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/083db8a8afd9eeb584ba9cbb5c9288f546a01be4

commit 083db8a8afd9eeb584ba9cbb5c9288f546a01be4
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Thu Mar 29 21:30:41 2018

Status: Fixed (was: Started)
Project Member

Comment 20 by bugdroid1@chromium.org, Mar 29 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/79f1148a319249b366fff2e61cf758764e78dac8

commit 79f1148a319249b366fff2e61cf758764e78dac8
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Thu Mar 29 21:55:58 2018

Project Member

Comment 21 by bugdroid1@chromium.org, Apr 2 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/93e53b499f175be04526e11ca76a6165c2678846

commit 93e53b499f175be04526e11ca76a6165c2678846
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Mon Apr 02 17:37:13 2018

Project Member

Comment 22 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/infra/lucifer/+/fa504ff4781ca662137ef22d304273e3b8736d53

commit fa504ff4781ca662137ef22d304273e3b8736d53
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Wed Apr 18 21:46:56 2018

skylab_swarming: Evaluate inventory data directory before use

The data checkout on skylab-drone is a symlink that should be evluated
before using the inventory data. The symlink can be missing for small
amounts of time (while it is updated) but once evaluated, the target
directory is guaranteed to exist for some time.

BUG= chromium:826488 
TEST=make check

Change-Id: I15ffca65df0ba0a85dd3be6b825eed0b89fa9fa7
Reviewed-on: https://chromium-review.googlesource.com/1011626
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>

[modify] https://crrev.com/fa504ff4781ca662137ef22d304273e3b8736d53/src/lucifer/cmd/skylab_swarming_worker/main.go

Project Member

Comment 23 by bugdroid1@chromium.org, May 16 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/610ad2e96df46a02413696b3e5d0377d02c8655f

commit 610ad2e96df46a02413696b3e5d0377d02c8655f
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Wed May 16 21:09:46 2018

Project Member

Comment 24 by bugdroid1@chromium.org, May 22 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/c5a6cba1eefba4e15f505d6cc3d8a46bcf7c23c6

commit c5a6cba1eefba4e15f505d6cc3d8a46bcf7c23c6
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue May 22 20:16:31 2018

Sign in to add a comment