atest server list --skylab is very slow |
|||||||
Issue descriptionFor example: chromeos-test@chromeos-staging-master2:~$ time /usr/local/autotest/cli/atest server list -r devserver --skylab --env staging -N >/dev/null real 0m17.499s user 0m34.312s sys 0m3.024s I suspect most of this time is spent initializing the local git clone. We could do two things to speed this up: (1) Use a shallow clone without any remote branches / history. I'm not sure this helps too much. (2) Use gitiles instead of cloning git repo for the readonly commands. (1) should be very easy to implement but I'm not sure of the benefit we'll get.
,
May 14 2018
Note that there are already some flags that help with this: pprabhu@pprabhu:skylab$ time atest server delete --skylab --env staging android1758-infra-devserver5.cros.corp.google.com --inventory-repo-dir /tmp/skylab_inventory2 --keep-repo-dir Server android1758-infra-devserver5.cros.corp.google.com is deleted. Please submit the CL at https://chrome-internal-review.googlesource.com/625622 to make the change effective. real 0m18.400s user 0m31.564s sys 0m1.272s pprabhu@pprabhu:skylab$ time atest server delete --skylab --env staging android1758-infra-devserver5.cros.corp.google.com --inventory-repo-dir /tmp/skylab_inventory2 --keep-repo-dir time atest server delete --skylab --env staging chromeos-gt-devserver18.cbf.corp.google.com --inventory-repo-dir /tmp/skylab_inventory2 --keep-repo-dir __bp_preexec_invoke_exec "$_" Server chromeos-gt-devserver18.cbf.corp.google.com is deleted. Please submit the CL at https://chrome-internal-review.googlesource.com/625623 to make the change effective. real 0m3.911s user 0m0.996s sys 0m0.536s These flags are not the default, and not very intuitive to use. But this bug remains P2 for now, because workarounds exist.
,
May 14 2018
,
May 19 2018
,
Jun 8 2018
"git clone --depth 1" can speed up readonly operations. not sure how it affects write operations though.
,
Jun 22 2018
,
Jun 22 2018
'git clone --depth 1' isn't suuuper fast either (still gets too many git objects). I'll start with that anyway.
,
Jun 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/af1bd7b4faad9bc74c7f797eb12fac9509cdc2e1 commit af1bd7b4faad9bc74c7f797eb12fac9509cdc2e1 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Sat Jun 23 03:19:27 2018 skylab_utils: Do a shallow clone to save time. BUG=chromium:842754 TEST=manual Change-Id: I92f7dedda150b2fe733d06d5a762d8458c9902fa Reviewed-on: https://chromium-review.googlesource.com/1112621 Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org> Tested-by: Prathmesh Prabhu <pprabhu@chromium.org> Reviewed-by: Jacob Kopczynski <jkop@chromium.org> [modify] https://crrev.com/af1bd7b4faad9bc74c7f797eb12fac9509cdc2e1/client/common_lib/revision_control.py [modify] https://crrev.com/af1bd7b4faad9bc74c7f797eb12fac9509cdc2e1/cli/skylab_utils.py
,
Jul 10
Issue 853387 has been merged into this issue.
,
Jul 10
Still too slow...
,
Sep 12
Not actively working on this. My personal workaround is to go the inventory sources, luke.
,
Sep 12
> [ ... ] My personal workaround is to go the inventory sources, luke. I think that for the use cases where the performance of `atest server list` matters, going to the source will be slower. So, arguably, that's not a workaround... Also, will this issue affect other skylab use cases, such as `dut-status` or `atest host list`? If the only affected use case is listing servers, it can be P2 or P3, but if everything that queries the inventory is affected, this really should become a blocker. |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by pprabhu@chromium.org
, May 14 2018