make test_importer asynchronous on push-to-prod in server lib task. |
|||||||
Issue descriptionI broke the lab today ( issue 730272 ) I spent ~10 + 10 minutes understanding what was wrong. I wpent ~10 + 10 minutes fixing it. I spent one day trying to push to prod (without any testing) and validate the changes. We should speed up push-to-prod to be less than 15 minutes. The folklore in the team is that the slow steps are: - build_externals - Start moving things to puppet wherever possible. - For the rest, add in smartness about when something needs to be updated. - test_importer (this parses test control files from the new sources and updates the local DB on all shards + master DB with the new tests) - Make this asynchronous. Have a cron job once a day to run test_importer. Most prod pushes don't care if the test list updates a little later.
,
Jun 8 2017
At one point, an additional slow step in push to prod was waiting for things like apache and scheduler to restart, which was slowed by Issue 673639 . Do you know if that was improved?
,
Jun 12 2017
,
Jun 15 2017
I am working on adding devserver push to push-to-prod. One of the steps is to replace deployment script with the server lib task. Therefore, I think it may be better to make the change there.
,
Jun 16 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/e0e2bdf0f0fad65ee00281c0b72e081bce7f0b79 commit e0e2bdf0f0fad65ee00281c0b72e081bce7f0b79 Author: Shuqian Zhao <shuqianz@chromium.org> Date: Fri Jun 16 22:55:40 2017
,
Jun 16 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/e0e2bdf0f0fad65ee00281c0b72e081bce7f0b79 commit e0e2bdf0f0fad65ee00281c0b72e081bce7f0b79 Author: Shuqian Zhao <shuqianz@chromium.org> Date: Fri Jun 16 22:55:40 2017
,
Jun 19 2017
,
Jun 20 2017
Due to issues during push I don't know how useful this is, but the push clocked 23 minutes https://bugs.chromium.org/p/chromium/issues/detail?id=734836
,
Jun 20 2017
23 minutes is definitely better than last week (> 1 hour)
,
Jun 26 2017
,
Jun 28 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/c165128e27bb9f0a46d8528284876a00d010d72d commit c165128e27bb9f0a46d8528284876a00d010d72d Author: Shuqian Zhao <shuqianz@chromium.org> Date: Wed Jun 28 21:45:13 2017 autotest: move test_importer out of deploy script test_importer is a time-consuming task in push-to-prod, Move it out of push to prod, and make it as a cronjob running in the backend once a day. BUG= chromium:730830 TEST=unittest Change-Id: I4aed9125ffee3cb93cdaf8abffe678439730b854 Reviewed-on: https://chromium-review.googlesource.com/537970 Commit-Ready: Shuqian Zhao <shuqianz@chromium.org> Tested-by: Shuqian Zhao <shuqianz@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> [modify] https://crrev.com/c165128e27bb9f0a46d8528284876a00d010d72d/site_utils/deploy_server_local.py |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by pprabhu@chromium.org
, Jun 7 2017