instantiate betty-informational builder to run long tests in VM |
|||||||||
Issue descriptionWe would like to run long running tests on betty (vm image) to clean up the VM. In particular suite:deqp (or the master subset), but probably at some point all of suite:arc-cts. Because these suites take so long (and many tests don't pass yet on betty) we can't simply add all these tests to suite:smoke. So we need to set up a clone of betty (call it say betty-informational) for informational purposes, that builds maybe once a day and runs these tests. One caveat, VMTest does not report to wmatrix etc. Maybe we can think of how to report to the CTS dashboard already and how to upload to sponge.
,
Aug 3 2017
@dgarret, @xixuan Are there any document/doc/code that show how the current tests' report mechanism works? Is it possible for us to get the VMTests' result show on wmatrix/CTS dashboard/sponge as well? If you can point me to the right place, I would be really appreciated.
,
Aug 3 2017
To the best of my knowledge, our current vmtest results don't appear anywhere other than the waterfall. xixuan@ or pprabhu@ might know better, or have advice about how to export the results.
,
Aug 3 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/a60f70853ce02134d3d6c45ea0b8243fb1dacb01 commit a60f70853ce02134d3d6c45ea0b8243fb1dacb01 Author: Po-Hsien Wang <pwang@chromium.org> Date: Thu Aug 03 22:13:56 2017 chromeos_config: Add new builder for vmtest First step to add new builder for vmtest as we want some results from vmtest and we do not want to affect existing vmtest builders. BUG= chromium:750374 TEST=run_tests Change-Id: Ia2cf8947f90257f3c01bda2ff891a004967546af Reviewed-on: https://chromium-review.googlesource.com/596788 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> [modify] https://crrev.com/a60f70853ce02134d3d6c45ea0b8243fb1dacb01/cbuildbot/commands.py [modify] https://crrev.com/a60f70853ce02134d3d6c45ea0b8243fb1dacb01/cbuildbot/chromeos_config.py [modify] https://crrev.com/a60f70853ce02134d3d6c45ea0b8243fb1dacb01/lib/config_lib.py [modify] https://crrev.com/a60f70853ce02134d3d6c45ea0b8243fb1dacb01/cbuildbot/config_dump.json [modify] https://crrev.com/a60f70853ce02134d3d6c45ea0b8243fb1dacb01/lib/constants.py [modify] https://crrev.com/a60f70853ce02134d3d6c45ea0b8243fb1dacb01/cbuildbot/builders/test_builders.py
,
Aug 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/13f7f69115d15795e5683ae12380fb0907e22485 commit 13f7f69115d15795e5683ae12380fb0907e22485 Author: Po-Hsien Wang <pwang@chromium.org> Date: Sun Aug 06 05:07:54 2017 chromeos_config: Add internal to betty-vmtest-informational Set internal flag for betty-vmtest-informational as it betty is in internal overlay. BUG= chromium:750374 TEST=run_suite Change-Id: I3ff56fa0bc1ff6ea409c9a3c3ace7c25d6e72c85 Reviewed-on: https://chromium-review.googlesource.com/602928 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> [modify] https://crrev.com/13f7f69115d15795e5683ae12380fb0907e22485/cbuildbot/config_dump.json [modify] https://crrev.com/13f7f69115d15795e5683ae12380fb0907e22485/cbuildbot/chromeos_config.py
,
Aug 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/ec8d2f17efc6f24fbd29e745f9da4fff5531ea49 commit ec8d2f17efc6f24fbd29e745f9da4fff5531ea49 Author: Po-Hsien Wang <pwang@chromium.org> Date: Wed Aug 09 21:05:09 2017 betty-vmtest-informational: Minor fix 1) Use internal template to assign this to internal overlay (betty is an internal board) 2) Fix stage name not found issue. BUG= chromium:750374 TEST=cbuildbot --remote -g 604471 betty-vmtest-informational Change-Id: I6d3973456ef517562a6798031710e21189c37490 Reviewed-on: https://chromium-review.googlesource.com/604471 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Po-Hsien Wang <pwang@chromium.org> [modify] https://crrev.com/ec8d2f17efc6f24fbd29e745f9da4fff5531ea49/cbuildbot/config_dump.json [modify] https://crrev.com/ec8d2f17efc6f24fbd29e745f9da4fff5531ea49/cbuildbot/chromeos_config.py [modify] https://crrev.com/ec8d2f17efc6f24fbd29e745f9da4fff5531ea49/cbuildbot/builders/test_builders.py
,
Aug 12 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/c06e75eb79962d6a925c3a549bc1851f891b4fb7 commit c06e75eb79962d6a925c3a549bc1851f891b4fb7 Author: Po-Hsien Wang <pwang@chromium.org> Date: Sat Aug 12 03:10:21 2017 Add vmtest-informational suite Add vmtest-informational suite to autotest BUG= chromium:750374 TEST=site_utils/gs_offloader_unittest.py Change-Id: I846b2b57e197bceff5c0a78f6df509cc4c1aaac9 Reviewed-on: https://chromium-review.googlesource.com/602511 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Stéphane Marchesin <marcheu@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> [modify] https://crrev.com/c06e75eb79962d6a925c3a549bc1851f891b4fb7/site_utils/attribute_whitelist.txt [add] https://crrev.com/c06e75eb79962d6a925c3a549bc1851f891b4fb7/test_suites/control.vmtest_informational
,
Aug 12 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/f90700a384bffe5aa392432fc8eeec42c98f884e commit f90700a384bffe5aa392432fc8eeec42c98f884e Author: Po-Hsien Wang <pwang@chromium.org> Date: Sat Aug 12 03:10:21 2017 graphics_dEQP: add to suite:vmtest-informational Add graphics_dEQP.bvt-vmtest to test if betty-vmtest-informational buildbot can run it successful. BUG= chromium:750374 TEST=cbuildbot Change-Id: I2e938f4159f068e145aecfd98a8fef15915afcf2 Reviewed-on: https://chromium-review.googlesource.com/602512 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Stéphane Marchesin <marcheu@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> [add] https://crrev.com/f90700a384bffe5aa392432fc8eeec42c98f884e/client/site_tests/graphics_dEQP/control.bvt-informational [modify] https://crrev.com/f90700a384bffe5aa392432fc8eeec42c98f884e/client/site_tests/graphics_dEQP/generate_controlfiles.py
,
Aug 14 2017
betty-vmtest-informational should be up and running suite:vmtest-informational right now. However, all graphics tests failed since we refuse to proceed while GPU hang check. As we are in VM(software rastering), I'm wondering if the check is necessary? or should we find a way to switch betty VM to use hw rastering instead?
,
Aug 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/5c7a025a12e3620fa9c0b0a5d777ea87594ce882 commit 5c7a025a12e3620fa9c0b0a5d777ea87594ce882 Author: Po-Hsien Wang <pwang@chromium.org> Date: Tue Aug 15 10:24:16 2017 graphics_utils: Disable GraphicsStateCheck in VM 1) As betty VM virtualize Cirrus Logic gpu, adding corresponding field in GraphicsKernelMemory so that it will not failed with unsupported architecture. 2) Adding option to GraphicsStateChecker so that it will even with software rasterizer. BUG= chromium:750374 TEST=build betty vm, run utils.is_virtual_machine TEST=test_that betty_vm graphics_dEQP.bvt-informational (tests are running, but all tests got segmentation fault.) Change-Id: I2e1257b3c0afb4d313dc46ca15ef40c4a55a070b Reviewed-on: https://chromium-review.googlesource.com/614058 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/5c7a025a12e3620fa9c0b0a5d777ea87594ce882/client/cros/graphics/graphics_utils.py
,
Aug 16 2017
Don@ The buildbot encountered a error showing that the access token is not available to access the google storage after the vmtest. https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/betty-vmtest-informational/builds/6 !!!VMTests failed!!! Logs are uploaded in the corresponding vm_test_results_None. This can be found by clicking on the artifacts link in the "Report" Stage. Specifically look for the test_harness/failed for the failing tests. For more particulars, please refer to which test failed i.e. above see the individual test that failed -- or if an update failed, check the corresponding update directory. [0m 16:16:09: INFO: Uploading artifacts to Google Storage... 16:16:09: INFO: RunCommand: tar --sparse -I /b/c/cbuild/repository/chroot/usr/bin/pigz -cf /b/c/cbuild/repository/trybot_archive/trybot-betty-vmtest-informational/R62-9844.0.0-b6/vm_test_results_None.tgz . in /b/c/cbuild/repository/trybot_archive/trybot-betty-vmtest-informational/R62-9844.0.0-b6/vm_test_results_None 16:16:09: INFO: Waiting up to forever for breakpad symbols ... 16:38:31: INFO: Refreshing due to a 401 (attempt 1/2) Is there a way to fix this?
,
Aug 16 2017
I don't know what command is trying to get the token. (Don't know that much about how we run VMTests). That value of "None" looks suspicious though. On other builders, that's an integer, probably the retry count.
,
Aug 19 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/cfeed9f40f901f58c0c4d3521baf2af62df2f210 commit cfeed9f40f901f58c0c4d3521baf2af62df2f210 Author: Po-Hsien Wang <pwang@chromium.org> Date: Sat Aug 19 06:08:47 2017 test_builders: Add debugStage to VMInformationalBuilder Add DebugSymbolsStage to VMInformationalBuilder. BUG= chromium:750374 TEST=tryjob Change-Id: I45b7cd3b3dc7bb037e7bd378f677b339e3e14ca5 Reviewed-on: https://chromium-review.googlesource.com/619616 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> [modify] https://crrev.com/cfeed9f40f901f58c0c4d3521baf2af62df2f210/cbuildbot/builders/test_builders.py
,
Aug 21 2017
,
Aug 21 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/autotest-cheets/+/50f0b61426444d8b4cbd932d58d0db68d45eb000 commit 50f0b61426444d8b4cbd932d58d0db68d45eb000 Author: Po-Hsien Wang <pwang@chromium.org> Date: Mon Aug 21 19:52:15 2017
,
Aug 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/4d3c6a4b2349183f228155596052c95104039e25 commit 4d3c6a4b2349183f228155596052c95104039e25 Author: Po-Hsien Wang <pwang@chromium.org> Date: Sat Aug 26 04:52:48 2017 add betty-vmtest-informational to chromeos waterfall BUG= chromium:750374 TEST=unittest Change-Id: I2b351fff72dbe5462a4bb3e2a6521f3e0b0303d1 Reviewed-on: https://chromium-review.googlesource.com/636685 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> [modify] https://crrev.com/4d3c6a4b2349183f228155596052c95104039e25/cbuildbot/config_dump.json [modify] https://crrev.com/4d3c6a4b2349183f228155596052c95104039e25/cbuildbot/waterfall_layout_dump.txt [modify] https://crrev.com/4d3c6a4b2349183f228155596052c95104039e25/cbuildbot/chromeos_config.py
,
Sep 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/crostestutils/+/bf7e0ce575953539b91ceda0a5e7d9a4df25ef2a commit bf7e0ce575953539b91ceda0a5e7d9a4df25ef2a Author: Po-Hsien Wang <pwang@chromium.org> Date: Wed Sep 06 06:01:10 2017 vm_au_worker: Output to stdout Background is we like to run long run vmtest for informational purpose. To prevent builder from stop working, we need to continuously output things to stdout. This CL Makes vm_au_worker output test result to stdout directly if verbose is set. BUG= chromium:750374 TEST=run ctest locally TEST=tryjob (Done) Change-Id: I7c0b9a95411602ed094cb95ca711125ff76e0598 Reviewed-on: https://chromium-review.googlesource.com/646946 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Daniel Wang <wonderfly@google.com> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/bf7e0ce575953539b91ceda0a5e7d9a4df25ef2a/au_test_harness/vm_au_worker.py
,
Sep 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/3936efa99c32d1484fe13d676638e672fcbd7506 commit 3936efa99c32d1484fe13d676638e672fcbd7506 Author: Po-Hsien Wang <pwang@chromium.org> Date: Wed Sep 06 18:50:55 2017 infra/config: scheduler betty vmtest info builder As suggest by Don@, the builder scheduler process is migrating. So using the same approach as CL:620067 to schedule our new betty-vmtest-informational builder to run every day at 3PM UTC. BUG= chromium:750374 TEST=None Change-Id: I7ec2a0d424948d8623411fa56867e7304c5fe63b Reviewed-on: https://chromium-review.googlesource.com/636694 Reviewed-by: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> [modify] https://crrev.com/3936efa99c32d1484fe13d676638e672fcbd7506/luci-scheduler.cfg
,
Sep 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/96d2d0d8d82685de60f5d4b091ff744144842b4f commit 96d2d0d8d82685de60f5d4b091ff744144842b4f Author: Po-Hsien Wang <pwang@chromium.org> Date: Wed Sep 06 21:01:07 2017 infra/config: Fix malformed config BUG= chromium:750374 TEST=None Change-Id: I757961c956bdbbdb9cdb0552c6112b87610e769b Reviewed-on: https://chromium-review.googlesource.com/653607 Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> [modify] https://crrev.com/96d2d0d8d82685de60f5d4b091ff744144842b4f/luci-scheduler.cfg
,
Sep 25 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/2063187dc72405a334cf11e6ba630a76baacdeb1 commit 2063187dc72405a334cf11e6ba630a76baacdeb1 Author: Po-Hsien Wang <pwang@chromium.org> Date: Mon Sep 25 22:52:43 2017 Betty-vmtest-informational: extend dEQP timeout extending dEQP timeout from 12h to 24h. BUG= chromium:750374 TEST=unit_test Change-Id: Ie759ec995f1c6c74ced4ec0a3ffc7762a457b6cd Reviewed-on: https://chromium-review.googlesource.com/682342 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/2063187dc72405a334cf11e6ba630a76baacdeb1/cbuildbot/config_dump.json [modify] https://crrev.com/2063187dc72405a334cf11e6ba630a76baacdeb1/cbuildbot/chromeos_config.py
,
Sep 25 2017
Can you make that 23:59? PS: 24 is hard limit for now. There should probably be a unittest to enforce that.
,
Sep 26 2017
Thanks for reminding. I'll decrease it to 23h instead. But maybe the correct way to do so is trying to make the infra treat vmtest the same as the hwtest (submitting to lab and executing it asynchronously).
,
Sep 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/3e0ed5b6793144678823bd10caba2de6ea27bac0 commit 3e0ed5b6793144678823bd10caba2de6ea27bac0 Author: Po-Hsien Wang <pwang@chromium.org> Date: Tue Sep 26 01:06:06 2017 Betty-vmtest-informational: decrease dEQP timeout Decrease the timeout value from 24h to 23h as 24h is the hard limit for the build process. BUG= chromium:750374 TEST=unit_test Change-Id: I868bd211c331a343e94817ff703711d3de777bb2 Reviewed-on: https://chromium-review.googlesource.com/683474 Reviewed-by: Don Garrett <dgarrett@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> [modify] https://crrev.com/3e0ed5b6793144678823bd10caba2de6ea27bac0/cbuildbot/config_dump.json [modify] https://crrev.com/3e0ed5b6793144678823bd10caba2de6ea27bac0/cbuildbot/chromeos_config.py
,
Sep 26 2017
We've debated different ways of doing that for a while, but no real progress yet.
,
Sep 26 2017
It shows 'unknown' clicked the 'stop build' button on the experiment run on chromeos waterfall. https://uberchromegw.corp.google.com/i/chromeos/builders/betty-vmtest-informational/builds/21/steps/steps/logs/interrupt https://luci-milo.appspot.com/buildbot/chromeos/betty-vmtest-informational/21 The experiment run on tryserver got aborted after 20hr run. https://luci-milo.appspot.com/buildbot/chromiumos.tryserver/betty-vmtest-informational/68 I searched the vmtest log and nothing suspicious about what's the reason of the abortion. And I can't find interrupt page as the one on chromeos waterfall. As the abortion is happened exactly in 20hr, @dgarrett, is it possible due to soft-limit or infra restriction?
,
Sep 26 2017
I don't know where a 20 hour timeout would come from. This is the buildbot version of that build page, and it doesn't seem to show either: https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/betty-vmtest-informational/builds/68 Here is the buildbucket request for it: https://apis-explorer.appspot.com/apis-explorer/?base=https://cr-buildbucket.appspot.com/_ah/api#p/buildbucket/v1/buildbucket.get?id=8967431429096769520&_h=1& Also not very helpful. I'm guessing that the timeout came from buildbot, but I don't know where. Here is the configuration for the waterfall in question. https://cs.chromium.org/chromium/build/masters/master.chromiumos.tryserver/master.cfg
,
Sep 29 2017
The new build finished in 19hr 28min, which is close to the 20hr limit with passed=171263, failed=521, notexecuted=2164 deqp tests. https://luci-milo.appspot.com/buildbot/chromeos/betty-vmtest-informational/23
,
Sep 29 2017
Very cool! Notice retries take super long. This is primarily because going through the already passing tests takes 1.5h but also because of the remaining failures causing extra problems. Let's change cheets_CTS_N.py so it does not retry on betty (for now). That way we get some results every 4 hours. 09/29 02:07:16.772 DEBUG| utils:0280| [stdout] Session Pass Fail Modules Complete Result Directory Test Plan Device serial(s) Build ID Product 09/29 02:07:16.772 DEBUG| utils:0280| [stdout] 0 172628 647 0 of 1 2017.09.28_10.39.00 cts 127.0.0.1:9228 R63-9983.0.9999 betty 09/29 02:07:16.772 DEBUG| utils:0280| [stdout] 1 172632 770 0 of 1 2017.09.28_14.35.00 retry.module.CtsDeqpTestCases.0 127.0.0.1:9228 R63-9983.0.9999 betty 09/29 02:07:16.772 DEBUG| utils:0280| [stdout] 2 172634 786 0 of 1 2017.09.28_17.25.27 retry.module.CtsDeqpTestCases.1 127.0.0.1:9228 R63-9983.0.9999 betty 09/29 02:07:16.773 DEBUG| utils:0280| [stdout] 3 172636 806 0 of 1 2017.09.28_19.57.07 retry.module.CtsDeqpTestCases.2 127.0.0.1:9228 R63-9983.0.9999 betty 09/29 02:07:16.773 DEBUG| utils:0280| [stdout] 4 172638 804 0 of 1 2017.09.28_23.23.49 retry.module.CtsDeqpTestCases.3 127.0.0.1:9228 R63-9983.0.9999 betty
,
Oct 2 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/715d5e9f7e5d7632f81d11dcc067733f241f84c9 commit 715d5e9f7e5d7632f81d11dcc067733f241f84c9 Author: Po-Hsien Wang <pwang@chromium.org> Date: Mon Oct 02 10:03:46 2017 cheets_CTS_N: disable max_retry on betty 1) Move retry relevant function to tradefed so that GTS can use it as well. 2) Turn off max_retry on betty to save time for the vmtest. BUG= chromium:750374 TEST=test_that Change-Id: Id01769eb6e9340cc6d95a59eec7319bff713d433 Reviewed-on: https://chromium-review.googlesource.com/693351 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/715d5e9f7e5d7632f81d11dcc067733f241f84c9/server/cros/tradefed_test.py [modify] https://crrev.com/715d5e9f7e5d7632f81d11dcc067733f241f84c9/server/site_tests/cheets_CTS_N/cheets_CTS_N.py
,
Oct 3 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/225ad8b8876bff93d141c764f6b618763d323edd commit 225ad8b8876bff93d141c764f6b618763d323edd Author: Kazuhiro Inaba <kinaba@chromium.org> Date: Tue Oct 03 06:45:07 2017 Fix for "cheets_CTS_N: disable max_retry on betty". This fix the error "Unhandled TypeError: _get_release_channel() takes exactly 2 arguments (1 given)". BUG= chromium:750374 BUG=b:67331384 TEST=test_that cheets_CTS_N.CtsUsbTests Change-Id: Ibccf5894a22fb918514cd1fce7be3c96f69b5ca6 Reviewed-on: https://chromium-review.googlesource.com/696745 Tested-by: Kazuhiro Inaba <kinaba@chromium.org> Reviewed-by: Po-Hsien Wang <pwang@chromium.org> Commit-Queue: Kazuhiro Inaba <kinaba@chromium.org> Trybot-Ready: Kazuhiro Inaba <kinaba@chromium.org> [modify] https://crrev.com/225ad8b8876bff93d141c764f6b618763d323edd/server/cros/tradefed_test.py
,
Oct 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78 commit 46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78 Author: Po-Hsien Wang <pwang@chromium.org> Date: Fri Oct 06 22:51:13 2017 vm_test_stages: Upload test result to dashboard 1) Add an option to enable test result to be upload to dashboard. 2) Helper function for CTS/GTS upload. BUG= chromium:750374 TEST=unittest Change-Id: I1dd3d9e0b25ddbff2fbc189b1605e4e83063a796 Reviewed-on: https://chromium-review.googlesource.com/627630 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78/cbuildbot/chromeos_config.py [modify] https://crrev.com/46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78/cbuildbot/stages/vm_test_stages_unittest.py [add] https://crrev.com/46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78/lib/cts_helper_unittest [modify] https://crrev.com/46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78/lib/config_lib.py [add] https://crrev.com/46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78/lib/cts_helper_unittest.py [modify] https://crrev.com/46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78/cbuildbot/config_dump.json [modify] https://crrev.com/46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78/cbuildbot/stages/vm_test_stages.py [add] https://crrev.com/46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78/lib/cts_helper.py [modify] https://crrev.com/46d601c0f2fcd0f1b619ebd9e8a7b7b920ec0a78/lib/constants.py
,
Oct 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/c6dfd90f7edb8ddb4dd546b1fe8056c471855dbf commit c6dfd90f7edb8ddb4dd546b1fe8056c471855dbf Author: Po-Hsien Wang <pwang@chromium.org> Date: Sat Oct 07 11:54:29 2017 vm_test_stages: use abs path for ReportCtsResults Use absolute path for _ReportCtsResults instead of relative path. BUG= chromium:750374 TEST=tryjob Change-Id: I8b3a2c3c1ac90d5b1cfd50431743d32c84d921cf Reviewed-on: https://chromium-review.googlesource.com/706535 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/c6dfd90f7edb8ddb4dd546b1fe8056c471855dbf/cbuildbot/stages/vm_test_stages.py [modify] https://crrev.com/c6dfd90f7edb8ddb4dd546b1fe8056c471855dbf/lib/cts_helper.py
,
Oct 10 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/autotest-cheets/+/fc5afa88590254cb52950908dd0016ecd068c52a commit fc5afa88590254cb52950908dd0016ecd068c52a Author: Po-Hsien Wang <pwang@chromium.org> Date: Tue Oct 10 03:07:57 2017
,
Oct 12 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/autotest-cheets/+/d4e001c4246678b0a92e8a358911873613b42d37 commit d4e001c4246678b0a92e8a358911873613b42d37 Author: Po-Hsien Wang <pwang@chromium.org> Date: Thu Oct 12 02:20:57 2017
,
Oct 15 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/autotest-cheets/+/6dd875c37967fb2b4881a1f1ed982be4f8a7531f commit 6dd875c37967fb2b4881a1f1ed982be4f8a7531f Author: Po-Hsien Wang <pwang@chromium.org> Date: Sat Oct 14 06:15:30 2017
,
Oct 15 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/autotest-cheets/+/695e041c8d8d6bc9ee93d4827450136332ebaf17 commit 695e041c8d8d6bc9ee93d4827450136332ebaf17 Author: Po-Hsien Wang <pwang@chromium.org> Date: Sat Oct 14 06:18:16 2017
,
Oct 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/11eb830852fe9d0feff88fd2544ca627ab6c6566 commit 11eb830852fe9d0feff88fd2544ca627ab6c6566 Author: Po-Hsien Wang <pwang@chromium.org> Date: Sat Oct 14 06:05:57 2017 cheets_CTS_N: disable max_retry on betty 1) Move retry relevant function to tradefed so that GTS can use it as well. 2) Turn off max_retry on betty to save time for the vmtest. BUG= chromium:750374 TEST=test_that Change-Id: Id01769eb6e9340cc6d95a59eec7319bff713d433 Reviewed-on: https://chromium-review.googlesource.com/693351 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Po-Hsien Wang <pwang@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> (cherry picked from commit 715d5e9f7e5d7632f81d11dcc067733f241f84c9) Reviewed-on: https://chromium-review.googlesource.com/720153 Tested-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/11eb830852fe9d0feff88fd2544ca627ab6c6566/server/cros/tradefed_test.py [modify] https://crrev.com/11eb830852fe9d0feff88fd2544ca627ab6c6566/server/site_tests/cheets_CTS_N/cheets_CTS_N.py
,
Oct 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/641a3845f7b05d6857bfab931f954597ce2464be commit 641a3845f7b05d6857bfab931f954597ce2464be Author: Kazuhiro Inaba <kinaba@chromium.org> Date: Sat Oct 14 06:06:39 2017 Fix for "cheets_CTS_N: disable max_retry on betty". This fix the error "Unhandled TypeError: _get_release_channel() takes exactly 2 arguments (1 given)". BUG= chromium:750374 BUG=b:67331384 TEST=test_that cheets_CTS_N.CtsUsbTests Change-Id: Ibccf5894a22fb918514cd1fce7be3c96f69b5ca6 Reviewed-on: https://chromium-review.googlesource.com/696745 Tested-by: Kazuhiro Inaba <kinaba@chromium.org> Reviewed-by: Po-Hsien Wang <pwang@chromium.org> Commit-Queue: Kazuhiro Inaba <kinaba@chromium.org> Trybot-Ready: Kazuhiro Inaba <kinaba@chromium.org> (cherry picked from commit 225ad8b8876bff93d141c764f6b618763d323edd) Reviewed-on: https://chromium-review.googlesource.com/719888 Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/641a3845f7b05d6857bfab931f954597ce2464be/server/cros/tradefed_test.py
,
Oct 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/4618adfa60b4fbbf2e1442fc1b6e0ee1896ccd75 commit 4618adfa60b4fbbf2e1442fc1b6e0ee1896ccd75 Author: Po-Hsien Wang <pwang@chromium.org> Date: Thu Oct 26 11:17:53 2017 cheets_CTS_N: modify max_retry logic 1) Change the max_retry logic to use the minimum retry values. 2) Fix get_channel to get the correct channel from DUT. BUG= chromium:750374 TEST=test locally test_that dut cheets_CTS_N.all Change-Id: I9a301c5596dbf6da15dc2a495e59c59a5d630d7c Reviewed-on: https://chromium-review.googlesource.com/710615 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/4618adfa60b4fbbf2e1442fc1b6e0ee1896ccd75/server/hosts/cros_host.py [modify] https://crrev.com/4618adfa60b4fbbf2e1442fc1b6e0ee1896ccd75/server/cros/tradefed_test.py [modify] https://crrev.com/4618adfa60b4fbbf2e1442fc1b6e0ee1896ccd75/server/site_tests/cheets_CTS_N/cheets_CTS_N.py [modify] https://crrev.com/4618adfa60b4fbbf2e1442fc1b6e0ee1896ccd75/client/common_lib/lsbrelease_utils.py
,
Nov 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/be15c4c2caa3e335e5cda8b0c1d7136630b0c083 commit be15c4c2caa3e335e5cda8b0c1d7136630b0c083 Author: Po-Hsien Wang <pwang@chromium.org> Date: Wed Nov 15 02:46:35 2017 cheets_CTS_N: modify max_retry logic 1) Change the max_retry logic to use the minimum retry values. 2) Fix get_channel to get the correct channel from DUT. BUG= chromium:750374 TEST=test locally test_that dut cheets_CTS_N.all Change-Id: I9a301c5596dbf6da15dc2a495e59c59a5d630d7c Reviewed-on: https://chromium-review.googlesource.com/710615 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/770516 [modify] https://crrev.com/be15c4c2caa3e335e5cda8b0c1d7136630b0c083/server/hosts/cros_host.py [modify] https://crrev.com/be15c4c2caa3e335e5cda8b0c1d7136630b0c083/server/cros/tradefed_test.py [modify] https://crrev.com/be15c4c2caa3e335e5cda8b0c1d7136630b0c083/server/site_tests/cheets_CTS_N/cheets_CTS_N.py [modify] https://crrev.com/be15c4c2caa3e335e5cda8b0c1d7136630b0c083/client/common_lib/lsbrelease_utils.py
,
Nov 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/9c4797f073ee9393de5f5fc935341a1665e8088d commit 9c4797f073ee9393de5f5fc935341a1665e8088d Author: Po-Hsien Wang <pwang@chromium.org> Date: Wed Nov 15 04:47:03 2017 cheets_CTS_N: modify max_retry logic 1) Change the max_retry logic to use the minimum retry values. 2) Fix get_channel to get the correct channel from DUT. BUG= chromium:750374 TEST=test locally test_that dut cheets_CTS_N.all Change-Id: I9b91ac8fa092f18a565d9308706841cf0fe8b0ee Reviewed-on: https://chromium-review.googlesource.com/770831 Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/9c4797f073ee9393de5f5fc935341a1665e8088d/server/hosts/cros_host.py [modify] https://crrev.com/9c4797f073ee9393de5f5fc935341a1665e8088d/server/cros/tradefed_test.py [modify] https://crrev.com/9c4797f073ee9393de5f5fc935341a1665e8088d/server/site_tests/cheets_CTS_N/cheets_CTS_N.py [modify] https://crrev.com/9c4797f073ee9393de5f5fc935341a1665e8088d/client/common_lib/lsbrelease_utils.py
,
Jan 9 2018
Whole CTS is running on betty. Other suite should be executed in parallel as CTS is consuming 16 of 20 hr on the builder.
,
Jan 9 2018
For now see b/71727384 before running two VMS in parallel. Also notice that we run the VM with 4 cores only. We should probably bump the number up ( issue 800554 ).
,
Jan 9 2018
|
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 by dgarr...@chromium.org
, Jul 28 2017