eCTS: Develop script to automate test run |
||
Issue descriptionThis is an umbrella bug for a test automation script. It's currently cts/cts.py.
,
Jun 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/ec/+/3e4d3fd71249d9511ff8b61b44dbc358191cb90f commit 3e4d3fd71249d9511ff8b61b44dbc358191cb90f Author: Daisuke Nojiri <dnojiri@chromium.org> Date: Sat Jun 17 04:10:54 2017 eCTS: Hide expected messages This patch hides expected messages from output to the terminal and reorganizes log directory as follows: - All test output goes under /tmp/ects/$dut/$module - Openocd output is recorded in openocd.log - uart outputs are recorded in uart_th.log and uart_dut.log - build output is recorded in build.log - Check exit code from all subprocess calls - Dump build log if build fails - Dump openocd log if openocd fails BUG= chromium:664309 BRANCH=none TEST=cts.py --setup and build images in chroot then run cts.py -m meta outside chroot. Change-Id: I13294c3bf777ad7ae590459d3cf5aea405d59f96 Signed-off-by: Daisuke Nojiri <dnojiri@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/409536 Reviewed-by: Randall Spangler <rspangler@chromium.org> [modify] https://crrev.com/3e4d3fd71249d9511ff8b61b44dbc358191cb90f/cts/cts.py [modify] https://crrev.com/3e4d3fd71249d9511ff8b61b44dbc358191cb90f/cts/common/board.py
,
Jun 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/ec/+/3bb2d756e579be92a87ff6a17f26f78af3350e4b commit 3bb2d756e579be92a87ff6a17f26f78af3350e4b Author: Daisuke Nojiri <dnojiri@chromium.org> Date: Sat Jun 17 04:10:54 2017 eCTS: Limit tty reads by boot counts Currently, read_tty reads characters from tty as long as there is something to read. This causes read_tty to loop forever if the board is in reboot loop. This patch makes cts.py stop reading tty if boot count exceeds max_boot_count. Reboot is detected by detecting REBOOT_MARKER. This patch also does: - Remove debug option: This adds complexity for no real value. Developers should debug tests using regular tools (make, uart console, etc.). - Remove html output. Nobody use it. Should be redone when it's needed using proper libraries. BUG= chromium:664309 BRANCH=none TEST=cts.py -m task/gpio/interrupt Change-Id: I51d1dd51c4097e8115ef04ad46853720295141b4 Signed-off-by: Daisuke Nojiri <dnojiri@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/410281 Reviewed-by: Randall Spangler <rspangler@chromium.org> [modify] https://crrev.com/3bb2d756e579be92a87ff6a17f26f78af3350e4b/cts/cts.py [modify] https://crrev.com/3bb2d756e579be92a87ff6a17f26f78af3350e4b/cts/common/board.py
,
Jun 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/ec/+/0696dd1e68692ca8dae17b737e0d895561f65cd4 commit 0696dd1e68692ca8dae17b737e0d895561f65cd4 Author: Daisuke Nojiri <dnojiri@chromium.org> Date: Sat Jun 17 08:12:42 2017 eCTS: Refactor return code handling This patch adds host only return codes to cts.rc and derive names from there. BUG= chromium:664309 BRANCH=none TEST=cts.py -m task, interrupt, gpio test name TH DUT debug_test SUCCESS SUCCESS success_test SUCCESS SUCCESS fail_dut_test SUCCESS FAILURE fail_th_test FAILURE SUCCESS fail_both_test FAILURE FAILURE bad_sync_and_success_test BAD_SYNC BAD_SYNC bad_sync_both_test BAD_SYNC BAD_SYNC bad_sync_failure_test FAILURE BAD_SYNC hang_test SUCCESS NO_RESULT post_corruption_success NO_RESULT NO_RESULT Change-Id: I169b2466646d6236571a8a4c5d3e208d928b9dd2 Signed-off-by: Daisuke Nojiri <dnojiri@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/410282 Reviewed-by: Randall Spangler <rspangler@chromium.org> [modify] https://crrev.com/0696dd1e68692ca8dae17b737e0d895561f65cd4/cts/cts.py [modify] https://crrev.com/0696dd1e68692ca8dae17b737e0d895561f65cd4/cts/common/cts.rc
,
Jun 20 2017
Closing bug. cts/cts.py can build & flash images and can run tests. Further updates on the script will have a separate bug.
,
Jun 28 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/ec/+/52eb3b3b6404b2c2b99715124e5ea9af307db8bf commit 52eb3b3b6404b2c2b99715124e5ea9af307db8bf Author: Daisuke Nojiri <dnojiri@chromium.org> Date: Wed Jun 28 07:59:06 2017 eCTS: Fix alignment of test results This patch fixes the alignment of test results. It also displays 'YES' or 'NO' instead of '1' or '0' for expected string columns: TEST NAME TH_RC DUT_RC TH_STR DUT_STR RESULT success_test SUCCESS SUCCESS YES YES PASS fail_dut_test SUCCESS FAILURE YES YES PASS fail_th_test FAILURE SUCCESS YES YES PASS fail_both_test FAILURE FAILURE YES YES PASS bad_sync_test BAD_SYNC SUCCESS YES YES PASS bad_sync_both_test BAD_SYNC BAD_SYNC YES YES PASS hang_test SUCCESS DID_NOT_END YES YES PASS did_not_start_test DID_NOT_END DID_NOT_START YES YES PASS BUG= chromium:664309 BRANCH=none TEST=Run cts.py -m meta Change-Id: I7d266b9a6458fcabafa4da484b54d9d1dffbad61 Signed-off-by: Daisuke Nojiri <dnojiri@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/545257 Reviewed-by: Randall Spangler <rspangler@chromium.org> [modify] https://crrev.com/52eb3b3b6404b2c2b99715124e5ea9af307db8bf/cts/meta/cts.testlist [modify] https://crrev.com/52eb3b3b6404b2c2b99715124e5ea9af307db8bf/cts/cts.py [modify] https://crrev.com/52eb3b3b6404b2c2b99715124e5ea9af307db8bf/cts/meta/dut.c [modify] https://crrev.com/52eb3b3b6404b2c2b99715124e5ea9af307db8bf/cts/meta/th.c
,
Jul 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/ec/+/b20a873d0f58809b77dd1c4d40706f0441e798e3 commit b20a873d0f58809b77dd1c4d40706f0441e798e3 Author: Daisuke Nojiri <dnojiri@chromium.org> Date: Sun Jul 09 03:38:53 2017 eCTS: Run openocd and lsusb as a root Running lsusb (to get iSerial) and openocd requires a privilege. This patch makes cts.py run these commands as a root. Opening ttyACM0 requires a privilege as well but it'll be done by asking a user to add themselves to dialout group. BUG= chromium:664309 BRANCH=none TEST=Run cts.py on a freshly set up cros_sdk chroot. Change-Id: Ib9d007459c4b1322b15f521a5253d833e2304c3c Signed-off-by: Daisuke Nojiri <dnojiri@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/558804 Reviewed-by: Randall Spangler <rspangler@chromium.org> [modify] https://crrev.com/b20a873d0f58809b77dd1c4d40706f0441e798e3/cts/common/board.py |
||
►
Sign in to add a comment |
||
Comment 1 by dnojiri@chromium.org
, Apr 4 2017