dashboard python tests very flaky |
|||||
Issue description
In particular, it's update_test_suites_test.ListTestSuitesTest.testPartialTestSuites
Log:
[1191/1195] update_test_suites_test.ListTestSuitesTest.testPartialTestSuites failed unexpectedly:
Traceback (most recent call last):
File "/b/build/slave/Catapult_Mac_Tryserver/build/catapult/dashboard/dashboard/update_test_suites_test.py", line 339, in testPartialTestSuites
update_test_suites.FetchCachedTestSuites2())
AssertionError: Lists differ: ['TEST_PARTIAL_TEST_SUITE:COMP... != ['TEST_PARTIAL_TEST_SUITE']
First differing element 0:
TEST_PARTIAL_TEST_SUITE:COMPOSITE
TEST_PARTIAL_TEST_SUITE
- ['TEST_PARTIAL_TEST_SUITE:COMPOSITE']
? ----------
+ ['TEST_PARTIAL_TEST_SUITE']
,
Jul 4
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/7b9299aedd379607a988dd3e62889fe0ae6cfc18 commit 7b9299aedd379607a988dd3e62889fe0ae6cfc18 Author: Juan Antonio Navarro Perez <perezju@chromium.org> Date: Wed Jul 04 12:24:19 2018 [dashboard] Skip flaky ListTestSuitesTest.testPartialTestSuites TBR=simonhatch@chromium.org,benjhayden@chromium.org Bug: chromium:860103 Change-Id: I7a7f7092ab7a616fecf728b47fa82736876ccf6e Reviewed-on: https://chromium-review.googlesource.com/1125920 Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org> [modify] https://crrev.com/7b9299aedd379607a988dd3e62889fe0ae6cfc18/dashboard/dashboard/update_test_suites_test.py
,
Jul 4
Pretty sure resetting the descriptors properly between test runs will fix this. Passing to Ben since he already has a cl to do that. Dropping to p2 since this is disabled.
,
Jul 4
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/905d3efd2b5b204b0390fcb370226e8d21fa8978 commit 905d3efd2b5b204b0390fcb370226e8d21fa8978 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Jul 04 16:03:03 2018 Roll src/third_party/catapult 5abd99f5f768..82213060d5ce (2 commits) https://chromium.googlesource.com/catapult.git/+log/5abd99f5f768..82213060d5ce git log 5abd99f5f768..82213060d5ce --date=short --no-merges --format='%ad %ae %s' 2018-07-04 perezju@chromium.org [Telemetry] Add flush-tracing and sync markers on first tab only 2018-07-04 perezju@chromium.org [dashboard] Skip flaky ListTestSuitesTest.testPartialTestSuites Created with: gclient setdep -r src/third_party/catapult@82213060d5ce The AutoRoll server is located here: https://catapult-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:854212 , chromium:860103 TBR=sullivan@chromium.org Change-Id: I964cb29eb623c77634a2b66357acdf4d71f852e2 Reviewed-on: https://chromium-review.googlesource.com/1126281 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#572594} [modify] https://crrev.com/905d3efd2b5b204b0390fcb370226e8d21fa8978/DEPS
,
Jul 5
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/5715e9860db3f501d6d0f63a54623de5d621ed64 commit 5715e9860db3f501d6d0f63a54623de5d621ed64 Author: benshayden <benjhayden@chromium.org> Date: Thu Jul 05 18:34:02 2018 Reset memoized Descriptor configuration between tests. Bug: 860103 Change-Id: I17c77c8478fffb50c0b3d93ed056f0059bf0f505 Reviewed-on: https://chromium-review.googlesource.com/1125130 Reviewed-by: Simon Hatch <simonhatch@chromium.org> Commit-Queue: Ben Hayden <benjhayden@chromium.org> [modify] https://crrev.com/5715e9860db3f501d6d0f63a54623de5d621ed64/dashboard/dashboard/common/descriptor.py [modify] https://crrev.com/5715e9860db3f501d6d0f63a54623de5d621ed64/dashboard/dashboard/update_test_suites_test.py [modify] https://crrev.com/5715e9860db3f501d6d0f63a54623de5d621ed64/dashboard/dashboard/common/descriptor_test.py
,
Jul 5
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/920bdc6101e9712c4e51ec36e98d316c58e72b87 commit 920bdc6101e9712c4e51ec36e98d316c58e72b87 Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Thu Jul 05 20:35:22 2018 Roll src/third_party/catapult 8355b184a56e..5715e9860db3 (1 commits) https://chromium.googlesource.com/catapult.git/+log/8355b184a56e..5715e9860db3 git log 8355b184a56e..5715e9860db3 --date=short --no-merges --format='%ad %ae %s' 2018-07-05 benjhayden@chromium.org Reset memoized Descriptor configuration between tests. Created with: gclient setdep -r src/third_party/catapult@5715e9860db3 The AutoRoll server is located here: https://catapult-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:860103 TBR=sullivan@chromium.org Change-Id: If29a814d4307912bdde59fd258bcc13a45508658 Reviewed-on: https://chromium-review.googlesource.com/1127320 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#572850} [modify] https://crrev.com/920bdc6101e9712c4e51ec36e98d316c58e72b87/DEPS
,
Jul 11
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/5c0d851abe6e90d31a70a59b4443632d3869dedd commit 5c0d851abe6e90d31a70a59b4443632d3869dedd Author: Simon <simonhatch@chromium.org> Date: Wed Jul 11 17:29:21 2018 Dashboard - Re-enable ListTestSuitesTest.testPartialTestSuites This was flaky, but that should be fixed. Will run this through CQ a couple times. Bug: chromium:860103 Change-Id: Idefdde1f1ab28a21b4a9881bd7e8b002664a5342 Reviewed-on: https://chromium-review.googlesource.com/1133322 Reviewed-by: Ben Hayden <benjhayden@chromium.org> Commit-Queue: Simon Hatch <simonhatch@chromium.org> [modify] https://crrev.com/5c0d851abe6e90d31a70a59b4443632d3869dedd/dashboard/dashboard/update_test_suites_test.py
,
Jul 11
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c5242ed0373704ac218c2d1af0c2af4b34def3ec commit c5242ed0373704ac218c2d1af0c2af4b34def3ec Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Jul 11 21:14:35 2018 Roll src/third_party/catapult 4c70b5037475..5c0d851abe6e (3 commits) https://chromium.googlesource.com/catapult.git/+log/4c70b5037475..5c0d851abe6e git log 4c70b5037475..5c0d851abe6e --date=short --no-merges --format='%ad %ae %s' 2018-07-11 simonhatch@chromium.org Dashboard - Re-enable ListTestSuitesTest.testPartialTestSuites 2018-07-11 dtu@chromium.org [pinpoint] Add --also-run-disabled-tests flag to Telemetry runs. 2018-07-11 dtu@chromium.org [pinpoint] Use new Change stub in the Change test. Created with: gclient setdep -r src/third_party/catapult@5c0d851abe6e The AutoRoll server is located here: https://catapult-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG= chromium:860103 TBR=sullivan@chromium.org Change-Id: I962dc9fa532702aa881636a36062de73b67c4d2d Reviewed-on: https://chromium-review.googlesource.com/1133778 Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#574338} [modify] https://crrev.com/c5242ed0373704ac218c2d1af0c2af4b34def3ec/DEPS
,
Nov 12
,
Jan 16
(6 days ago)
,
Jan 16
(6 days ago)
Seems like this was just not closed. |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by perezju@chromium.org
, Jul 4