New issue
Advanced search Search tips

Issue 860103 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jan 16
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

dashboard python tests very flaky

Project Member Reported by nednguyen@chromium.org, Jul 4

Issue description

In particular, it's update_test_suites_test.ListTestSuitesTest.testPartialTestSuites

Log:
[1191/1195] update_test_suites_test.ListTestSuitesTest.testPartialTestSuites failed unexpectedly:
  Traceback (most recent call last):
    File "/b/build/slave/Catapult_Mac_Tryserver/build/catapult/dashboard/dashboard/update_test_suites_test.py", line 339, in testPartialTestSuites
      update_test_suites.FetchCachedTestSuites2())
  AssertionError: Lists differ: ['TEST_PARTIAL_TEST_SUITE:COMP... != ['TEST_PARTIAL_TEST_SUITE']
  
  First differing element 0:
  TEST_PARTIAL_TEST_SUITE:COMPOSITE
  TEST_PARTIAL_TEST_SUITE
  
  - ['TEST_PARTIAL_TEST_SUITE:COMPOSITE']
  ?                          ----------
  
  + ['TEST_PARTIAL_TEST_SUITE']


 
Project Member

Comment 2 by bugdroid1@chromium.org, Jul 4

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/7b9299aedd379607a988dd3e62889fe0ae6cfc18

commit 7b9299aedd379607a988dd3e62889fe0ae6cfc18
Author: Juan Antonio Navarro Perez <perezju@chromium.org>
Date: Wed Jul 04 12:24:19 2018

[dashboard] Skip flaky ListTestSuitesTest.testPartialTestSuites

TBR=simonhatch@chromium.org,benjhayden@chromium.org

Bug:  chromium:860103 
Change-Id: I7a7f7092ab7a616fecf728b47fa82736876ccf6e
Reviewed-on: https://chromium-review.googlesource.com/1125920
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org>

[modify] https://crrev.com/7b9299aedd379607a988dd3e62889fe0ae6cfc18/dashboard/dashboard/update_test_suites_test.py

Labels: -Pri-1 Pri-2
Owner: benjhayden@chromium.org
Pretty sure resetting the descriptors properly between test runs will fix this. Passing to Ben since he already has a cl to do that. Dropping to p2 since this is disabled.
Project Member

Comment 4 by bugdroid1@chromium.org, Jul 4

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/905d3efd2b5b204b0390fcb370226e8d21fa8978

commit 905d3efd2b5b204b0390fcb370226e8d21fa8978
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Wed Jul 04 16:03:03 2018

Roll src/third_party/catapult 5abd99f5f768..82213060d5ce (2 commits)

https://chromium.googlesource.com/catapult.git/+log/5abd99f5f768..82213060d5ce


git log 5abd99f5f768..82213060d5ce --date=short --no-merges --format='%ad %ae %s'
2018-07-04 perezju@chromium.org [Telemetry] Add flush-tracing and sync markers on first tab only
2018-07-04 perezju@chromium.org [dashboard] Skip flaky ListTestSuitesTest.testPartialTestSuites


Created with:
  gclient setdep -r src/third_party/catapult@82213060d5ce

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:854212 , chromium:860103 
TBR=sullivan@chromium.org

Change-Id: I964cb29eb623c77634a2b66357acdf4d71f852e2
Reviewed-on: https://chromium-review.googlesource.com/1126281
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#572594}
[modify] https://crrev.com/905d3efd2b5b204b0390fcb370226e8d21fa8978/DEPS

Project Member

Comment 6 by bugdroid1@chromium.org, Jul 5

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/920bdc6101e9712c4e51ec36e98d316c58e72b87

commit 920bdc6101e9712c4e51ec36e98d316c58e72b87
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Thu Jul 05 20:35:22 2018

Roll src/third_party/catapult 8355b184a56e..5715e9860db3 (1 commits)

https://chromium.googlesource.com/catapult.git/+log/8355b184a56e..5715e9860db3


git log 8355b184a56e..5715e9860db3 --date=short --no-merges --format='%ad %ae %s'
2018-07-05 benjhayden@chromium.org Reset memoized Descriptor configuration between tests.


Created with:
  gclient setdep -r src/third_party/catapult@5715e9860db3

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:860103 
TBR=sullivan@chromium.org

Change-Id: If29a814d4307912bdde59fd258bcc13a45508658
Reviewed-on: https://chromium-review.googlesource.com/1127320
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#572850}
[modify] https://crrev.com/920bdc6101e9712c4e51ec36e98d316c58e72b87/DEPS

Project Member

Comment 7 by bugdroid1@chromium.org, Jul 11

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/5c0d851abe6e90d31a70a59b4443632d3869dedd

commit 5c0d851abe6e90d31a70a59b4443632d3869dedd
Author: Simon <simonhatch@chromium.org>
Date: Wed Jul 11 17:29:21 2018

Dashboard - Re-enable ListTestSuitesTest.testPartialTestSuites

This was flaky, but that should be fixed. Will run this through CQ a
couple times.

Bug:  chromium:860103 
Change-Id: Idefdde1f1ab28a21b4a9881bd7e8b002664a5342
Reviewed-on: https://chromium-review.googlesource.com/1133322
Reviewed-by: Ben Hayden <benjhayden@chromium.org>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>

[modify] https://crrev.com/5c0d851abe6e90d31a70a59b4443632d3869dedd/dashboard/dashboard/update_test_suites_test.py

Project Member

Comment 8 by bugdroid1@chromium.org, Jul 11

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c5242ed0373704ac218c2d1af0c2af4b34def3ec

commit c5242ed0373704ac218c2d1af0c2af4b34def3ec
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Wed Jul 11 21:14:35 2018

Roll src/third_party/catapult 4c70b5037475..5c0d851abe6e (3 commits)

https://chromium.googlesource.com/catapult.git/+log/4c70b5037475..5c0d851abe6e


git log 4c70b5037475..5c0d851abe6e --date=short --no-merges --format='%ad %ae %s'
2018-07-11 simonhatch@chromium.org Dashboard - Re-enable ListTestSuitesTest.testPartialTestSuites
2018-07-11 dtu@chromium.org [pinpoint] Add --also-run-disabled-tests flag to Telemetry runs.
2018-07-11 dtu@chromium.org [pinpoint] Use new Change stub in the Change test.


Created with:
  gclient setdep -r src/third_party/catapult@5c0d851abe6e

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:860103 
TBR=sullivan@chromium.org

Change-Id: I962dc9fa532702aa881636a36062de73b67c4d2d
Reviewed-on: https://chromium-review.googlesource.com/1133778
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#574338}
[modify] https://crrev.com/c5242ed0373704ac218c2d1af0c2af4b34def3ec/DEPS

Cc: -eakuefner@chromium.org

Comment 10 by seanmccullough@chromium.org, Jan 16 (6 days ago)

Owner: simonhatch@chromium.org

Comment 11 by simonhatch@chromium.org, Jan 16 (6 days ago)

Status: Fixed (was: Assigned)
Seems like this was just not closed.

Sign in to add a comment