New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 718339 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: May 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Implement retry mechanism for gtest-parallel

Project Member Reported by ehmaldonado@chromium.org, May 4 2017

Issue description

Implement retry mechanism for gtest-parallel to deal with flaky tests.
 
Project Member

Comment 1 by bugdroid1@chromium.org, May 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/5aa3f361299ab60d6ed6e8d7bd469fef8dba3b6b

commit 5aa3f361299ab60d6ed6e8d7bd469fef8dba3b6b
Author: ehmaldonado <ehmaldonado@webrtc.org>
Date: Thu May 04 08:02:28 2017

Roll gtest-parallel c9bb1c9e8a..1e816d0426

* src/third_party/gtest-parallel: https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+log/c9bb1c9e8a..1e816d0426

BUG= chromium:718339 
NOTRY=True

Review-Url: https://codereview.webrtc.org/2859133002
Cr-Commit-Position: refs/heads/master@{#18005}

[modify] https://crrev.com/5aa3f361299ab60d6ed6e8d7bd469fef8dba3b6b/DEPS

Project Member

Comment 2 by bugdroid1@chromium.org, May 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/8417610a79b4097f0fe4b34b610aea65d20377ae

commit 8417610a79b4097f0fe4b34b610aea65d20377ae
Author: ehmaldonado <ehmaldonado@webrtc.org>
Date: Thu May 04 08:15:02 2017

Revert of Roll gtest-parallel. (patchset #1 id:1 of https://codereview.webrtc.org/2859133002/ )

Reason for revert:
There might be a subtle bug where all tests are always reported as passing.

Original issue's description:
> Roll gtest-parallel c9bb1c9e8a..1e816d0426
>
> * src/third_party/gtest-parallel: https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+log/c9bb1c9e8a..1e816d0426
>
> BUG= chromium:718339 
> NOTRY=True
>
> Review-Url: https://codereview.webrtc.org/2859133002
> Cr-Commit-Position: refs/heads/master@{#18005}
> Committed: https://chromium.googlesource.com/external/webrtc/+/5aa3f361299ab60d6ed6e8d7bd469fef8dba3b6b

TBR=kjellander@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= chromium:718339 

Review-Url: https://codereview.webrtc.org/2856963004
Cr-Commit-Position: refs/heads/master@{#18007}

[modify] https://crrev.com/8417610a79b4097f0fe4b34b610aea65d20377ae/DEPS

Project Member

Comment 3 by bugdroid1@chromium.org, May 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/c20c379b0edd188055732b0427f5b70757311125

commit c20c379b0edd188055732b0427f5b70757311125
Author: ehmaldonado <ehmaldonado@webrtc.org>
Date: Thu May 04 20:16:53 2017

Roll gtest-parallel c9bb1c9e8a..c3e4b0a6d1

* src/third_party/gtest-parallel: https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+log/c9bb1c9e8a..c3e4b0a6d1

BUG= chromium:718339 

Review-Url: https://codereview.webrtc.org/2861943003
Cr-Commit-Position: refs/heads/master@{#18020}

[modify] https://crrev.com/c20c379b0edd188055732b0427f5b70757311125/DEPS

Project Member

Comment 4 by bugdroid1@chromium.org, May 5 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/2a28035627ab93463005d9549b2f47db6600aa69

commit 2a28035627ab93463005d9549b2f47db6600aa69
Author: ehmaldonado <ehmaldonado@webrtc.org>
Date: Fri May 05 11:33:57 2017

Add --retry_failed=3 flag to gtest-parallel.

This means the test will be executed 4 times max, one run and three retries.

BUG= chromium:718339 

Review-Url: https://codereview.webrtc.org/2861043002
Cr-Commit-Position: refs/heads/master@{#18028}

[modify] https://crrev.com/2a28035627ab93463005d9549b2f47db6600aa69/tools-webrtc/mb/mb.py
[modify] https://crrev.com/2a28035627ab93463005d9549b2f47db6600aa69/tools-webrtc/mb/mb_unittest.py

Shall we mark this as fixed now?
I wonder if we should make our non-parallel tests use the same script with --workers=1 in order to benefit from the retry mechanism? Or should we refactor the script into another name before that? I already did such a change for webrtc_nonparallel_tests in https://codereview.webrtc.org/2806373002, but we have a few more tests, like the video_capture_tests, voe_auto_test etc.
Status: Fixed (was: Assigned)
I'll mark this as fixed.
Can you please open a bug to use gtest-parallel with --workers=1 for the other non-parallel tests?
I vote we keep the name as is.
Yes, I filed  bug 719879  for #7.

Sign in to add a comment