"isolate" is flaky |
||||||
Issue description"isolate" is flaky. This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label. We have detected 3 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyEgsSBUZsYWtlIgdpc29sYXRlDA. This flaky test/step was previously tracked in issue 669391 .
,
Dec 6 2016
,
Dec 6 2016
Dave/Michael, want to dig deeper? That's related to the isolate go client.
,
Dec 6 2016
That looks like the GCS service returning the 500s. That's definitely not surprising (it's known to be flaky on uploads), but my reading of the isolate cmd is that it will retry on 500 errors. I'll have to look deeper.
,
Dec 6 2016
My only concern is that I thought the automatic retry should have kicked in. :/
,
Dec 7 2016
A cursory look at this is that (at least from archive) we're passing in a nil retry.Factory [1] which prevents retries [2] [1] https://github.com/luci/luci-go/blob/master/client/cmd/isolate/archive.go#L68 [2] https://godoc.org/github.com/luci/luci-go/common/retry#Factory
,
Dec 7 2016
</shamecube>
,
Dec 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/64707122e72f7405ce9b04b38323cf8bbf7cbf2d commit 64707122e72f7405ce9b04b38323cf8bbf7cbf2d Author: djd <djd@chromium.org> Date: Tue Dec 13 02:22:05 2016 common/lhttp: close the Response.Body on non-200s Refactor lhttp NewRequest to make sure that the response's body is always drained/closed in the case that the handler is not invoked (for 5xx responses, etc.). Add text to the returned error from NewRequest to make it how many attempts were attempted before failing. BUG= 671592 Review-Url: https://codereview.chromium.org/2562293002 [modify] https://crrev.com/64707122e72f7405ce9b04b38323cf8bbf7cbf2d/common/lhttp/client.go [modify] https://crrev.com/64707122e72f7405ce9b04b38323cf8bbf7cbf2d/common/lhttp/client_test.go
,
Dec 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/f84d8594c980fc2ed8a7cedf87bba9344f570afe commit f84d8594c980fc2ed8a7cedf87bba9344f570afe Author: Dave Day <djd@chromium.org> Date: Tue Dec 13 03:41:00 2016 Roll luci-go Rolling forward to @0451078 to pick up latest exparchive changes and fixes to lhttp retry logic/logging. BUG= 598990 , 671592 Change-Id: I9540c694bb69994a58e8836d72d425c61a919b44 Reviewed-on: https://chromium-review.googlesource.com/418103 Reviewed-by: Dave Sansome <dsansome@chromium.org> Commit-Queue: Dave D <djd@chromium.org> [modify] https://crrev.com/f84d8594c980fc2ed8a7cedf87bba9344f570afe/DEPS
,
Dec 14 2016
Detected 18 new flakes for test/step "isolate". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyEgsSBUZsYWtlIgdpc29sYXRlDA. This message was posted automatically by the chromium-try-flakes app.
,
Dec 19 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3736c107fc832c28af2961351bfa31675c4e92da commit 3736c107fc832c28af2961351bfa31675c4e92da Author: djd <djd@chromium.org> Date: Mon Dec 19 01:21:13 2016 Roll isolate binaries generated at infra@e0081b8. This includes the luci-go repo luci/luci-go@e12a0ea Isolate sha1s were taken from the following builds: https://build.chromium.org/p/chromium.infra/builders/infra-continuous-precise-64/builds/5079 https://build.chromium.org/p/chromium.infra/builders/infra-continuous-mac-10.10-64/builds/4186 https://build.chromium.org/p/chromium.infra/builders/infra-continuous-win-64/builds/5006 BUG= 671592 , 598990 Review-Url: https://codereview.chromium.org/2581823002 Cr-Commit-Position: refs/heads/master@{#439393} [modify] https://crrev.com/3736c107fc832c28af2961351bfa31675c4e92da/tools/luci-go/linux64/isolate.sha1 [modify] https://crrev.com/3736c107fc832c28af2961351bfa31675c4e92da/tools/luci-go/mac64/isolate.sha1 [modify] https://crrev.com/3736c107fc832c28af2961351bfa31675c4e92da/tools/luci-go/win64/isolate.exe.sha1
,
Dec 19 2016
The latest version of isolate has additional logging to try to work out why these uploads are failing.
,
Dec 28 2016
,
Mar 1 2017
Detected 3 new flakes for test/step "isolate". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyEgsSBUZsYWtlIgdpc29sYXRlDA. This message was posted automatically by the chromium-try-flakes app.
,
Mar 1 2017
Ignore flakes today as it is due to issue 697275.
,
Mar 24 2017
Detected 7 new flakes for test/step "isolate". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyEgsSBUZsYWtlIgdpc29sYXRlDA. This message was posted automatically by the chromium-try-flakes app.
,
May 2 2017
We haven't see any new flakes since Mar 24th. I'm going to close this bug for now. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by rogerta@chromium.org
, Dec 6 2016