Return 409 instead of 500 when build number conflicts on upload to test-results happen |
|||||
Issue descriptionWe see a lot of "build number conflict" errors on test-results.appspot.com: https://pantheon.corp.google.com/logs/viewer?project=test-results-hrd&key1=go&minLogLevel=0&expandAll=false&resource=appengine.googleapis.com&logName=projects%2Ftest-results-hrd%2Flogs%2Fappengine.googleapis.com%252Frequest_log&filters=text:%22build%20number%20conflict%22&lastVisibleTimestampNanos=1474395724696921000. They are caused by test launcher which upload incremental_results.json files for the same builds multiple times. We should figure out why this happens and fix it. For now I am planning to just change the HTTP response code returned by test-results from 500 to 200 and simply log the warning when this happens.
,
Sep 20 2016
,
Sep 20 2016
The reason to remove 500 HTTP error codes is to reduce overall number of 500s so that we can do better monitoring for other errors, which would otherwise be masked by large number of 'build number conflict' errors.
,
Sep 20 2016
Please return a 409 Conflict in this case. Clients shouldn't retry 4xx errors and it will let is still track the rate of these errors in graphs (vs everything looking like a 200 success).
,
Sep 20 2016
Ack. Also I've noticed that those errors do not necessarily happen when uploading incremental_results.json. They also happen when uploading full_results.json. It could be even that they only come from full_results.json.
,
Sep 20 2016
Cofirmed - they only happen on full_results.json since the following query returns no results in the last 2 days: https://pantheon.corp.google.com/logs/viewer?project=test-results-hrd&key1=go&minLogLevel=0&expandAll=false&resource=appengine.googleapis.com&logName=projects%2Ftest-results-hrd%2Flogs%2Fappengine.googleapis.com%252Frequest_log&advancedFilter=metadata.serviceName%3D%22appengine.googleapis.com%22%0Alog%3D%22appengine.googleapis.com%2Frequest_log%22%0A%22build%20number%20conflict%22%0A-%22updateFullResults%22%0A.
,
Sep 22 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/bb281b54acb9d6af56ff0c1696b32fe3b4f201bc commit bb281b54acb9d6af56ff0c1696b32fe3b4f201bc Author: sergiyb <sergiyb@chromium.org> Date: Thu Sep 22 00:29:30 2016 Return 409 error when uploading build with same number again R=estaab@chromium.org BUG= 648716 Review-Url: https://codereview.chromium.org/2355353002 [modify] https://crrev.com/bb281b54acb9d6af56ff0c1696b32fe3b4f201bc/go/src/infra/appengine/test-results/frontend/upload.go [modify] https://crrev.com/bb281b54acb9d6af56ff0c1696b32fe3b4f201bc/go/src/infra/appengine/test-results/frontend/upload_test.go
,
Sep 22 2016
,
Sep 22 2016
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by serg...@chromium.org
, Sep 20 2016