New issue
Advanced search Search tips

Issue 648716 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Sep 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: ----



Sign in to add a comment

Return 409 instead of 500 when build number conflicts on upload to test-results happen

Project Member Reported by serg...@chromium.org, Sep 20 2016

Issue description

We see a lot of "build number conflict" errors on test-results.appspot.com: https://pantheon.corp.google.com/logs/viewer?project=test-results-hrd&key1=go&minLogLevel=0&expandAll=false&resource=appengine.googleapis.com&logName=projects%2Ftest-results-hrd%2Flogs%2Fappengine.googleapis.com%252Frequest_log&filters=text:%22build%20number%20conflict%22&lastVisibleTimestampNanos=1474395724696921000.

They are caused by test launcher which upload incremental_results.json files for the same builds multiple times. We should figure out why this happens and fix it. For now I am planning to just change the HTTP response code returned by test-results from 500 to 200 and simply log the warning when this happens.
 
Labels: -Restrict-View-Google
Status: Available (was: Untriaged)
The reason to remove 500 HTTP error codes is to reduce overall number of 500s so that we can do better monitoring for other errors, which would otherwise be masked by large number of 'build number conflict' errors.

Comment 4 by estaab@chromium.org, Sep 20 2016

Please return a 409 Conflict in this case. Clients shouldn't retry 4xx errors and it will let is still track the rate of these errors in graphs (vs everything looking like a 200 success).
Summary: Figure out why do test launchers are uploading full_results.json for the same build multiple times (was: Figure out why do test launchers are sending incremental_results.json for the same builders multiple times)
Ack. Also I've noticed that those errors do not necessarily happen when uploading incremental_results.json. They also happen when uploading full_results.json. It could be even that they only come from full_results.json.
Summary: Return 409 instead of 500 when build number conflicts on upload to test-results happen (was: Figure out why do test launchers are uploading full_results.json for the same build multiple times)
Owner: serg...@chromium.org
Status: Fixed (was: Available)

Sign in to add a comment