crbug.com/639476 was a symbol upload failure on the official builders only. This bug was not caught by a presubmit and triggered a failure that did not include the name of the file that was failing. This made the investigation tricky and it ended up taking almost a week to get fixed.
To fix the bug a chrome-internal script change (https://chromereviews.googleplex.com/495017013) was made to add a line of diagnostic output. This allowed the root cause to be identified and then https://codereview.chromium.org/2271583005 was landed to fix the bug.
It would be good to make further changes to the symbol upload script to make future issues easier to investigate. The most basic change would be to always print the name of the file being processed first, so that any exceptions will have context. Currently (after my first change) the name is printed when an exception occurs, but even this is not ideal because it requires expert knowledge to realize that the issue is that a .pdb file is being processed when PE files are supposed to be. If all of the file names were printed then this break in the pattern would be obvious. This change may be applicable to other scripts as well.
It would also be good to enforce some of the FILES.cfg rules (such as "'symsrc' tag cannot be applied to .pdb files", and presumably others) in a presubmit check.
Comment 1 by grt@chromium.org
, Aug 25 2016