bpastene@: since this is an Android issue, can you help to triage it? In particular, is it sufficiently bad to require an owner, or can we leave it as Available?

I'm assigning it to you for triage, but feel free to make it available or assign to someone else. Thanks!

PS. The link appears to have expired... Is there a better link or steps to get fresh screenshots / repro the issue reliably?

Whoops, didn't realize result_details expire. See org.chromium.chrome.browser.preferences.password.SavePasswordsPreferencesTest and org.chromium.chrome.browser.media.remote.CastSwitchVideoTest in the result_details of chrome_public_test_apk in https://ci.chromium.org/buildbot/chromium.android/Lollipop%20Phone%20Tester/17818.

All the same screenshot. Assigning to hzl@ as he owns result_details.

Yes. I will look into this, though I didn't work on screenshots.

#2: The signed URLs that cloud storage hands out expire. If you want to paste one of those here, you need to copy the result_details URL from the link on the build page; in that case, https://storage.cloud.google.com/chromium-result-details/html/chrome_public_test_apk_Lollipop%20Phone%20Tester_17818_2018_01_06_T11_25_14-UTC

I locally tested out the screenshots functionality (which is slightly different with the one used in bots) and the screenshots taken by multiple retries are different.

However one thing I noticed is that if a retry happens right after the first attempt and no more than one minute has passed between the first and second attempt and the third attempt, the screenshot should be exactly the same right? Since the test case is the same and fails for the same reason and no more than one minute has passed (meaning the time stamp is the same), is there any reason that the screenshot should be different?

If the second/third attempt happens more than 1 minute after the first attempt, then the screenshot should change. And that means the remote version of the screenshot functionality has some bug. So does the second/third attempt happens right after the first/second attempt?

Oh I misunderstood the problem. The problem is multiple test cases have the same screenshot. That's definitely something wrong with our remote screenshot module.

hmm I am still looking into why local screenshot works and remote screenshot does not. Suspecting that somehow the upload path becomes the same for all these screenshots:
https://codesearch.chromium.org/chromium/src/build/android/pylib/output/remote_output_manager.py?type=cs&l=74

After some investigation, the post test screenshot seems to be working on other bots. 

For example here:
https://ci.chromium.org/buildbot/chromium.android/Lollipop%20Phone%20Tester/17945

1 Go to chrome_public_test_apk result_details
2 Then click on 'ModalDialogManagerTest' suite
3 You will see the three post test screenshots working properly.

Is it possible that in the bot Ben mentioned here, the screen happens to be frozen for some unknown reason?

I also tried locally using the remote output manager to test the screenshot functionality and it seems to be working fine. Let me know what you think.

The screen might have been frozen. Though, if that were the case, I'd expect more than just those handful of tests to fail.

That said, if things look sane and you feel comfortable thinking it was a flake, feel free to close this. I haven't seen any reoccurrences of it (though I haven't been looking too hard.)

I suspect the content-addressed nature of the screenshots is responsible for this unusual but not wrong behavior. If the failures all look the same and run in the same minute, we'll only upload one screenshot & will use that for all of them.

Take, for example, this: https://ci.chromium.org/buildbot/chromium.android/Marshmallow%20Phone%20Tester%20%28rel%29/1670

You will notice that the failing test fails three times, and that *two of the three* screenshots are the same. Both of those ran at 2:03 according to the phone's clock on screen. The third has a different hash and ran at 2:04 but otherwise looks the same.

I think you can close this.

re #11: That wouldn't really explain the images captured for org.chromium.chrome.browser.preferences.password.SavePasswordsPreferencesTest#testExportMenuEnabled in https://ci.chromium.org/buildbot/chromium.android/Lollipop%20Phone%20Tester/17818

Each attempt ran several minutes apart (https://chromium-swarm.appspot.com/task?id=3ae3f5c9c28c8d10) yet had the same contents. Zhiling's theory that the screen is frozen might explain it, but if that was the case I'd expect more than 17 tests to fail in the first few attempts.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/21a3bb7aa1fb72ec4b6a8dd244341397877bbd10

commit 21a3bb7aa1fb72ec4b6a8dd244341397877bbd10
Author: Zhiling Huang <hzl@chromium.org>
Date: Tue Jan 23 23:54:39 2018

Minor fix in remote output manager.

When passing content type to google storage helper, we need to specify
that we are passing in content type. Otherwise google  storage helper's
upload function will consider content type as gs_args and then cause an
error.

Bug:  799598 
Change-Id: Iac18782af63d38b66b4094ab32472f1debb68f92
Reviewed-on: https://chromium-review.googlesource.com/862882
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Zhiling Huang <hzl@chromium.org>
Cr-Commit-Position: refs/heads/master@{#531388}
[modify] https://crrev.com/21a3bb7aa1fb72ec4b6a8dd244341397877bbd10/build/android/pylib/output/remote_output_manager.py