Just to confirm. The request attempts were not using disk cache. I tried even after deleting cache. In fact. There are thousands of images. And I have tried dozens of images which I never opened in my browser. But still, the behavior is same. Shows CORS error if "disable cache" is not selected. Shows fine if "disable cache" is selected. 

I again inspected using Fiddler. If I do same action (that request the mentioned CloudFront resource from javascript) in Firefox, it makes a normal request and received correct Access-Control-Allow-Origin = * header in the response (because request includes security header) and so the request is completed fine without any issue. 

In case of chrome when the same image is displayed on the page (not access from javascript) it doesn't include security header in request hence response doesn't have ACAO header. This response is now cached and when next request is made to the same image from javascript it fails with CORS error (If "disable cache" is unchecked). 

This clearly explains why it works with "disable cache" and not without it. 

https://stackoverflow.com/questions/31732533/s3-cors-always-send-vary-origin

I think there are a few similar cases in the issue tracker -
Issues  809891 ,  409090 , 741674, 741958
 Issue 809891  comment 15 has some solution, it might be applicable in your case as well (in short, leverage the Vary header).

Well, such solutions are always there. I've already made a workaround fix for us by creating Lambda Edge Function for CloudFront distribution which will always include CORS headers no matter whether browser requested them (with Origin/Security headers) or not. 

The right question is, why does chrome behave like this and other browsers not?

See https://github.com/whatwg/fetch/issues/402#issuecomment-257323396, it seems Firefox uses two different caches for <img> and XHR.

Not an expert here of course. But, I believe it is only more logical to have separate cache. I think the chrome's behavior is simply wrong ill-conceived. Uness there's an important reason why it is doing so, which I am not aware of. 

#7 - having one single cache or multiple separate caches is an implementation detail that should not affect the correctness of your code, in my opinion.
You should be ready for either one of the situations by using the Vary header with the "Origin" value, I believe, which is probably the correct thing to do anyway.

Now, I just hope that CORS headers are also cached...

As this issue is out of scope of triaging at TE end, adding 'TE-NeedsTriageHelp' label and requesting someone from Blink>SecurityFeature>CORS team to look into the issue and help in further triaging.

Thanks..

yhirano@: Mind triaging this?

The spec also recommends to use Vary for complex Access-Control-Allow-Origin. https://fetch.spec.whatwg.org/#cors-protocol-and-http-caches

We don't have separate entries for various Access-Control-Allow-Origin headers because we don't think it's worth doing.