New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 23 users

Issue metadata

Status: Fixed
Last visit > 30 days ago
Closed: Apr 2017
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocked on:
issue 309693
issue 558829
issue 600636

issue 519042

Sign in to add a comment

Issue 505048: Chrome makes more conditional re-validation requests than other browsers

Reported by, Jun 26 2015

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36

Example URL:

Steps to reproduce the problem:
A while back, we noticed that a significant percentage of requests to Facebook's CDN for javascript & css resources were If-Modified-Since requests that resulted in a 304 Not Modified response. Interestingly, Chrome had a substantially higher fraction of such requests than other browsers. We found one substantial cause of this, that Chrome had a non-standard behavior that re-validated resources on a page that was redirected to after a POST (eg, when a user made a post to log in to the site and we redirected them to the home page).

Since this fix was applied (see  issue 294030 ), we ran the same analysis again. Chrome seems to have improved substantially -- in the past 60% of the requests Chrome sent for JS/CSS resources were conditional requests resulting in a 304 not modified. That number has dropped to 24%. However, Chrome still sends us nearly 2X the ratio of conditional requests.

Chrome     24%
Firefox    12%
IE         10%
Safari     14%

What is the expected behavior?
Chrome should have the same level of revalidation requests as other browsers

What went wrong?
Chrome had a higher level

Did this work before? N/A 

Chrome version: 43.0.2357.130  Channel: stable
OS Version: OS X 10.10.3
Flash Version: Shockwave Flash 18.0 r0

Comment 1 by, Jun 26 2015

Additional context:!topic/net-dev/ZQjvaV4ogTM

One thing that was mentioned was the FrameLoadTypeSame behavior possibly causing this. (But that was just something that came to mind on a whim. Enumerating everything that sets LOAD_VALIDATE_CACHE is probably not going to be worth anyone's time.)

At a glance, Firefox does not seem to have this same-URL behavior. We could maybe just get rid of it. This link also seems to suggest we and Safari are alone in it. +japhet

Of course, given how old this is, I'm sure we'll discover tons of tiny subtle things everywhere rely on it and will have to chase down bugs. :-) And by old, I mean nearly six years older than the public release of Chrome: 

(That was quite a git traversal. FrameLoader got renamed so many times...)

Comment 2 by, Jun 26 2015

I tried to rip out FrameLoadTypeSame a year or two ago and it got reverted. We could certainly take another stab at it.

Comment 3 by, Jun 26 2015

Could we make FrameLoadTypeSame so that it does a revalidation request on the root document, but that it does not revalidate sub resources? That might be a reasonable compromise between not revalidating at all and revalidating everything.

Comment 4 by, Jun 26 2015

Ben: in the report you indicated "OS-Mac". Is this data for Mac users only, or does it apply to all platforms?

Comment 5 by, Jun 26 2015

Our statistics show that this happens to users across all platforms.

Comment 6 by, Jun 26 2015

Labels: -OS-Mac OS-All
Thanks, updated the bug.

Comment 7 by, Jun 26 2015

japhet: Do you remember why it got reverted?

Comment 8 by, Jun 26 2015

Alternatively, I could have just looked:

Although, that turns Same into Reload. I was thinking turning Same into Standard.

Comment 9 by, Jun 26 2015

#3: That's an option... there's some logic there for POST I think. But before we think about adding complexity for compromises, we should probably confirm we even want the top-level to revalidate. :-P

Comment 10 by, Jun 26 2015

I dug up the patch. It says it broke an android test, but links to the bot results instead of listing the failing tests. Since bot results get purged, the test list is gone.

@ben.maurer, I'm open to the idea of revalidating the main resource of the main frame only, especially if removing the FrameLoadTypeSame concept is infeasible for some reason. Do you prefer that we explicitly keep the main resource revalidation behavior?

Comment 11 by, Jun 26 2015

Personally, I don't really care one way or another if we revalidate the main resource. For Facebook, our main resources are always no-cache (since they are dynamic content). My main concern is to avoid revalidating sub resources

Comment 12 by, Jun 27 2015

Labels: -Cr-Internals-Network Cr-Internals-Network-Cache
I think the cache is the appropriate network sub-bucket for this, though I understand we may be driving the behavior in question from outside the cache.

Comment 13 by, Jun 29 2015

Status: Untriaged

Comment 14 by, Jun 30 2015

Ben - do you have a simple test case that shows that behavior? (conditional requests being sent when nothing should have been sent)

Comment 15 by, Jun 30 2015

Unfortunately, we don't have a specific test case here -- though, the same page navigation seems like a clear case where chrome differs from other browsers.

Comment 16 by, Jul 1 2015

Labels: Cr-Blink-Loader Cr-UI-Browser-Navigation
re #1: If by "same-url" behavior you mean that hitting enter into the omnibox without navigating away triggers a reload, that was an intentional UI decision (I'm not saying it cannot change). The background was that most users don't really mess with URLs (as opposed to devs) and if there is a URL there and the user hits enter, the real desired action is to display again the current page, possibly fixing any error or "hang" that may be happening (effectively, what reload means) (I don't think there's data about it). I don't know if that matters at this point, but I still think that enter==reload is a natural assumption.

As far as I understand, that is also part of the meaning of pressing F5: if there is something missing (or wrong?) somewhere in the current page, it should be fixed by asking the server again. I'm not sure we want to limit that bahavior to the main resource on a page, but maybe I misunderstand the proposal. Reload (or force reload) is not a regular navigation by any means.

Now, if we end up using the flag without a clear user interaction, we should fix that. And if users are reloading a lot we should also fix that.

On another note, as a user I'd like to see a better separation of use cases:
1. I just want the latest version of a page (there's nothing that I'm trying to fix).
2. There is something wrong or the page is stuck, so try again
3. This is completely broken start from scratch (don't even ask for revalidation, and force all intermediary to also go back to the server).

Right now 1 & 2 mean Reload (F5 or enter), but I don't need special flags for #1.
3 means force reload (ctrl-F5).

Of course, most people don't know that force-reload even exists. (again, no data to back this up).

Comment 17 by, Jul 7 2015

Ben: do you have any data on number of same-page reloads for your use case? Can it explain some of the difference between Chrome and other browsers?

Comment 18 by, Jul 7 2015

Not sure how we'd get that data -- there's no way for us to tell if a navigation was caused by the refresh button or by the URL bar.

Comment 19 by, Jul 8 2015

Facepalm, right.. Checking our UMA metrics, I don't see anything tracking this either. Perhaps this is a good place to start on this end.

Comment 20 by, Jul 9 2015

We looked at some data about what % of FB loads are reloads.

We see about 2.2% of page loads from Chrome being reloads, 1.9% being back-forward the rest being normal navigation. IE has roughly the same ratios, firefox is lower on reloads (1%ish). Given that many navigations to FB happen with a warm cache it's not unreasonable that 2% of page loads being reloads result in 20% of the misses in the browser cache (and thus hits on our CDN).

We aren't sure if these reloads are explicitly triggered by the user or are via same page navigation.

Comment 21 Deleted

Comment 22 by, Jul 17 2015

Additional data confirming similar Chrome behavior from one of our Google products (with lots of traffic):

| browser        | 304     |
| Chrome Desktop |   38.8% |
| Opera Mini     |   15.6% |
| Opera          |   13.4% |
| Chrome Android |   12.3% |
| Safari Mac     |   12.3% |
| iOS            |   4.8%  |
| IE             |   4.2%  |
| Android        |   1.5%  |
| FF             |   1.2%  |
| other          |   0.1%  |

For desktop, ~39% of Chrome requests are revalidations as compared to ~4% of IE requests - that's a very big difference. On mobile things are looking a bit better, but we're still 3x the revalidation rate of mobile Safari.

Unfortunately this data does shed any light on the root cause.. need to keep digging.

Comment 23 by, Jul 17 2015

In no particular order, brainstorming some mechanisms that can trigger this:

- Lots of reload extensions:
- DevTools: we know that we have many developers using DevTools, can that skew data? 
- Offline auto-reload: chrome://flags/#enable-offline-auto-reload, chrome://flags/#enable-offline-auto-reload-visible-only
- Restart / tab restore?
- Reload button
- URL bar + enter (we revalidate)
- "hard refresh" (cmd-r and equivalent)

Anything else? An "initiator" histogram on reloads would be awesome to have.

Comment 24 by, Jul 17 2015

I think it makes most sense to start by getting rid of the same-URL thing. That's a concrete thing we know of where we behave different from other browsers and probably can just get rid of it.

There's enough navigation entry points right now and places where the relevant bits can get set that plumbing through an "initiator" enum would likely be rather a mess.

Comment 25 by, Jul 18 2015

Project Member
The following revision refers to this bug:

r199143 | | 2015-07-18T00:24:40.633597Z

Changed paths:

Don't revalidate all resources for FrameLoadTypeSame

The main resource of the top-loading frame probably need to be revalidated, but the rest of the page can probably prefer the cache.

Also, simplify setting of load type FrameLoadTypeSame.

BUG= 505048 

Review URL:

Comment 26 by, Jul 18 2015

Project Member
The following revision refers to this bug:

r199155 | | 2015-07-18T07:48:08.981112Z

Changed paths:

Oilpan: fix webkit unit tests following r199143.

Need to promptly detach the local frame created in

BUG= 505048 

Review URL:

Comment 27 by, Jul 20 2015

See  issue 511699  for one particular case where we send revalidation requests when we apparently shouldn't.  The repro is extension-related, but may happen at other times as well, not sure.

Comment 28 by, Jul 20 2015

> I think it makes most sense to start by getting rid of the same-URL thing. That's a concrete thing we know of where we behave different from other browsers and probably can just get rid of it.

+1. At a minimum I'd like to see a counter and an experiment to see how frequently this is triggered. For reference, background on this behavior:

Comment 29 by, Nov 17 2015


Comment 30 by, Nov 18 2015

On mobile, there is another popular* flow which is similar to the same-URL navigation pattern: pull to refresh. From a quick test, it appears that we conditionally revalidate everything for this flow.

The gesture comes from native apps where the expectation is one of refreshing the content of the app not reloading the app (i.e. re-installing/fixing errors). Can we also apply the same tweak here?

*: used by 24% of our uma users (7 days); the reload button in the menu is used by 6% of our uma users (7 days).

Comment 31 by, Nov 18 2015


Comment 32 by, Nov 20 2015

Fb stats suggest that we see roughly the same number of revalidations across all chrome platforms:

Windows 19% 
Android	21%
Mac OS X 21%
Chrome OS 18%

Comment 33 by, Nov 20 2015

Blockedon: chromium:558829
Talked to our UX team, they would like to play with the new behavior for pull to refresh.
Filed  issue 558829 

Comment 34 by, Dec 10 2015

OK, the change to navigation to same URL from Omnibox didn't have a huge impact on the numbers as seen from Facebook. I'm assuming that it's not popular enough. Higher hopes for pull to refresh.

However, pull to refresh, while a huge opportunity, is only relevant to mobile. This issue being cross platform, this suggests that there is definitely something else at play.

Comment 35 by, Jan 11 2016

Labels: DevRel-Faceboook

Comment 36 by, Jan 11 2016

Labels: -DevRel-Faceboook DevRel-Facebook

Comment 37 by, Jan 19 2016

Status: Started
I looked into Tab evictions. Here is what I'm seeing:
 1. On Android, a call to setNeedsReload is made for tabs that activated after an eviction. This flags triggers a NO_RELOAD load (i.e. loads without revalidating the cache). See [1] below for details.

 2. On other platforms, evicted tabs are reloaded with type=RELOAD which triggers a cache revalidation. See [2] for details. 

I also found an existing bug for this that was scoped to Chrome OS: issue 309693

//Wall of urls//

[1]: bread crumbs: 
 - setNeedsReload called upon tab eviction at:

 - restoreIfNeeded calling requestRestoreLoad if needsReload is set:

 - requestRestoreLoad calling native function :

 - native setNeedsReload called:

 - loadIfNecessary checking for needsreload and using NO_RELOAD (i.e. without revalidating the cache):

 [2]: bread crumbs
 - reload called for discarded tabs:
 - reloadInternal called:

 - call to reload(true):

 - reloadinternal called with type set to RELOAD:

 - call to NavigateToPendingEntry with Type set to RELOAD

 - call to NavigateToPendingEntryInternal, type = RELOAD:

Comment 38 by, Jan 19 2016

Blockedon: chromium:309693

Comment 39 by, Jan 19 2016

Nice find!

We saw high numbers of reloads across all platforms. Do tab evictions apply in windows and mac?

Comment 40 by, Jan 19 2016

Not historically.  That may have changed at some point, though.

Comment 41 by, Jan 26 2016

It's currently off by default on Windows and Mac OS X.

However, there are Tab Evictions experiments running for Windows and Mac. It's unclear how much 304s we can attribute to evictions though.

Regardless, it's still worth considering fixing issue 309693 because Tab Evictions will eventually ship (guess) and make the problem even worse.

Comment #22 indicates that revalidations on Chrome desktop happens far more often than Chrome mobile, hence the hope that the reload behavior differences on tab restore would be an important reason.

Comment 42 by, Jan 29 2016

Adding myself and chrisha@ on CC, as we're actively working on tab discarding/eviction for desktop.

Comment 43 by, Feb 25 2016

Was wondering if we managed to work out anything new here in the last couple of weeks, or if this is part of some bigger plan to fix issues? Thanks

Comment 44 by, Feb 25 2016

This still is definitely on our radar among many other things.
We have identified 2 causes (see blocked on) so far but clearly they do not explain it all and we are still looking for more.

 The initial patch for 558829 didn't WAI. Toyoshim had a working version a week or so ago but the reloading paths are overly complex which lead to the desire to clean up the mess. Hopefully, this would help us tame the beast without having to track individual triggers.

Comment 45 by, Feb 25 2016

Just off the press: a working patch for 558829 has been submitted and *should* be reflected in the Dev channel for Android behind chrome://flags#enable-non-validating-reload-on-refresh-content

Comment 46 by, Feb 25 2016

Great to know, I've enabled the flag.
May I ask why it's not on by default in Dev (or later beta)? I don't think
many users turn that on just out of curiosity. (as I think the localization
of the flag is a bit confusing)

Comment 47 by, Feb 25 2016

Labels: Reload

Comment 48 by, Feb 25 2016

For what it's worth at fb we're seeing this on desktop and mobile. So the pull to refresh fix is great, but there also seems to be a substantial number of revalidations coming from desktop refreshes.

Comment 49 by, Feb 25 2016

#48: I know that there is more, hence this meta-bug still being "started".

#46: Definitely, we don't expect that a lot of folks know about nor should care about chrome://flags. This is where our experiments live. Provided nothing surprising shows up, we'll eventually turn this ON for everyone.

Comment 50 by, Feb 28 2016

We started logging window.performance.navigation.type.

On Chrome/Windows, 4.3% of navigations to the FB home page are TYPE_RELOAD -- compare to 5% for Firefox, 2.7% for Edge.

On Chrome/Android,  14.3% of navigations to the FB home page are TYPE_RELOAD.

This suggests that pull to refresh does result in a much higher number of reload operations on mobile. Removing revalidations on PTR will likely be a big win in the mobile experience.

That said, we're still seeing almost 5% of desktop navigations be reloads. This seems to not be out of the range of other desktop browsers.

Comment 51 by, Feb 28 2016

Keep in mind that it wouldn't be unusual for 5% of our navigations to result in 20% of our CDN requests being 304s. Many users navigate to FB and have their resources cached. So the 5% may account for a high percentage of the uncached page loads.

Comment 52 by, Feb 29 2016

Thanks Ben!

So, if the type=Reload is inline with other browsers, it could mean 2 things:
 1. the behavior of our type=reload over-triggers revalidations.
 2. something else is driving the extra revalidations.

Is there anything odd going on with the other values (esp. TYPE_BACK_FORWARD) ?
I'll do some tests and report back.

Comment 53 by, Feb 29 2016

We didn't see anything unusual with other values. I'll point out a 3rd possibility is that chrome is just generally more cache efficient than other browsers and so has fewer non-reload requests.

That said, if 5% of our navigations are reloads, and these reloads are actually being requested by real users we really want to find a way to avoid the excessive revalidations in these cases. This is especially true if reloads are driven by user frustrations on slow connections.

Comment 54 by, Mar 1 2016

>This is especially true if reloads are driven by user frustrations on slow connections.

Ben, can  you segment this out in your analytics? If not, Chrome for Android supports downlinkMax:

Comment 55 by, Mar 1 2016

Worth noting that slow connections and spotty connections may both have problems here.  Not sure spotty connections are reflected downlinkMax, or ones that are technically fast but in practice slow.

Comment 56 by, Mar 1 2016

there looks to be at least some correlation between location and number of refreshes. For example on desktop we see this percentage of requests being reloads:

north america 3.8%
europe 4%
africa 5.7%
oceania 4.7%
asia 4.2%

That said, the pattern doesn't hold on mobile (in fact africa gets more reloads than north america). It's possible that the PTR behavior on android is skewing the stats from people who reload out of frustration.

Comment 57 by, Apr 5 2016

Blockedon: 600636

Comment 58 by, Apr 11 2016

As part of this issue, here are our thoughts on Reload behaviors

Comment 59 by, May 10 2016

Blocking: 519042

Comment 60 by, Jan 13 2017

Just checking in - is this issue now resolved?

Comment 61 by, Mar 31 2017

Should we consider this fixed by now due to all the reload-reloaded work?

Comment 62 by, Apr 1 2017

Status: Fixed (was: Started)
Yes, I believe that we are in a much better place now.

Sign in to add a comment