New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 122548 link

Starred by 115 users

Issue metadata

Status: Assigned
Owner:
Last visit > 30 days ago
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocking:
issue kbsslenforcer:103



Sign in to add a comment

Chrome displays insecure content warnings for content handled by the WebRequest API

Reported by sazpai...@7chan.org, Apr 7 2012

Issue description

Chrome Version       : 18.0.1025.151
OS Version: 6.1 (Windows 7, Windows Server 2008 R2)
URLs (if applicable) : N/a
Other browsers tested:
Add OK or FAIL after other browsers where you have tested this issue:
     Safari 5: N/A
  Firefox 4.x: N/A
     IE 7/8/9: N/A

What steps will reproduce the problem?
1. Install any extension that redirects or blocks URLS using WebRequest (HTTPS Everywhere or Adblock)
2. Go to secure page with insecure elements
3. Redirect or block insecure URLs

What is the expected result?
Chrome should display the page as fully secure

What happens instead?
Chrome displays the insecure content icon and the console displays something along the lines of: The page at https://www.example.com/ ran insecure content from http://thisshouldbeblocked.com/blockme.js.

Please provide any additional information below. Attach a screenshot if
possible.

Since the URL is being intercepted by WebRequest, the insecure version of a page is not loading, therefore, Chrome shouldn't use this to determine if a page has insecure elements.

UserAgentString: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.151 Safari/535.19


 
Cc: battre@chromium.org
Labels: -Area-Undefined Area-Internals Feature-Security Feature-Extensions
Cc: tsepez@chromium.org

Comment 3 by jhurwich@google.com, Apr 13 2012

Battre, what's the desired behavior here?  If you block insecure elements, does that make the page "fully secure"?

Comment 4 by battre@chromium.org, Apr 14 2012

I think if you block all insecure elements that page should be considered secure.
Cc: mpcomplete@chromium.org
Status: Available
(marking as Available to get out of triage queue)

Comment 6 Deleted

The below data is provided based on user reports in 'GoogleFeedback'.

120+ users have reported this issue in 'GoogleFeeback' for version:18.0.1025.168

Chrome Version       : 18.0.1025.168 
URLs                 : google.com / facebook.com

There is yellow bar below omnibox which displays Error message" The page has insecure/unsafe content"

For more details, refer the user reports below:
http://goto.google.com/273867807
http://goto.google.com/273866803
http://goto.google.com/281671322

For further more user reports, refer the below cluster URL
http://goto.google.com/1754859

Note: Most of users have reported this issue on Win Vista SP1 and XP SP3

Comment 8 by battre@chromium.org, May 14 2012

Cc: rsleevi@chromium.org

Comment 9 by battre@chromium.org, May 14 2012

I am not sure this bug report is related to the cluster.

If you consider http://goto.google.com/zttpr or http://goto.google.com/yglbv you see no indication that AdBlock or HTTPS Everywhere is installed.
I believe the decision re: secure / mixed content is handled at a WebKit layer, and focuses on whether or not you made requests, not whether or not they were actually loaded.

My own take is that if a page is trying to /load/ insecure elements, that should at least be surfaced as a warning somewhere, otherwise it will never get fixed for all the users not using the extension. The resource itself is definitely "insecure", even if it was loaded (mostly) securely.
This bug is pretty bad.  In fact I'm not sure why it took us HTTPS Everywhere developers so long to notice that this was causing so much silliness.  It's pretty clear to us that if you use an extension to rewrite every HTTP request to HTTPS, the final page should be considered secure!

Here's a nice example of this bug at work:

1. git clone https://git.torproject.org/https-everywhere.git
2. cd https-everywhere
3. ./makecrx.sh today
4. install the resulting .crx file from the pkg/ directory
5. open about:net-internals or wireshark with BPF filter "port 80" to observe network traffic
6. go to http://xkcd.com 
7. observe that the page's CSS is blocked due to "mixed content"
8. click "load anyway"
9. note that the https:// is red and crossed out
10. check back with wireshark or about:net-interals -- with the posisble exception of OCSP, you should see that no HTTP requests were made while loading the page

(Until now, we had been busily disabling rulesets for sites like xkcd in the Chrome version of HTTPS Everywhere because we thought the mixed content warning was real...)

Comment 12 by palmer@google.com, Feb 14 2013

Cc: palmer@chromium.org
Owner: palmer@chromium.org
Status: Assigned
Adding an HSTS rule via chrome://net-internals/#hsts for xkcd.com with includeSubdomains as true also demonstrates this behaviour.

When I traced the onBeforeRequest code for WebRequest and in looking at how HSTS handles URL rewriting, both seem to be following the same paths and rewrite behaviours, so this may be a generic issue with WebKit's CachingResourceLoader (as my best guess) for where this may be coming. That or in the FrameLoader.
Cc: -nikhilalm@chromium.org -arsids@chromium.org -mnarula@chromium.org
Nudge!  Any progress on this?
Project Member

Comment 16 by bugdroid1@chromium.org, Mar 10 2013

Labels: -Area-Internals -Feature-Security -Feature-Extensions Cr-Platform-Extensions Cr-Security Cr-Internals
Cc: ajwong@chromium.org nasko@chromium.org
nasko, awong: I think you two have been looking at this recently, in the context of some of the other security work going on. The issue (AIUI) at present is that the WebKit layer requests a load for an HTTP resource, and then the Chromium network stack (whether through HSTS data or through the NetworkObserver/extension mechanisms) synthesizes a 307 to notify WebKit. This synthetic 307 is indistinguishable from a "real" 307, which is why I understand WK as triggering it as a mixed-content load.

It seems like we may need to plumb up some of this URL rewriting into WK - or some knowledge of a "synthetic"/secure rewrite versus an organic/insecure (because it came over HTTP) rewrite.

Thoughts?
@rsleevi: I actually have almost no understanding about most parts of the code you are referring to.  Sorry :(
Possibly relevant:

We just noticed that, while browser-internal rewrites in Firefox alter DOM elements' .src attributes, this doesn't happen with WebRequest API in Chrome at the moment.  

We noticed this because HTTPS Everywhere includes a rewrite that we were planning to use to promote the extension, but only for people who haven't already installed it:

<rule from="^https://www\.eff\.org/sites/all/themes/frontier/images/get-https-e(-chrome)?.png" 
   to="https://www.eff.org/sites/all/themes/frontier/images/got-https-e$1.png"/>

As demonstrated by this page: https://w2.eff.org/tmp/httpse.html, in Chrome, even though the rewrite occurs, the DOM is not adjusted to reflect that fact.  In Firefox, it is.  

Labels: Cr-Platform-Extensions-API
Owner: ----
Status: Available
Unfortunately, I don't have the time to look into this right now. I've got too much other stuff on my plate at the moment.
Cc: -ajwong@chromium.org abarth@chromium.org
pde: Correct, that was part of what I was referring to in comment #17 - WebKit has no knowledge that the network stack (or extensions) have altered the request in a 'trusted' rewrite. From the perspective of WebKit, the request rewriting is seen as just another network rewrite. Even though this is still a secure rewrite, WK still sees the load/redirect as potentially dangerous.

I think the case can be made for either behaviour, but I'm actually slightly partial to the existing behaviour in Chromium. The reason for this is that it ensures that the Web Inspector and various web security checks reflect the true contents of the page - if the page attempted to load insecure resources, it warns - the same as would happen if there was no HSTS rewrite (or no extension rewrite) to begin with.

I'll leave it to the extensions team to comment on whether or not this is the desired behaviour, long term. I'm not sure that the Web Request APIs were intended to have the side-effect of manipulating content at that level - and there are plenty of ways it can result in 'strange' or 'inaccurate' results (eg: JS loading via string concatenation like 'var url = "http://" + origin + "/path"' would not be rewritten in source, but would on the network)

Also cc'd abarth@ in the event I missed any implications of one behaviour or the other.
rsleevi: I also don't have a strong opinion about the "right" behaviour with respect to .src properties in the DOM tree.  

I do however have the strong opinion that the headline bug (mixed content warnings even though there is no mixed content) is a bug that the WebRequest API was intended not to have :)
rsleevi: here is an example of a YouTube bug that seems to be caused by the fact that .src attributes do change when a request is rewritten: https://trac.torproject.org/projects/tor/ticket/4408 (actually in HTTPS Everywhere for Firefox right now it seems that .src attributes get rewritten for images but not for script, urgh).

Of course, the fact that such bugs exist does not necessarily mean that there wouldn't be more or worse bugs if we changed the API's behaviour.

Comment 24 by Deleted ...@, Apr 8 2013

Do these problems also exist in SRWARE's Iron Browser?
How can we get some resources allocated to this issue?  It's flagrantly a bug, with no possible reason for not fixing it.  And at the moment it's a major contributor to the fact that HTTPS Everywhere (and the associated security benefits) are ten times more popular on Firefox than on Chrome :(
Owner: battre@chromium.org
Status: Assigned
battre: Could you own this or pass it on to the appropriate person on extensions?

From discussions with palmer@ and others, we're of consensus that the HSTS side of things *should* continue to trigger the mixed content warnings. That is, if a site is HSTS, but attempts to load over HTTP, then *even though* the resource was fetched over HTTPS (through transparent HSTS rewrites), we still want to signal to the site author of an error - for example, if users which did not have HSTS visited.

However, for extensions, they represent discrete actions that the end-user has taken to install. When using the Web Request (declarative or legacy) to rewrite URLs, these do not represent actions taken by the site operator - instead, actions taken by the user. In this case, it MAY (at the extensions team's discretion) be worth suppressing the error.
Labels: -OS-Windows OS-All
Any further developments/progress regarding this?
Cc: vabr@chromium.org
Owner: ----
Status: Available
No progress so far. I currently don't have the cycles to work on this at the moment. I don't know how knows the code that triggers the warnings.
Owner: jered@chromium.org
Status: Assigned
rsleevi@ or battre@, whom should I talk to on the extensions team? Are we agreed the current behavior is a bug? I'd be happy to spend some cycles to fix this since it's rather annoying for users of this popular extension.
jered: It's a bug *only* for the case of extensions using WebRequest. That is, HSTS SHOULD still trigger mixed-content warnings if loads are attempted over HTTP.

This will likely require a fair bit of diving on the WebKit side, since that is what is responsible for driving the mixed content status - and it has absolutely zero knowledge that
1) A [transparent] rewrite happened - compared to an HTTPS redirect
2) A rewrite was 'approved' (eg: extension) or not (eg: HSTS)
A couple of questions:

1. What technical obstacles exist to calling over to the WebRequest code path right at the point where MCB is about to occur, "hey before I block this http resource tell me if you were going to rewrite it to https?"  I realise that that might be a slow cross-thread call, but since it's only happening in a pathological situation (MCB is about to fire) I can't see how it would create real performance problems.

2. I hear that you guys have decided to let MCB fire on HSTS sites.  I might be missing something, but it seems like that is the wrong call.  One vision of HSTS is as a thing that helps operators secure their sites more easily; the other vision is that HSTS is the cherry you put on top of an already-flawless HTTPS deployment.  I think HSTS will be much more useful to the world with the former interpretation, which means autorewriting to HTTPS if the site set HSTS for this domain.  I think Mozilla is going in the same direction too.

Am I missing any good arguments for sticking with the more pedantic interpretation of HSTS? 
Mozilla's bug is https://bugzilla.mozilla.org/show_bug.cgi?id=838395 , which contains more of the discussion re: UI.

Comment 34 by jered@chromium.org, Jul 10 2013

It's just a nasty plumbing mess. I think a workable fix for the declarative api would be to have ChromeRenderViewObserver::allowDisplayingInsecureContent() return true if ExtensionWebRequestEventRouter::OnBeforeRequest() [gotten at somehow via ChromeNetworkDelegate] returns an https rewrite. webkit might get angry and print something in protest, but it should actually load. It should separately be possible to fix the security state afterwards in case that still looks funny.

It looks much harder to fix the blocking api. That seems like it would require making DocumentThreadableLoader loadRequest() asynchronously call back.
The mozilla bug rsleevi posted is a semi-duplicate of https://bugzilla.mozilla.org/show_bug.cgi?id=878890 
I am not very familiar with the HSTS implementation. A challenge I see is that the WebRequest APIs (declarative and old) work on HTTP requests, i.e. they assume that HTTP request headers are available. This is not the case when WebKit fires the requests.

@Peter: What is "MCB" referring to?
@battre "MCB" is me being lazy about typing "mixed content blocking" over and over again :)

That header situation sounds like bad news :\.  Sounds like the best one could do would be to rev the API to support this use (which is horrible).  Probably saner to fight the plumbing and move the MCB implementation to a toally different place.  That's what we're doing over in Mozilla land.
Cc: jered@chromium.org
Owner: abarth@chromium.org
I don't see a technical approach for fixing this bug.  In the case of "active" mixed content, we block the request before we even send it, so the WebRequest API never gets to see the request because the request never happens.  For "display" mixed content, we also flag the error before issuing the request.  That means that even if the request fails, we'll still log the error.

We can leave this bug open if you like, but until we have a technical plan for how to change this behavior, it's unlikely to be fixed.

Comment 39 by jered@chromium.org, Jul 11 2013

abarth@, to elaborate on comment #34, I think WebKit does the relevant mixed content check at core/loader/DocumentThreadableLoader.cpp:435 where it calls canDisplayInsecureContent(). That method calls allowDisplayingInsecureContent() with the insecure subresource KURL. I was speculating we could plumb that to ChromeRenderViewObserver::allowDisplayingInsecureContent(), and build a stub GET request for that KURL to send ExtensionWebRequestEventRouter::OnBeforeRequest(). I think, but am not certain, that would give the declarative API a chance to set *new_url to an https URL, so that the request could be allowed instead of blocked. As far as WebKit is concerned, this is an insecure resource but we'll allow it anyway. Does this sound remotely sane?

As far as the right answer... it seems correct to detect mixed content as early as possible. But why do the blocking in WebKit instead of in chrome? We could just tell WebKit to always allow mixed content, but communicate isMixedContent() to poison the request, and do the blocking/didFail() in Chrome prior to actually starting the URLRequest. That sounds hard but seems more technically correct.
> Does this sound remotely sane?

That sounds kind of crazy.  :)  What happens if the extension does something different when the real request goes through the pipeline?

> But why do the blocking in WebKit instead of in chrome?

We originally did the blocking in Chromium, but the implementation was buggy and never worked correctly.  I re-implemented it in WebKit and now it works correctly.  The problem is that Chromium doesn't have a detailed enough understanding of the requests to do the mixed content blocking correctly.  For example, a correct implementation needs to understand the interactions with the MemoryCache.

> We could just tell WebKit to always allow mixed content

There's no reason to "fake out" WebKit.  If we moved mixed content detection to Chromium, we'd just rip out the mixed content detection code in WebKit.

I don't believe it's possible to implement well on the Chromium side because I tried for several years.  The current approach works correctly for the majority of users who don't install security extensions.  I don't think we should introduce bugs for those users just to fix this issue.
@abarth : I presume what you mean by "no technical approach" is "no really easy technical approach".  Since the request doesn't hit the network until after WebRequest inspects it, there isn't any deep reason why MCB couldn't be moved/reimplemented using WebRequest itself or (more realistically) with custom code right after WebRequest is finished, right?

PS -- we've been working on a patch to do exactly this sort of refactoring in Mozilla's MCB implementation at the moment: https://bugzilla.mozilla.org/show_bug.cgi?id=878890#c19 .  We have a lot more Firefox users than Chrome users (probably because and othe MCB problems haven't landed on us in Mozilla yet!) so it's a higher priority for us.


Comment 42 by jered@chromium.org, Jul 12 2013

> > Does this sound remotely sane?
> That sounds kind of crazy.  :)  What happens if the extension does something different when the real request goes through the pipeline?

I think we have to assume that we trust the extension here, since the way we get into this mess is when a webrequest api hook has rewritten an insecure base page url to a secure one; if we trusted it to rewrite the base page URL, it can already show anything it wants.

>> But why do the blocking in WebKit instead of in chrome?
> ... Chromium doesn't have a detailed enough understanding of the requests to do the mixed content blocking correctly.  For example, a correct implementation needs to understand the interactions with the MemoryCache.
The MemoryCache does indeed seem pretty special, but I'm unsure how webrequest works at all if its transparent rewrites are MemoryCached on the pre-rewrite url. That would just be wrong. So in this case (webrequest has overriden base page), I think it'd be correct to ignore it and check in chromium, no? Are there other gotchas?
> I think we have to assume that we trust the extension here

It's not a matter of trust.  It's a matter of building a system that works consistently and correctly.

> Are there other gotchas?

Yes.

Comment 44 by jered@chromium.org, Jul 12 2013

How about calling RVObserver and propagating the rewrite back to WebKit prior to the canDisplayInsecureContent call? Also how do we detect mixed content if webrequest api does an insecure rewrite?
Owner: jered@chromium.org
> How about calling RVObserver and propagating the rewrite back to WebKit prior to the canDisplayInsecureContent call?

That might work.  I'd need to study the code more carefully.  We might not have fully prepared the request when we check it for mixed content, but we can move the mixed content check later if needed.

> Also how do we detect mixed content if webrequest api does an insecure rewrite?

The only way to do this correctly are for changes made by the webrequest API to be treated as redirects.  That means Blink will get to see them again via didReceiveRedirect.

I'm assigning this issue back to jered who seems to have some ideas how to make progress here.
> We originally did the blocking in Chromium, but the implementation was buggy and never worked correctly.  I re-implemented it in WebKit and now it works correctly.  The problem is that Chromium doesn't have a detailed enough understanding of the requests to do the mixed content blocking correctly.  For example, a correct implementation needs to understand the interactions with the MemoryCache.

It sounds like Jered has some ideas for a workaround for these issues, but another one to put on the table is to have the existing webkit MCB implementation mark requests with a new mixed-content-to-be-blocked flag, and then a second piece of code in chrome reads the flag after WebRequest has happened, and only cancels the request if it is still not secure at that point.
Just returned from vacation and will look at this some this week.
Since xkcd has fixed their mixed content, I've made a cleaner repro case.

https://eff.org/files/mcb-bug.html

With HTTPS Everywhere installed, that page should embed jQuery successfully.  However because MCB fires before WebRequest, jQuery is completely blocked.

Unfortunately, eff.org uses HSTS so that page actually won't display the MCB override UI.  I don't have access to any non-HSTS HTTPS webservers at this instant, so perhaps someone should put the file on a different HTTPS server for comparison.
Ok, plumbing a bit down seems best [the extension doesn't use the non-blocking API anyway so my hack wouldn't have helped]:
- ResourceFetcher sets ResourceRequest::m_isMixedContent based on checkInsecureContent(), and enforces a Reload memory cache policy.
- We send that all the way down to URLRequest.
- URLRequest::Start() lets NotifyBeforeURLRequest() suggest a redirect; if it doesn't, URLRequest cancels, else it starts a URLRequestRedirectJob.
- That should call didReceiveRedirect() which I _think_ gets us back to OnReceivedRedirect() in webkitland and lets us try again.

I've got a combined patch but am still debugging it.
Jered, it sounds like you had a promising lead on this in August. Any updates since then?
No, I got too busy with my main project and dropped this. I may have some time this month, but someone else should feel free to grab this.
Jered, do you think you could post your partial/buggy patch so that if we start working on this ourselves we have a snapshot of your thinking?

Comment 53 by jered@chromium.org, Jun 24 2014

Sorry Peter, I can't find my patch from #49, and doubt it would apply now. I am trying to page this code back into my head but it's different from last time I looked. Let's chat more off thread.

Comment 54 by k...@kbit.dk, Jul 30 2014

Blocking: kbsslenforcer:103
CloudFlare just deployed HTTPS for around 2 million new sites.  If this bug were fixed, most of those sites could be secured by simply setting HSTS.  While this bug is present, each of those 2 million sites will require significant effort to find and change every http:// URL before security is an option.

(also someone should edit this bug title to make it clear that the bug applies to attempts by both WebRequest and HSTS rewrites to fix mixed content before it is blocked)
Peter,

The bug title is accurate, as only the WebRequest behavior is considered for change. The HSTS behavior is WAI.

The security benefits are known to be questionable. To benefit, these sites need to fix the links.

Let's keep the bug focused on AebRequest.

Comment 57 by mkwst@chromium.org, Oct 16 2014

Cc: mkwst@chromium.org
Labels: Hotlist-Recharge
This issue likely requires triage.  The current issue owner maybe inactive (i.e. hasn't fixed an issue in the last 30 days).  Thanks for helping out!

-Anthony
This issue is breaking HTTPS Everywhere for some of it's rules

Comment 60 by vabr@chromium.org, Feb 8 2016

Cc: -vabr@chromium.org
Cc: -mpcomplete@chromium.org
Any process with this issue?

Sign in to add a comment