Hmm... I won't have a chance to dive into this tonight, but cc'ing some folks that might be interested.

My guess is that this is because Privacy Badger has its own private key, and that might be somehow behaving badly with our content verification - but that's just a guess.

My guess is, it's related to the fact that CWS can't inject the Google update URL into the extension's manifest.json (because the CRX is uploaded to CWS already signed), and there is no update_url property in that manifest as-is.

   "update_url": "https://clients2.google.com/service/update2/crx",

That would definitely do it.  We now require that the update url be present for the extension to be considered "from the store".

James, do we have any contacts with the Privacy Badger extension we can reach out to?

Yep, I have a thread going with them related to CRX₃. That being said, I think they don't want to specify the update_url in the manifest because they also distribute the extension off-Webstore. I'll send them a link to this bug, let's see what they say.

Either way, it's possible there are other extensions that fall into this case.

Thanks Josh. Do we have a way of identifying the other extensions that fall into this case?

Privacy Badger dev here. We are going to add the following to Privacy Badger's manifest and do an update later today:

"update_url": "http://clients2.google.com/service/update2/crx

I believe the reason we removed this originally was to try working around content verification issues (https://bugs.chromium.org/p/chromium/issues/detail?id=643814), but we clearly need it now.

It's too bad this is a surprise and we are losing users because of it.

@amiagkov - thanks for the quick response, and sorry for the hassle this has caused.  I think that adding back the update url should fix the issue, but it will be good to confirm.   Issue 643814  should also be fixed.

I can definitely appreciate that having this as a surprise is very unfortunate.  I wish we had better coverage for this use case (where the extension is being distributed through the webstore, but signed by a third party), but at a practical level, we just don't have as much engineering bandwidth to cover it as would be necessary.  We will, of course, do everything we can to ensure that these issues *don't* happen, but it would be really helpful if you could also semi-regularly test your extensions on early builds of chrome (Canary/Dev/Beta) to help catch these early on.  If you ever notice anything wrong, please do file a bug and cc me on it, and we'll work as quickly as we can to address it - hopefully fixing it before it gets to this point. :)

Thanks again for the quick action on your part!

Possibly dumb question: Do we somehow try to update extensions which do not have an update_url in the manifest? 

If we do not, does this mean that users which had one of the versions without an update_url have not been getting new versions of the extensions?

We uploaded a Privacy Badger update a bit earlier with update_url added to the manifest. Status is still "Publishing in progress".

We do run automated tests on Chrome beta, but our current tests wouldn't flag this problem as it's a sporadic issue that seems to happen during Chrome upgrades. I am not even able to reproduce this locally (is this a Windows-only problem?).

Privacy Badger users have been getting upgraded on Chrome just fine without an explicit update_url in the manifest. Which makes update_url seem like a magic incantation to me. I don't have the full history of why we removed it in the first place, just a vague idea that it was to work around content verification issues described in  Issue 643814 .

This is from install verification, which is enabled on Windows, Mac, and ChromeOS, but not Linux.  You can enable it on Linux with the command line flag --extensions-install-verification=enforce (or --extensions-install-verification=enforce_strict).  You can enable content verification on linux with --extension-content-verification=enforce (or --extension-content-verification=enforce_strict).  That should replicate windows/mac/cros behavior.

Does it matter if I specify "http://clients2.google.com/service/update2/crx" or "https://clients2.google.com/service/update2/crx" for `update_url` in the manifest? HTTP vs. HTTPS.

@13 Yes, it will matter.  The HTTPS url should be used ("https://clients2.google.com/service/update2/crx").

Another way of checking is that the update_url field should match that of another extension downloaded from the webstore that isn't packed separately.  You can easily view the extension manifest on disk by looking in the Extensions folder of the user data directory (if you haven't done this before, I can give more detailed instructions :)).

I launched with the HTTP URL because I copied the field from HTTPS Everywhere's manifest (and didn't notice the non-secure protocol). Everything seems fine so far, but I will update to the HTTPS update_url as part of the next update.

Assigning to myself to get this out of triage.  amiagkov@, let us know if this works, and if there's anything else we can help with on our end!

OK, will do, thanks for all your help!

Incidentally, I found some extensions in my main profile folder with http update_url entries ... At least some are Google extensions! For example:

$ cat ~/.config/google-chrome/Default/Extensions/pjkljhegncpnkpknbcohdijeoejaedia/8.1_0/manifest.json

```json
{
   "app": {
      "launch": {
         "container": "tab",
         "web_url": "https://mail.google.com/mail/ca"
      },
      "urls": [ "*://mail.google.com/mail/ca" ]
   },
   "default_locale": "en",
   "description": "__MSG_appDesc__",
   "icons": {
      "128": "128.png"
   },
   "key": "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDCuGglK43iAz3J9BEYK/Mz6ZhloIMMDqQSAaf3vJt4eHbTbSDsu4WdQ9dQDRcKlg8nwQdePBt0C3PSUBtiSNSS37Z3qEGfS7LCju3h6pI1Yr9MQtxw+jUa7kXXIS09VV73pEFUT/F7c6Qe8L5ZxgAcBvXBh1Fie63qb02I9XQ/CQIDAQAB",
   "manifest_version": 2,
   "name": "__MSG_appName__",
   "options_page": "https://mail.google.com/mail/ca/#settings",
   "permissions": [ "notifications" ],
   "update_url": "http://clients2.google.com/service/update2/crx",
   "version": "8.1"
}
```

Updating using the new https URL seems to be working fine.

Any other issues here before we close this out?

None from my end.

The NextAction date has arrived: 2018-10-05