I am not that familiar with non-extension docs and our use of github repos for them. cc'ing Istiaque and Devlin, in case they have more context, and know what might be wrong.

Crystal and I were talking about the possibility of moving the webstore docs into the extension docs folder, as a way to work around this. What do others think about this work-around?

Do you have context into why the webstore docs stopped working? Also, do you know if the devtools and multidevice docs are still updating fine. Seems all of them are to be served from GCS buckets- https://cs.chromium.org/chromium/src/chrome/common/extensions/docs/templates/json/content_providers.json?type=cs&q=webstore+docs&sq=package:chromium&l=100.

No idea why - most of devtools has been migrated off the website and no updates to multidevice have been submitted since discovering this issues. So I am unsure if they are effected by this. I could try submitting a test update if that would help? 

Yeah that should. Thanks.

Was able to push a patch to multi-device, but I am unsure if the update will be reflected in the new built. However, the DevTools legacy docs threw and issue with travis:
https://github.com/GoogleChrome/devtools-docs/pull/322

https://travis-ci.org/GoogleChrome/devtools-docs/jobs/345375575

This might be related to the overall issue? 

No idea regarding the dev-tools issue. Let me know once you figure out if the multi-device docs updated or not. 

Also, it seems no one really knows how this is set-up. I'll try to find some time to investigate. In the meanwhile, if anyone wants to take it, feel free.

The DevTools update was successfully merged into the GitHub repo - but it doesn't look like either Multi-Device or the DevTools update took. So it seems all three are broken. Let me know if I can do anything else to help narrow the issue down! 

Thanks Crystal. I'll try and investigate when I get some time. 

So I looked at this and the issue seems to be simply that the Cloud storage buckets haven't been updated with the latest commits in the github repository. A simple fix is to just upload the updated files through the google cloud website. From what I can see, the webstore bucket hasn't been updated after 30th August.

mkearney@, Devlin: Do you know whether this process of updating the buckets is supposed to be automated and whether this is broken? Or did we simply forget that these buckets have to be updated manually?

TL;DR: I'm not sure.

I thought that we did automatically pull in from github automatically using either github_file_system.py [1] or new_github_file_system.py [2], but looking more closely at those, I only see them reference app samples.  So maybe we never did pull them in automatically?  Given we'd like to migrate away from these, I'm also not sure it's worth changing that now.

Meggin, do you know?

[1] https://chromium.googlesource.com/chromium/src/+/993e5eaf14e467c24caf216dab919f673d2edeee/chrome/common/extensions/docs/server2/github_file_system.py
[2] https://chromium.googlesource.com/chromium/src/+/993e5eaf14e467c24caf216dab919f673d2edeee/chrome/common/extensions/docs/server2/new_github_file_system.py

I am not sure if these files are even being used.

content_providers.json [1] lists all the different content providers. Earlier the webstore, devtools docs etc. were served using github content provider [2] But this was disabled [3] and then we moved on to the GCS content provider [4]. What I am currently unsure is whether the process to update the gcs buckets was ever automated. If not, it seems just manually updating the buckets should work.

[1] https://cs.chromium.org/chromium/src/chrome/common/extensions/docs/templates/json/content_providers.json?q=content_providers.json&sq=package:chromium&dr

[2] https://codereview.chromium.org/67283005/patch/250001/260001

[3] https://codereview.chromium.org/138443007

[4] https://codereview.chromium.org/139303023/patch/1940001/1950018

Think mkearney@ should know.

Yeah pretty sure these Github file systems are not being used. These were removed from content_providers.py in https://codereview.chromium.org/1151283007. cc'ing rockot@ in case he has more context.

Unfortunately I can't remember why we decided to kill the Github filesystem at the time, but I believe it was broken somehow and not getting updated anyway.

We didn't automate the process of pushing the Github contents into GCS. I thought we did add a script to do this manually but I can't find it.

Seems it might be possible to automate this via the cron we run. Will file a bug for that and look into it when I get some time. And also for cleaning up the github file system code.

As for this, I don't think there's anything to do here. We just need to upload the files to the GCS bucket. I think mkearney@ already has access. 

Just sent all this same info to Karan in a chat, so adding it to this ticket, as well.


The original code to get this working (from the GitHub side of things, at least) I believe was implemented by Renato Mangini. Adding him to the ticket to see if he can at least remember how it worked way back when with the GitHub filesystem.

For sure, in the beginning, when you published updates in GitHub, eventually they were updated in the site without manual intervention. Then this stopped working, and I think that's when Ken, made it possible to manually run the script to update the docs.

But that stopped working too. So now there's no way to update.

I'm a little confused how the manual updates to GCS would happen, given at some point the templates need to get applied to the markdown files to create the final html content. 

Cc: mangini@chromium.org 
Cc: mangini@google.com

Karan just checked and pointed out that most of the webstore docs are html already (my bad, they aren't markdown by default)

That said, there is still an application of templates against the files to render the final html. 

The start and end meta tags indicate the template to use.

And Karan just sent me a link to the files in GCS-- they are identical to the files in GitHub, so maybe it is just an uploading of files to GCS. That's a relief!

Just tried a manual update and doesn't seem to be working.

This probably requires a version bump. Will send out a CL.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/91cede693fe9eaf76fdc920ccd62f2784e8fc15e

commit 91cede693fe9eaf76fdc920ccd62f2784e8fc15e
Author: Karan Bhatia <karandeepb@chromium.org>
Date: Sat Mar 10 01:57:58 2018

Extensions Docserver: Bump the version.

This CL bumps the Docserver version to 3.55.0. We uploaded new templates to the
chrome webstore GCS bucket and this is required to see whether the update works.

BUG= 796031 

Change-Id: Ibfeb8e549c642390f47918bb6c0a8e1a1279577c
Reviewed-on: https://chromium-review.googlesource.com/957813
Reviewed-by: Istiaque Ahmed <lazyboy@chromium.org>
Commit-Queue: Karan Bhatia <karandeepb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#542313}
[modify] https://crrev.com/91cede693fe9eaf76fdc920ccd62f2784e8fc15e/chrome/common/extensions/docs/server2/app.yaml

Closing this, updates can be made currently by updating the GCS buckets manually. Automating this is tracked in issue 820605.