This is again a problem today, even in incognito mode, canary, and Firefox. mstensho@ is seeing this too.

Still happening.

+sergiyb@ who also spotted the 500 errors on Friday.
+benjhayden@ for any ideas on how to prevent future errors with the jinja2 dependency.

When it broke on Friday evening, it was because I deployed a new version, which included https://crrev.com/c/1259563. It looks like that CL inadvertently added a dependency on jinja2, which is included in perf dashboard, but not Pinpoint. I ended up rolling back to the previous version, which caused the errors to clear up over the weekend. So the solution here will be to:
1. Roll back to the previous version, to clear up the immediate errors. (DONE)
2. Rework https://crrev.com/c/1259563 to remove the jinja2 dependency.
3. Add some kind of PRESUBMIT check to prevent future errors. +benjhayden@ if he has any ideas.

  File "/base/data/home/apps/s~chromeperf/pinpoint:clean-dtu-985e1309.413550217473948116/dashboard/pinpoint/models/job.py", line 16, in <module>
    from dashboard import update_bug_with_results
  File "/base/data/home/apps/s~chromeperf/pinpoint:clean-dtu-985e1309.413550217473948116/dashboard/update_bug_with_results.py", line 20, in <module>
    from dashboard.common import request_handler
  File "/base/data/home/apps/s~chromeperf/pinpoint:clean-dtu-985e1309.413550217473948116/dashboard/common/request_handler.py", line 10, in <module>
    import jinja2
ImportError: No module named jinja2

+tandrii@

Today's error looks like an HTTP 403 when fetching the patch details from Gerrit. Since we haven't changed anything in perf dashboard or Pinpoint, it's likely they changed something in Gerrit-on-Borg. tandrii@'s comment last night on issue 892756 said that AppEngine apps are banned from GoB by default, so maybe the fact that it worked previously was a fluke due to lax enforcement on GoB's side. So the solution here will be to:
1. Temporarily remove authentication from the Gerrit calls, to clear up the immediate errors.
2. Request whitelist access on GoB for perf dashboard and Pinpoint.
3. Re-add auth to all Git and Gerrit calls.

  File "/base/data/home/apps/s~chromeperf/pinpoint:clean-dtu-3f7d74f7.413040965508556777/dashboard/pinpoint/models/change/patch.py", line 45, in AsDict
    self.server, self.change, fields=('ALL_REVISIONS', 'DETAILED_ACCOUNTS'))
  File "/base/data/home/apps/s~chromeperf/pinpoint:clean-dtu-3f7d74f7.413040965508556777/dashboard/services/gerrit_service.py", line 22, in GetChange
    return request.RequestJson(url, o=fields)
  File "/base/data/home/apps/s~chromeperf/pinpoint:clean-dtu-3f7d74f7.413040965508556777/dashboard/services/request.py", line 31, in RequestJson
    content = Request(*args, **kwargs)
  File "/base/data/home/apps/s~chromeperf/pinpoint:clean-dtu-3f7d74f7.413040965508556777/dashboard/services/request.py", line 86, in Request
    content = _RequestAndProcessHttpErrors(url, use_auth, scope, **kwargs)
  File "/base/data/home/apps/s~chromeperf/pinpoint:clean-dtu-3f7d74f7.413040965508556777/dashboard/services/request.py", line 112, in _RequestAndProcessHttpErrors
    'HTTP status code %s: %s' % (response['status'], content))
HTTPException: HTTP status code 403

 Issue 899824  has been merged into this issue.

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/25da2e8be18d819ef6b5223a8b9dbf2223344cda

commit 25da2e8be18d819ef6b5223a8b9dbf2223344cda
Author: Dave Tu <dtu@chromium.org>
Date: Mon Oct 29 18:56:01 2018

[pinpoint] Temporarily remove auth from gerrit_service.GetChange().

TBR=benjhayden
Bug:  chromium:899427 
Change-Id: Id09d7d200f8d9d058564f4ce4e00926e17dbcbf6
Reviewed-on: https://chromium-review.googlesource.com/c/1305713
Reviewed-by: Dave Tu <dtu@chromium.org>
Commit-Queue: Dave Tu <dtu@chromium.org>
Auto-Submit: Dave Tu <dtu@chromium.org>

[modify] https://crrev.com/25da2e8be18d819ef6b5223a8b9dbf2223344cda/dashboard/dashboard/services/gerrit_service.py
[modify] https://crrev.com/25da2e8be18d819ef6b5223a8b9dbf2223344cda/dashboard/dashboard/services/gerrit_service_test.py

This still doesn't work.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b3b7012b37e94bc9bf3c35a2a88a71ded0e174c9

commit b3b7012b37e94bc9bf3c35a2a88a71ded0e174c9
Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Date: Mon Oct 29 20:22:59 2018

Roll src/third_party/catapult 625dca84765d..25da2e8be18d (2 commits)

https://chromium.googlesource.com/catapult.git/+log/625dca84765d..25da2e8be18d


git log 625dca84765d..25da2e8be18d --date=short --no-merges --format='%ad %ae %s'
2018-10-29 dtu@chromium.org [pinpoint] Temporarily remove auth from gerrit_service.GetChange().
2018-10-29 erikchen@chromium.org Reduce verbosity of logging for ADB commands.


Created with:
  gclient setdep -r src/third_party/catapult@25da2e8be18d

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:899427 ,chromium:891898
TBR=sullivan@chromium.org

Change-Id: I78a868219581286ce001a19c20005054e6a50c64
Reviewed-on: https://chromium-review.googlesource.com/c/1305893
Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#603604}
[modify] https://crrev.com/b3b7012b37e94bc9bf3c35a2a88a71ded0e174c9/DEPS

Now it works! Thank you!

 Issue 899880  has been merged into this issue.

I could imagine making presubmit process the yaml files and importing the script modules to check that it only imports jinja2 if it's specified in the yaml file, but I'm not sure if it would be worth it since I'm planning on removing the jinja2 dependency next year.
Has this error happened before?

Comment 11: yes, once before. https://crrev.com/c/1191983
Ok, if we will remove jinja2 next year, it likely won't be worth it.

The immediate P0 is fixed, issue 892756 will follow-up on getting Pinpoint access to Gitiles and Gerrit.