New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 873102 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 27
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 872758



Sign in to add a comment

All perf benchmark uploading fail with internal server error

Project Member Reported by nednguyen@chromium.org, Aug 10

Issue description

Project Member

Comment 1 by bugdroid1@chromium.org, Aug 10

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1018bbc64430009d70edff683a55633207dc097f

commit 1018bbc64430009d70edff683a55633207dc097f
Author: Ned Nguyen <nednguyen@google.com>
Date: Fri Aug 10 11:03:36 2018

Revert "Maximize the number of parallel processes for processing & uploading perf data"

This reverts commit a92b95bb53e21fc4874ab3b8b1833fe3b990e3b0.

Reason for revert: suspect this causes dashboard internal error. 
BUG: chromium:873102 

Original change's description:
> Maximize the number of parallel processes for processing & uploading perf data
> 
> A big portion of perf data uploading latency is due to I/O operation (uploading),
> hence maximize the number of parallel processes to improve the wall-time.
> 
> Bug:857283
> Change-Id: I0c3246849676a56801fe99846ade7265d13d94c0
> 
> TBR=simonhatch@chromium.org
> 
> Change-Id: I0c3246849676a56801fe99846ade7265d13d94c0
> 
> NOTRY=true  # android_arm64_dbg_recipe flake
> 
> Change-Id: I0c3246849676a56801fe99846ade7265d13d94c0
> Reviewed-on: https://chromium-review.googlesource.com/1170083
> Commit-Queue: Ned Nguyen <nednguyen@google.com>
> Reviewed-by: Ned Nguyen <nednguyen@google.com>
> Reviewed-by: Ethan Kuefner <eakuefner@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#581991}

TBR=simonhatch@chromium.org,eakuefner@chromium.org,nednguyen@google.com

Change-Id: I02d1e84a065bd9c315ec2242fc6a8a0bbd6593a1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 857283
Reviewed-on: https://chromium-review.googlesource.com/1169800
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#582113}
[modify] https://crrev.com/1018bbc64430009d70edff683a55633207dc097f/tools/perf/process_perf_results.py

Owner: sullivan@chromium.org
If you go to https://chromeperf.appspot.com/ you'll see the error "ImportError: cannot import name gae_ts_mon". Reverting https://chromium-review.googlesource.com/c/catapult/+/1170043 and re-deploying.
Labels: -Pri-0 Pri-2
Owner: benjhayden@chromium.org
Since the revert will take a while to submit, temporarily patched in a CL that comments out the ts_mon lines in dispatcher.py and re-deployed. Dashboard is up and running now. Lowered to P2 since dashboard is up.

Reassigning to benjhayden@ to reland the change safely and lead the postmortem (should cover dashboard push process, testing of changes to dispatcher.py)
Ah thanks Annie!

For posteriority, I will not reland #1 because that didn't seem to speed up perf uploading (it actually slow things down a big) 
Blocking: 872758
#1 was not intended to speed up the perf dashboard. It was intended to reduce the error rate. Whether or not it stays landed should be based on the logs for whether machine instances are allocated better. I'll leave it to Ethan.
Issue 873082 has been merged into this issue.
Issue 873093 has been merged into this issue.
Project Member

Comment 9 by bugdroid1@chromium.org, Aug 10

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/ce9fa204b82dd09676fb02107be2e2333f0703eb

commit ce9fa204b82dd09676fb02107be2e2333f0703eb
Author: Annie Sullivan <sullivan@chromium.org>
Date: Fri Aug 10 12:05:13 2018

Revert "Configure gae_ts_mon in chromeperf dashboard."

This reverts commit 5e7ddfe4f2b219d3e95a675bf8e09aaa0773d935.

Reason for revert: Dashboard failing with error "ImportError: cannot import name gae_ts_mon"

Bug:  873102 

Original change's description:
> Configure gae_ts_mon in chromeperf dashboard.
> 
> This CL configures the chromeperf dashboard to report metrics using gae_ts_mon.
> 
> https://dev-benjhayden-1155789-dot-chromeperf.appspot.com
> 
> Bug: chromium:867060
> Change-Id: I59615f76634bfac5e2da1c813af07923de0fc39c
> Reviewed-on: https://chromium-review.googlesource.com/1155789
> Commit-Queue: Ben Hayden <benjhayden@chromium.org>
> Reviewed-by: Annie Sullivan <sullivan@chromium.org>

TBR=sullivan@chromium.org,benjhayden@chromium.org,seanmccullough@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: chromium:867060
Change-Id: I5dd977f13be9bdf96d165d9252e6174f4bee5ef0
Reviewed-on: https://chromium-review.googlesource.com/1170043
Reviewed-by: Annie Sullivan <sullivan@chromium.org>
Commit-Queue: Annie Sullivan <sullivan@chromium.org>

[modify] https://crrev.com/ce9fa204b82dd09676fb02107be2e2333f0703eb/dashboard/app.yaml
[modify] https://crrev.com/ce9fa204b82dd09676fb02107be2e2333f0703eb/dashboard/dashboard/__init__.py
[modify] https://crrev.com/ce9fa204b82dd09676fb02107be2e2333f0703eb/dashboard/dashboard/dispatcher.py
[modify] https://crrev.com/ce9fa204b82dd09676fb02107be2e2333f0703eb/dashboard/v2spa.yaml
[modify] https://crrev.com/ce9fa204b82dd09676fb02107be2e2333f0703eb/dashboard/cron.yaml

Project Member

Comment 10 by bugdroid1@chromium.org, Aug 10

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4e054f53bfa1ade0310f11f8f5b09ba0cba04255

commit 4e054f53bfa1ade0310f11f8f5b09ba0cba04255
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri Aug 10 13:19:05 2018

Roll src/third_party/catapult 0d25dda9b148..ce9fa204b82d (2 commits)

https://chromium.googlesource.com/catapult.git/+log/0d25dda9b148..ce9fa204b82d


git log 0d25dda9b148..ce9fa204b82d --date=short --no-merges --format='%ad %ae %s'
2018-08-10 sullivan@chromium.org Revert "Configure gae_ts_mon in chromeperf dashboard."
2018-08-10 perezju@chromium.org [dashboard] Report templates for system health benchmark


Created with:
  gclient setdep -r src/third_party/catapult@ce9fa204b82d

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:873102 ,chromium:867060,chromium:870324
TBR=sullivan@chromium.org

Change-Id: I610b49c80a5dcc958f6491b47518d67c5ea817fa
Reviewed-on: https://chromium-review.googlesource.com/1169938
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#582131}
[modify] https://crrev.com/4e054f53bfa1ade0310f11f8f5b09ba0cba04255/DEPS

Status: Fixed (was: Untriaged)

Sign in to add a comment