New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 763375 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

blink_perf tests failed uploading to cloud storage

Project Member Reported by nedngu...@google.com, Sep 8 2017

Issue description

See it happens to a bunch of blink_perf tests that use tracing:

blink_perf.css: https://chromium-swarm.appspot.com/task?id=387a0b3b2323db10&refresh=10&show_raw=1

blink_perf.owp_storage: https://chromium-swarm.appspot.com/task?id=387a0b4ecd11ea10&refresh=10&show_raw=1

Log:
(INFO) 2017-09-08 04:33:52,509 exception_formatter.PrintFormattedException:19  Try printing formatted exception: <class 'py_utils.cloud_storage.CloudStorageError'> BucketNotFoundException: 404 gs://None bucket does not exist.
 <traceback object at 0x7f298631aea8>

Traceback (most recent call last):
  <module> at /b/s/w/ir/tools/perf/run_benchmark:26
    sys.exit(main())
  main at /b/s/w/ir/tools/perf/run_benchmark:22
    return benchmark_runner.main(config, [trybot_command.Trybot])
  main at /b/s/w/ir/third_party/catapult/telemetry/telemetry/benchmark_runner.py:352
    return command_instance.Run(options)
  Run at /b/s/w/ir/third_party/catapult/telemetry/telemetry/benchmark_runner.py:222
    return min(255, self._benchmark().Run(args))
  Run at /b/s/w/ir/third_party/catapult/telemetry/telemetry/benchmark.py:108
    return story_runner.RunBenchmark(self, finder_options)
  RunBenchmark at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py:355
    results.UploadTraceFilesToCloud()
  UploadTraceFilesToCloud at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/results/page_test_results.py:496
    value.UploadToCloud()
  UploadToCloud at /b/s/w/ir/third_party/catapult/telemetry/telemetry/value/trace.py:144
    self._upload_bucket, self._remote_path, fh.GetAbsPath())
  Insert at /b/s/w/ir/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:378
    _RunCommand(command_and_args)
  _RunCommand at /b/s/w/ir/third_party/catapult/common/py_utils/py_utils/cloud_storage.py:153
    raise GetErrorObjectForCloudStorageStderr(stderr)
CloudStorageError: BucketNotFoundException: 404 gs://None bucket does not exist.

Locals:
  args       : ['/b/s/w/ir/third_party/catapult/third_party/gsutil/gsutil', 'cp', '/b/s/w/itkqfyWh/tmpGvm5Sg.html', 'gs://None/None']
  gsutil     : <subprocess.Popen object at 0x7f298c833710>
  gsutil_env : None
  stderr     : 'BucketNotFoundException: 404 gs://None bucket does not exist.\n'
  stdout     : ''

(WARNING) 2017-09-08 04:33:52,520 ps_util._ListAllSubprocesses:89  Telemetry leaks these processes: python (3071) - []

I suspect this is due to Ben's change.
 
Owner: nedngu...@google.com
Status: Started (was: Assigned)
I have a CL
Cc: benjhayden@chromium.org
Owner: benjhayden@chromium.org
Discussed offline, here's the plan:

0. Announce API change on telemetry-announce.
1. Add prospective code to tools/perf/benchmarks/blink_perf.py and tools/perf/measurements/timeline_controller.py to plumb file_path/remote_path/upload_bucket/cloud_url via kwargs from results.telemetry_info if they exist.
2. Re-land https://codereview.chromium.org/3007063002
3. Clean up blink_perf.py and timeline_controller.py

Project Member

Comment 5 by bugdroid1@chromium.org, Sep 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/95db8734dfcfc6a93b75d3e77301f0599287cbd2

commit 95db8734dfcfc6a93b75d3e77301f0599287cbd2
Author: Ben Hayden <benjhayden@chromium.org>
Date: Fri Sep 08 23:33:03 2017

Telemetry metrics: plumb trace file info from results to TraceValue.

The TraceValue API will change when this catapult CL is re-landed:
https://codereview.chromium.org/3007063002

Bug:  #763375 
Change-Id: Id67c9825baacba6199b07e2c2d6d02c288ab183b
Reviewed-on: https://chromium-review.googlesource.com/658263
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ben Hayden <benjhayden@chromium.org>
Cr-Commit-Position: refs/heads/master@{#500739}
[modify] https://crrev.com/95db8734dfcfc6a93b75d3e77301f0599287cbd2/tools/perf/benchmarks/blink_perf.py
[modify] https://crrev.com/95db8734dfcfc6a93b75d3e77301f0599287cbd2/tools/perf/measurements/timeline_controller.py

Project Member

Comment 6 by bugdroid1@chromium.org, Sep 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a30e79d6a533a3cf9d03de7e73c6a2c2b03d39ad

commit a30e79d6a533a3cf9d03de7e73c6a2c2b03d39ad
Author: Ben Hayden <benjhayden@chromium.org>
Date: Mon Sep 11 20:41:10 2017

Fix Telemetry TraceValues in blink_perf, timeline_controller.

Manually tested this time -_-
http://www/~benjhayden/763375.html

https://codereview.chromium.org/3005313002
https://codereview.chromium.org/3008223002

Bug:  chromium:763375 
Change-Id: I084d59a6bf4f34902b2810b048748a57704bb502
Reviewed-on: https://chromium-review.googlesource.com/661177
Commit-Queue: Ben Hayden <benjhayden@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#501026}
[modify] https://crrev.com/a30e79d6a533a3cf9d03de7e73c6a2c2b03d39ad/tools/perf/benchmarks/blink_perf.py
[modify] https://crrev.com/a30e79d6a533a3cf9d03de7e73c6a2c2b03d39ad/tools/perf/measurements/timeline_controller.py

Project Member

Comment 7 by bugdroid1@chromium.org, Sep 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b9df3441dc4c014462b6d0a29d2ade02abdc9a83

commit b9df3441dc4c014462b6d0a29d2ade02abdc9a83
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Thu Sep 14 02:20:45 2017

Roll src/third_party/catapult/ c4a8d9578..444bcc528 (5 commits)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/c4a8d9578ef1..444bcc528f7b

$ git log c4a8d9578..444bcc528 --date=short --no-merges --format='%ad %ae %s'
2017-09-13 achuith Revert of Disable testWebPageReplay on chromeos. (patchset #1 id:1 of https://codereview.chromium.org/3012103003/ )
2017-09-13 simonhatch Dashboard - Fix kicking off Pinpoint functional bisects.
2017-09-13 benjhayden Reland Plumb trace canonicalUrl through TelemetryInfo.
2017-09-13 benjhayden Plumb device id through Histogram Pipeline.
2017-09-13 rnephew [BattOr] Improve logging when start or stop tracing time is not present.

Created with:
  roll-dep src/third_party/catapult
BUG= 762819 , 763375 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I92e768e4c369c1cf8719686403008ab4357fac28
Reviewed-on: https://chromium-review.googlesource.com/666009
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#501845}
[modify] https://crrev.com/b9df3441dc4c014462b6d0a29d2ade02abdc9a83/DEPS

Status: Fixed (was: Started)

Sign in to add a comment