New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 865725 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Jul 20
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocking:
issue 828917



Sign in to add a comment

Telemetry failure log has no exceptions or clear errors

Project Member Reported by sullivan@chromium.org, Jul 19

Issue description

Blocking: 828917
Note: I'm seeing similar logs on Linux (first link is Mac): https://pinpoint-dot-chromeperf.appspot.com/job/11e25c3ea40000
Cc: perezju@chromium.org
Owner: nednguyen@chromium.org
Status: Started (was: Untriaged)
I am able to reproduce the problem on my Mac. Steps:
$ git cl patch 1143585
$ ./tools/perf/run_benchmark system_health.common_desktop --story-filter multitab:misc:typical24 --pageset-repeat 1 --browser=system --also-run-disabled-tests

Investigating
With https://chromium-review.googlesource.com/c/catapult/+/1145042, I can now see the error is due to trace processing:
(ERROR) 2018-07-20 07:48:49,201 page_test_results.Fail:540  Failure recorded: TraceImportError: Unexpected token Ö in JSON at position 0
    at JSON.parse (<anonymous>)
    at TraceEventImporter.createAsyncSlices_ (/tracing/extras/importer/trace_event_importer.html:1645:31)
    at TraceEventImporter.importEvents (/tracing/extras/importer/trace_event_importer.html:1150:12)
    at importer (/tracing/importer/import.html:198:65)
    at task.subTask (/tracing/importer/import.html:145:32)
    at Task.run (/tracing/base/task.html:80:50)
    at Function.Task.RunSynchronously (/tracing/base/task.html:125:25)
    at Import.importTraces (/tracing/importer/import.html:74:17)
    at createModelFromTraceData (/Users/nednguyen/projects/chromium/src/third_party/catapult/tracing/tracing/mre/map_single_trace_cmdline.html:38:9)
    at eval (/Users/nednguyen/projects/chromium/src/third_party/catapult/tracing/tracing/mre/map_single_trace_cmdline.html:57:25)
[  FAILED  ] system_health.common_desktop/multitab:misc:typical24@{'case': 'multitab', 'group': 'misc'} (139881 ms)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ]  system_health.common_desktop/multitab:misc:typical24@{'case': 'multitab', 'group': 'misc'}

1 FAILED TEST

So it seems like we collected some weird character in the multitab traces.
Thanks! I'm going to try to just re-do the recording in that CL. 
Tried re-recording on two different machines, I still get the same issue. I don't have the cycles to dig into the weird character in the trace at the moment.
Status: Fixed (was: Started)
Annie: can you move #7 to issue 828917? 

This should be fixed with https://chromium-review.googlesource.com/c/catapult/+/1145042 landing now. Feel free to reopen if you still see this
Yep, moving #7.
Project Member

Comment 10 by bugdroid1@chromium.org, Jul 20

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/213eaf4f79bcdfc088cb7a6512948b7777403c7f

commit 213eaf4f79bcdfc088cb7a6512948b7777403c7f
Author: Nghia Nguyen <nednguyen@google.com>
Date: Fri Jul 20 14:53:05 2018

Make sure Telemetry log error recorded by results failure

There are two ways error with benchmark running are logged on Telemetry:
(1) Python exception stack are directly logged by call sites (see example in https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py?rcl=d26f5c315d5ed3e9389ea209a51611f979a18db7&l=100)
(2) Call sites create their own exception message and invokes results.Fail(error_message)

This ensures that results.Fail(..) always log error message created by call site. This is the safest
way to ensure that we never miss logging error by mistakes from call sites.

The cons of this approach is sometimes client may mistakenly log error twice, but such error is less severe
and easier to fix than missing error log.

Bug:  chromium:865725 
Change-Id: I88608306cde54e5e6901c3caa9c8a053a4c4d243
Reviewed-on: https://chromium-review.googlesource.com/1145042
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>

[modify] https://crrev.com/213eaf4f79bcdfc088cb7a6512948b7777403c7f/telemetry/telemetry/internal/results/page_test_results.py
[modify] https://crrev.com/213eaf4f79bcdfc088cb7a6512948b7777403c7f/telemetry/telemetry/internal/story_runner.py
[modify] https://crrev.com/213eaf4f79bcdfc088cb7a6512948b7777403c7f/telemetry/telemetry/internal/story_runner_unittest.py

Project Member

Comment 11 by bugdroid1@chromium.org, Jul 20

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b241b3ee47b8f78e72ec61220aae5b5e4957a5a2

commit b241b3ee47b8f78e72ec61220aae5b5e4957a5a2
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri Jul 20 17:43:15 2018

Roll src/third_party/catapult 58189855d207..213eaf4f79bc (1 commits)

https://chromium.googlesource.com/catapult.git/+log/58189855d207..213eaf4f79bc


git log 58189855d207..213eaf4f79bc --date=short --no-merges --format='%ad %ae %s'
2018-07-20 nednguyen@google.com Make sure Telemetry log error recorded by results failure


Created with:
  gclient setdep -r src/third_party/catapult@213eaf4f79bc

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:865725 
TBR=sullivan@chromium.org

Change-Id: I76b5afa08638ea9431056a0a373c92edbdabd6e0
Reviewed-on: https://chromium-review.googlesource.com/1145441
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#576914}
[modify] https://crrev.com/b241b3ee47b8f78e72ec61220aae5b5e4957a5a2/DEPS

Comment 12 by benhenry@google.com, Jan 16 (6 days ago)

Components: Test>Telemetry

Comment 13 by benhenry@google.com, Jan 16 (6 days ago)

Components: -Speed>Telemetry

Sign in to add a comment