New issue
Advanced search Search tips

Issue 754514 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Aug 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

blink_perf.binding.reference failing on perf waterfall

Project Member Reported by nedngu...@google.com, Aug 11 2017

Issue description

 https://build.chromium.org/p/chromium.perf/builders/Win%207%20Nvidia%20GPU%20Perf/builds/1113


Log:
[  FAILED  ] sequence-conversion-custom-iterator.html (4824 ms)
c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\image_processing\_bitmap.py:152: UserWarning: Using pure python png decoder, which could be very slow. To speed up, consider installing numpy & cv2 (OpenCV).
  'Using pure python png decoder, which could be very slow. To speed up, '
(INFO) 2017-08-10 10:01:58,720 browser.Close:268  Closing browser (pid=3508) ...
(INFO) 2017-08-10 10:01:59,026 desktop_browser_backend._TryCooperativeShutdown:618  Successfully shut down browser cooperatively
(INFO) 2017-08-10 10:01:59,030 browser.Close:283  Browser is closed.
(INFO) 2017-08-10 10:01:59,914 exception_formatter.PrintFormattedException:19  Try printing formatted exception: None None None
Traceback (most recent call last):
  RunBenchmark at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\story_runner.py:340
    expectations=expectations, metadata=benchmark.GetMetadata())
  Run at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\story_runner.py:203
    _RunStoryAndProcessErrorIfNeeded(story, results, state, test)
  _RunStoryAndProcessErrorIfNeeded at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\story_runner.py:105
    state.RunStory(results)
  traced_function at c:\b\s\w\ir\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py:52
    return func(*args, **kwargs)
  RunStory at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\page\shared_page_state.py:319
    self._current_page, self._current_tab, results)
  traced_function at c:\b\s\w\ir\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py:52
    return func(*args, **kwargs)
  ValidateAndMeasurePage at c:\b\s\w\ir\tools\perf\benchmarks\blink_perf.py:326
    results.current_page, metric, units, values))
  __init__ at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\value\list_of_scalar_values.py:83
    assert len(values) > 0
AssertionError
Locals:
  description           : None
  grouping_keys         : None
  important             : True
  improvement_direction : None
  name                  : 'sequence-conversion-custom-iterator'
  none_value_reason     : None
  page                  : <telemetry.page.Page object at 0x038F61B0>
  std                   : None
  tir_label             : None
  units                 : u'runs/s'
  values                : []

Typically, this shouldn't be a red step as it is a run of a reference build benchmark. I suspect it has to do with the exception being uncaught by core Telemetry, hence it caused invalid test result --> red step.

I will make fix for this in blink_perf & see if this fixes it.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Aug 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2e47fb825382917ced309f9219ca263d284e5adf

commit 2e47fb825382917ced309f9219ca263d284e5adf
Author: Ned Nguyen <nednguyen@google.com>
Date: Fri Aug 11 02:07:02 2017

[blink_perf] Handle case of missing test results

When a blink perf test result is empty, blink_perf benchmarks will hit an
unrecoverable exception due to it trying to add an empty list to
list_of_scalar_values.

This makes the failure recoverable by throws legacy_page_test.MeasurementFailure
when test results is empty.

Before this, running this will fail only one time:
./tools/perf/run_benchmark blink_perf.bindings --browser=reference --story-filter=custom-iterator.html --pageset-repeat=3

After this fix, the command above will fail 3 times.

TBR=charliea@chromium.org, eyaich@chromium.org,

Bug:  754514 
Change-Id: I06b8ae4f60fde1876294e17dba9ce239fa87a4a8
Reviewed-on: https://chromium-review.googlesource.com/611410
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#493645}
[modify] https://crrev.com/2e47fb825382917ced309f9219ca263d284e5adf/tools/perf/benchmarks/blink_perf.py

Status: Fixed (was: Assigned)
It's yellow now: https://build.chromium.org/p/chromium.perf/builders/Win%207%20x64%20Perf/builds/1036 :D

Sign in to add a comment