New issue
Advanced search Search tips

Issue 923125 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

"Results DashboardUpload Failure" performance_test_suite

Project Member Reported by crouleau@chromium.org, Jan 17 (5 days ago)

Issue description

Comment 1 by crouleau@chromium.org, Jan 17 (5 days ago)

Owner: benjhayden@chromium.org
Status: Assigned (was: Untriaged)
My guess is that this is caused by the catapult roll: https://chromium.googlesource.com/catapult.git/+log/4ec170a7bb08..79517a0b0327

https://chromium.googlesource.com/catapult.git/+/3ad33e96e78963037240b77028d773e02d827792

Comment 2 by crouleau@chromium.org, Jan 17 (5 days ago)

Error:

Traceback (most recent call last):
  File "C:\b\swarming\w\ir\cache\builder\src\third_party\catapult\tracing\bin\add_reserved_diagnostics", line 94, in <module>
    sys.exit(main())
  File "C:\b\swarming\w\ir\cache\builder\src\third_party\catapult\tracing\bin\add_reserved_diagnostics", line 82, in main
    dicts, names_to_values)
  File "C:\b\swarming\w\ir\cache\builder\src\third_party\catapult\tracing\tracing\value\diagnostics\add_reserved_diagnostics.py", line 168, in AddReservedDiagnostics
    histograms.ImportDicts(dicts_across_names)
  File "C:\b\swarming\w\ir\cache\builder\src\third_party\catapult\tracing\tracing\value\histogram_set.py", line 88, in ImportDicts
    hist = histogram_module.Histogram.FromDict(d)
  File "C:\b\swarming\w\ir\cache\builder\src\third_party\catapult\tracing\tracing\value\histogram.py", line 646, in FromDict
    hist._bins[i].FromDict(bin_dct)
  File "C:\b\swarming\w\ir\cache\builder\src\third_party\catapult\tracing\tracing\value\histogram.py", line 469, in FromDict
    diagnostic_map_dict))
  File "C:\b\swarming\w\ir\cache\builder\src\third_party\catapult\tracing\tracing\value\histogram.py", line 388, in FromDict
    dm.AddDicts(dct)
  File "C:\b\swarming\w\ir\cache\builder\src\third_party\catapult\tracing\tracing\value\histogram.py", line 392, in AddDicts
    for name, diagnostic_dict in dct.items():
AttributeError: 'NoneType' object has no attribute 'items'
ERROR:root:Error uploading perf result of system_health.memory_desktop.reference
Traceback (most recent call last):
  File "C:\b\swarming\w\ir\cache\builder\src\tools\perf\process_perf_results.py", line 425, in _upload_individual_benchmark
    return _upload_individual(*params)
  File "C:\b\swarming\w\ir\cache\builder\src\tools\perf\process_perf_results.py", line 414, in _upload_individual
    service_account_file, oj)
  File "C:\b\swarming\w\ir\cache\builder\src\tools\perf\process_perf_results.py", line 119, in _upload_perf_results
    return upload_results_to_perf_dashboard.main(args)
  File "C:\b\swarming\w\ir\cache\builder\src\tools\perf\core\upload_results_to_perf_dashboard.py", line 131, in main
    dashboard_json = _GetDashboardHistogramData(options)
  File "C:\b\swarming\w\ir\cache\builder\src\tools\perf\core\upload_results_to_perf_dashboard.py", line 83, in _GetDashboardHistogramData
    perf_dashboard_machine_group=options.perf_dashboard_machine_group)
  File "C:\b\swarming\w\ir\cache\builder\src\tools\perf\core\results_dashboard.py", line 183, in MakeHistogramSetWithDiagnostics
    subprocess.check_call(cmd)
  File "C:\b\swarming\w\ir\cipd_bin_packages\bin\Lib\subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['C:\\b\\swarming\\w\\ir\\cache\\vpython\\9649ae\\Scripts\\python.exe', 'C:\\b\\swarming\\w\\ir\\cache\\builder\\src\\third_party\\catapult\\tracing\\bin\\add_reserved_diagnostics', '--benchmarks', 'system_health.memory_desktop', '--bots', 'win-10-perf', '--builds', '1760', '--masters', 'ChromiumPerf', '--is_reference_build', 'true', '--log_urls_k', 'Buildbot stdio', '--log_urls_v', 'http://build.chromium.org/p/chromium/builders/win-10-perf/builds/1760/steps/system_health.memory_desktop/logs/stdio', '--build_urls_k', 'Build Status', '--build_urls_v', 'https://ci.chromium.org/p/chrome/builders/luci.chrome.ci/win-10-perf/1760', '--webrtc_revisions', '455d27c49abfff587babed348ae90f1aba75969e', '--chromium_commit_positions', '621080', '--chromium_revisions', 'None', '--v8_revisions', '677a6110e4c5e25b35d36aba42b36741c8ce1ea2', 'c:\\b\\swarming\\w\\ir\\tmp\\t\\tmpa_tzhe\\system_health.memory_desktop.reference\\merged_perf_results.json', '--output_path', 'c:\\b\\swarming\\w\\ir\\tmp\\t\\tmp0al3qu']' returned non-zero exit status 1


from here: https://logs.chromium.org/logs/chrome/buildbucket/cr-buildbucket.appspot.com/8924820682873154608/+/steps/performance_test_suite_on_Intel_GPU_on_Windows_on_Windows-10/0/logs/Merge_script_log/0

Comment 4 by benjhayden@google.com, Jan 18 (5 days ago)

I managed to repro even with the breakdown change rolled back.

I also found a similar error on the JS side in merge_histograms.

Current theory is that HistogramBin.addBin is somehow slipping undefined values into the "sampled stream" of sample diagnostic maps.
https://github.com/catapult-project/catapult/blob/master/tracing/tracing/value/histogram.html#L96

I don't think there have been any recent changes around there, so I'm not sure.

It might be worth it to try to simplify addBin/mergeSampledStreams regardless of whether it's necessary to fix this bug.
Project Member

Comment 5 by bugdroid1@chromium.org, Jan 18 (4 days ago)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c47537d42e1cfb7f3ff655a450fd0b026a15b33b

commit c47537d42e1cfb7f3ff655a450fd0b026a15b33b
Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Date: Fri Jan 18 11:33:56 2019

Roll src/third_party/catapult a0e5e3328cd7..8e77731b1363 (6 commits)

https://chromium.googlesource.com/catapult.git/+log/a0e5e3328cd7..8e77731b1363


git log a0e5e3328cd7..8e77731b1363 --date=short --no-merges --format='%ad %ae %s'
2019-01-17 crouleau@chromium.org Revert "Stop splitting logcat output on carriage returns."
2019-01-17 crouleau@chromium.org Revert "Ignore zeros in Breakdowns."
2019-01-17 uwyiming@google.com Add an option in WPR replay to prefer previously unmatched responses.
2019-01-17 crouleau@chromium.org [Telemetry] Cleanup: Remove unused argument from FindAllBrowserTypes
2019-01-17 uwyiming@google.com Adding uwyiming (myself) as an owner to WPR.
2019-01-17 skym@chromium.org Stop splitting logcat output on carriage returns.


Created with:
  gclient setdep -r src/third_party/catapult@8e77731b1363

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG=chromium:919223,chromium:919223,chromium:923125, chromium:922734 ,chromium:922828,chromium:919223
TBR=sullivan@chromium.org

Change-Id: I7fc110d1697c8624cf391c634fa26204b3bef617
Reviewed-on: https://chromium-review.googlesource.com/c/1419365
Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#624077}
[modify] https://crrev.com/c47537d42e1cfb7f3ff655a450fd0b026a15b33b/DEPS

Comment 6 by crouleau@chromium.org, Jan 18 (4 days ago)

Comment 7 by crouleau@chromium.org, Jan 18 (4 days ago)

Labels: -Pri-1 Pri-2

Comment 8 by benjhayden@chromium.org, Jan 19 (3 days ago)

Cc: benjhayden@chromium.org u...@chromium.org mythria@chromium.org
 Issue 923181  has been merged into this issue.

Sign in to add a comment