New issue
Advanced search Search tips

Issue 786091 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Oct 26
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature

Blocked on:
issue 791708



Sign in to add a comment

support kernel tracing (ftrace) on cast platform

Project Member Reported by spang@chromium.org, Nov 16 2017

Issue description

Traces on cast are missing kernel traces, particular the time slice data gathered from scheduler tracing. This makes traces much harder to interpret.

This can be fixed with a cast tracing agent that works similarly to the Chrome OS agent - ask a system level service to enable kernel tracing, and embed its output into the trace within the systemTraceEvents section.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Dec 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0436dc8ad15ff81b5c7cbdb408929836036688db

commit 0436dc8ad15ff81b5c7cbdb408929836036688db
Author: Michael Spang <spang@chromium.org>
Date: Fri Dec 01 20:25:27 2017

chromecast: Add tracing agent to collect kernel traces (ftrace)

Add a tracing agent and matching service so that tracing on Cast devices
can collect kernel traces. The implementation is based on the CrOS
tracing agent and debugd service.

This is very barebones at the moment (service only depends on //base) to
keep the size down. We don't have dbus, although we might be able to use
mojo to manage the socket.

BUG= 786091 

Change-Id: I76157157506628004d812db3f452a486b2962845
Reviewed-on: https://chromium-review.googlesource.com/775557
Reviewed-by: Dmitry Gozman <dgozman@chromium.org>
Reviewed-by: dsinclair <dsinclair@chromium.org>
Reviewed-by: Luke Halliwell <halliwell@chromium.org>
Reviewed-by: Alex Sakhartchouk <alexst@chromium.org>
Commit-Queue: Michael Spang <spang@chromium.org>
Cr-Commit-Position: refs/heads/master@{#521036}
[modify] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/chromecast/BUILD.gn
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/chromecast/tracing/BUILD.gn
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/chromecast/tracing/OWNERS
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/chromecast/tracing/ftrace.cc
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/chromecast/tracing/ftrace.h
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/chromecast/tracing/system_tracer.cc
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/chromecast/tracing/system_tracer.h
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/chromecast/tracing/system_tracing_common.cc
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/chromecast/tracing/system_tracing_common.h
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/chromecast/tracing/tracing_service_main.cc
[modify] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/content/browser/BUILD.gn
[modify] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/content/browser/tracing/DEPS
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/content/browser/tracing/cast_tracing_agent.cc
[add] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/content/browser/tracing/cast_tracing_agent.h
[modify] https://crrev.com/0436dc8ad15ff81b5c7cbdb408929836036688db/content/browser/tracing/tracing_controller_impl.cc

Project Member

Comment 2 by bugdroid1@chromium.org, Dec 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/466c589014e6b6f60ba3754c896bc5029c0e9c96

commit 466c589014e6b6f60ba3754c896bc5029c0e9c96
Author: Michael Spang <spang@chromium.org>
Date: Mon Dec 04 19:51:16 2017

chromecast: trace.py: Switch to new tracing config style

The tracing request format currently in use is deprecated and does not
support enabling some features such as system tracing. Switch to the new
format. This requires us to tokenize the category filter ourselves;
this is done using the same algorithm as chrome.

This also fixes a regression where the category filter was set to '"*"'
(double quotes included) which does not work properly, disabling all
events. We should use the empty string, which simply means to apply the
defaults. The android backend appears to be prepared to handle this as
well (mapping it to "_DEFAULT_CHROME_CATEGORIES").

BUG= 786091 

Change-Id: Ic60a3db7f4decc5ee046ce50bfc58d3c22211558
Reviewed-on: https://chromium-review.googlesource.com/804857
Reviewed-by: Luke Halliwell <halliwell@chromium.org>
Commit-Queue: Michael Spang <spang@chromium.org>
Cr-Commit-Position: refs/heads/master@{#521417}
[modify] https://crrev.com/466c589014e6b6f60ba3754c896bc5029c0e9c96/chromecast/tools/trace.py
[modify] https://crrev.com/466c589014e6b6f60ba3754c896bc5029c0e9c96/chromecast/tools/tracinglib.py

Comment 3 by spang@chromium.org, Dec 4 2017

Blocking: 791708

Comment 4 by spang@chromium.org, Dec 4 2017

Blocking: -791708

Comment 5 by spang@chromium.org, Dec 4 2017

Blockedon: 791708
Project Member

Comment 6 by bugdroid1@chromium.org, Dec 5 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/db45aa6f63f1b2ab83a5594ab8c6928b72d1e878

commit db45aa6f63f1b2ab83a5594ab8c6928b72d1e878
Author: Michael Spang <spang@chromium.org>
Date: Tue Dec 05 18:22:15 2017

chromecast: trace.py: Add system tracing support

This adds supports for system tracing via trace.py. It is disabled by
default but can be enable via the --systrace parameter. Requesting
system tracing does no harm on devices where it is not available.

This requires switching the way data is read back over the websocket, as
the "ReportEvents" style cannot return data from secondary tracing agents
such as CastTracingAgent.

BUG= 786091 

Change-Id: I8b5b39c047ae03073d90d462a4febf1770a8ccd5
Reviewed-on: https://chromium-review.googlesource.com/804559
Commit-Queue: Michael Spang <spang@chromium.org>
Reviewed-by: Luke Halliwell <halliwell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#521750}
[modify] https://crrev.com/db45aa6f63f1b2ab83a5594ab8c6928b72d1e878/chromecast/tools/trace.py
[modify] https://crrev.com/db45aa6f63f1b2ab83a5594ab8c6928b72d1e878/chromecast/tools/tracinglib.py

Project Member

Comment 7 by bugdroid1@chromium.org, Dec 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3e66922c5410072d0ec6a2a64e9a19c5c713263b

commit 3e66922c5410072d0ec6a2a64e9a19c5c713263b
Author: Michael Spang <spang@chromium.org>
Date: Wed Dec 06 19:04:23 2017

chromecast: tracing: Add system tracing categories to category list

This allows the tracing infrastructure to query the list of categories we
recognize, so they are shown in the UI.

BUG= 786091 

Change-Id: I6f2fa08855944fd0e0c04f4431704477be551706
Reviewed-on: https://chromium-review.googlesource.com/811587
Reviewed-by: dsinclair <dsinclair@chromium.org>
Commit-Queue: Michael Spang <spang@chromium.org>
Cr-Commit-Position: refs/heads/master@{#522150}
[modify] https://crrev.com/3e66922c5410072d0ec6a2a64e9a19c5c713263b/content/browser/tracing/cast_tracing_agent.cc

Project Member

Comment 8 by bugdroid1@chromium.org, Dec 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/aa48230b7d1fea72b117471f99759598f4639df8

commit aa48230b7d1fea72b117471f99759598f4639df8
Author: Michael Spang <spang@chromium.org>
Date: Thu Dec 07 20:01:15 2017

tracing: Fix length calculation in base64 module

Base64.DecodeToTypedArray() only returns the correct length if the
decoded length is one less than a multiple of three, such as in the test
string "hello". The other two cases either overestimate or underestimate
the length by 1. The empty string results in a length of -1.

One out of every three traces should have lost its last byte in the
trace2html importer because of this. Hopefully it was whitespace.

Fix it by making all lengths exact.

BUG= chromium:786091 

Change-Id: I9a88a4e2c7ed78ece782b91d1c90cb66817d259c
Reviewed-on: https://chromium-review.googlesource.com/813056
Reviewed-by: lpy <lpy@chromium.org>
Commit-Queue: Michael Spang <spang@chromium.org>

[modify] https://crrev.com/aa48230b7d1fea72b117471f99759598f4639df8/tracing/tracing/base/base64.html
[modify] https://crrev.com/aa48230b7d1fea72b117471f99759598f4639df8/tracing/tracing/base/base64_test.html

Project Member

Comment 9 by bugdroid1@chromium.org, Dec 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8188f965af68a034e5aa52ec6292f59353530c6d

commit 8188f965af68a034e5aa52ec6292f59353530c6d
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Fri Dec 08 14:15:23 2017

Roll src/third_party/catapult/ 16753e063..3572e89d9 (9 commits)

https://chromium.googlesource.com/catapult.git/+log/16753e0633fc..3572e89d92c2

$ git log 16753e063..3572e89d9 --date=short --no-merges --format='%ad %ae %s'
2017-12-08 nednguyen Revert "Remove platform_backend's DidStartBrowser, DidCreateBrowser and WillCloseBrowser hooks"
2017-12-08 perezju [Telemetry] Cleanup cros_forwarder
2017-12-07 rnephew [Telemetry/TADA] Add trailing comment support to expectations_parser.py
2017-12-07 rnephew [Telemetry/TADA] Make benchmark_runner and story_runner use TADA expectations.
2017-12-07 simonhatch Dashboard - Trim histogram level diagnostics
2017-12-06 spang tracing: Fix length calculation in base64 module
2017-12-07 eakuefner [TBMv2] Define Diagnostic.__ne__ in Python
2017-12-06 simonhatch Dashboard - Set name correctly when saving SparseDiagnostic
2017-12-07 nednguyen Remove platform_backend's DidStartBrowser, DidCreateBrowser and WillCloseBrowser hooks

Created with:
  roll-dep src/third_party/catapult
BUG= 786091 


The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I1dc1361f455443f916ba30ba8c0d7d804d3ead9b
Reviewed-on: https://chromium-review.googlesource.com/817094
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#522776}
[modify] https://crrev.com/8188f965af68a034e5aa52ec6292f59353530c6d/DEPS

Project Member

Comment 10 by bugdroid1@chromium.org, Dec 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/cb66d9d95233c2e1a399a00e0084426334db1c30

commit cb66d9d95233c2e1a399a00e0084426334db1c30
Author: Michael Spang <spang@chromium.org>
Date: Fri Dec 15 22:07:39 2017

tracing: Add support for system tracing via chrome://inspect?tracing

Tracing via remote debugging (chrome://inspect) cannot support system
tracing or any other secondary tracing agents supported by chrome
tracing. The option is shown in the UI but the collected system trace
events will not be visible in the returned data.

This is due to the limitations of the devtools protocol's "ReportEvents"
transfer mode, which streams individual chrome trace events. System
tracing events are embedded into the trace.json format as a blob of text,
not the chrome event format.

The "ReturnAsStream" transport delivers the full trace.json including any
embedded system trace events, so change the inspector tracing controller
to prefer this mode. This will ensure feature parity between features
supported by chrome://tracing and chrome://inspect?tracing going forward.

Also opt into compression if it is available, as compressed traces are
typically 1/10th the size of uncompressed ones.

This is backwards compatible with devtools endpoints that do not support
gzip, as well as old endpoints that do not support the "ReturnAsStream"
transfer mode.

BUG= chromium:786091 

Change-Id: I8fa200b371d5237f62140c88160cd250a7c84fed
Reviewed-on: https://chromium-review.googlesource.com/815731
Commit-Queue: Michael Spang <spang@chromium.org>
Reviewed-by: Ethan Kuefner <eakuefner@chromium.org>

[modify] https://crrev.com/cb66d9d95233c2e1a399a00e0084426334db1c30/tracing/trace_viewer.gypi
[modify] https://crrev.com/cb66d9d95233c2e1a399a00e0084426334db1c30/tracing/tracing/ui/extras/about_tracing/inspector_tracing_controller_client_test.html
[modify] https://crrev.com/cb66d9d95233c2e1a399a00e0084426334db1c30/tracing/tracing/ui/extras/about_tracing/inspector_tracing_controller_client.html
[modify] https://crrev.com/cb66d9d95233c2e1a399a00e0084426334db1c30/tracing/tracing_examples/chrome_inspect_test_shell.html
[modify] https://crrev.com/cb66d9d95233c2e1a399a00e0084426334db1c30/tracing/tracing/ui/extras/about_tracing/profiling_view.html
[modify] https://crrev.com/cb66d9d95233c2e1a399a00e0084426334db1c30/tracing/tracing/ui/extras/about_tracing/inspector_connection.html
[modify] https://crrev.com/cb66d9d95233c2e1a399a00e0084426334db1c30/tracing/tracing/ui/extras/about_tracing/tracing_controller_client.html
[add] https://crrev.com/cb66d9d95233c2e1a399a00e0084426334db1c30/tracing/tracing/ui/extras/about_tracing/devtools_stream.html

Project Member

Comment 11 by bugdroid1@chromium.org, Dec 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9161f6b9435b297e25da90ddf6d226b9fd9bbb34

commit 9161f6b9435b297e25da90ddf6d226b9fd9bbb34
Author: Michael Spang <spang@chromium.org>
Date: Sat Dec 16 20:02:11 2017

chromecast: trace.py: Turn on system tracing by default

Now that we have compression enabled, system tracing doesn't cause
flakes due to filling up /tmp, and we should be able to safely turn
it on by default.

Also increase the websocket timeout, since very large traces might take
more than 10s to compress.

BUG= 786091 
TEST=trace.py

Change-Id: I65726a723eb4b28466f4bff0dd818db916165227
Reviewed-on: https://chromium-review.googlesource.com/830916
Reviewed-by: Luke Halliwell <halliwell@chromium.org>
Commit-Queue: Michael Spang <spang@chromium.org>
Cr-Commit-Position: refs/heads/master@{#524612}
[modify] https://crrev.com/9161f6b9435b297e25da90ddf6d226b9fd9bbb34/chromecast/tools/trace.py

Project Member

Comment 12 by bugdroid1@chromium.org, Feb 1 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4942a898972b932e92c9bca1a2216e280e48d255

commit 4942a898972b932e92c9bca1a2216e280e48d255
Author: Michael Spang <spang@google.com>
Date: Thu Feb 01 00:40:40 2018

chromecast: tracing: Add support for tracing on older kernels

The tracing control directory was originally part of the debug filesystem
at /sys/kernel/debug/tracing. We have to support this location in order
for tracing to work on older kernels.

Bug:  786091 
Test: chromecast/tools/trace.py

Change-Id: Iccde78471201ab0bf9f4577057c28935f0909ce3
Reviewed-on: https://chromium-review.googlesource.com/896625
Reviewed-by: Alex Sakhartchouk <alexst@chromium.org>
Commit-Queue: Michael Spang <spang@chromium.org>
Cr-Commit-Position: refs/heads/master@{#533493}
[modify] https://crrev.com/4942a898972b932e92c9bca1a2216e280e48d255/chromecast/tracing/ftrace.cc

Status: Fixed (was: Assigned)

Sign in to add a comment