Unable to get perf profiling data on Android |
|||||
Issue descriptionI have a Nexus 4 running occam-userdebug LMY48T (freshly flashed), and I have been unable to get it to produce perf profiles using Telemetry. I've tried following these directions: https://www.chromium.org/developers/telemetry/profiling Adjusting for the fact that Android now supports GN instead of GYP, I built with these gn args: $ cat out/Android-profiling/args.gn target_os = "android" target_cpu = "arm" is_component_build = false is_debug = false symbol_level = 1 use_goma = true goma_dir = "/src/goma" enable_profiling = true I've confirmed that this does add -fno-omit-frame-pointer to the cflags. I've then tried installing the ChromePublic apk (which works), and running a number of variants on: $ tools/perf/run_benchmark --browser=android-chromium blink_perf.dom --story-filter=select-long-word --profiler=perf But the profiles I get back always complain about having no samples when I use the provided invocation of perfhost_trusty (regardless of which process I choose, though I'm interested in the renderer). However, the files are several hundred kilobytes large. I've also tried manually running the provided /data/local/tmp/profilers/perf via adb shell, constructing my own srcfs tree with the copy of libchrome_public.so from lib.unstripped, and lets perf run, claiming to have many samples, but the data I get back is garbage (all of the time spent in code that shouldn't run at all in this test, call stacks going through 0x0, etc.). How does one get valid profiling data off an ARM Android device?
,
May 3 2016
adb_profile_chrome is the main way to do this
,
May 3 2016
adb_profile_chrome has also failed to get plausible results for me. 1. It fails to find libchrome_public.so (even if I specify $CHROMIUM_OUT_DIR). It seems to be expect the gyp layout of the out/ dirs, even though Clank has already deprecated support for gyp in favour of gn. 2. If I help it along (cp out/Android-profiling/lib.unstripped/libchrome_public.so /tmp/jbroman-perf-symfs/data/app/org.chromium.chrome-1/lib/arm/), I get the same implausible results as I did from invoking perf myself: https://gist.github.com/jeremyroman/b906d6d55b57c5e37ac15061050f94a9 It's simply implausible to me that cricket::SrtpFilter::SetRtpParams is responsible for most of the CPU time in a DOM benchmark. I'm not sure if the samples are incorrect, or whether something is causing the host perf to map the symbols incorrectly. Any idea?
,
May 3 2016
Oh, and I forgot to add, but I've verified that these are matching libraries (i.e. they have the same .text section, but obviously the unstripped one was pushed to device): $ adb pull /data/app/org.chromium.chrome-1/lib/arm/libchrome_public.so 7916 KB/s (41780796 bytes in 5.154s) $ readelf -p.text libchrome_public.so | sha1sum ac42d2135c5a2786d03d218a9f5b9ca3fed23c19 - $ readelf -p.text out/Android-profiling/lib.unstripped/libchrome_public.so | sha1sum ac42d2135c5a2786d03d218a9f5b9ca3fed23c19 -
,
May 3 2016
(ahem, obviously the *stripped* one was pushed to device)
,
May 3 2016
Primiano, any idea if something has changed in address symbolization in the last ~six months that might confuse perf?
,
May 3 2016
Is enable_profiling=true disabling the crazylinker relro packing? If not you have your answer there. Relro packing breaks any assumption on ELF memory layout that all the existing tools made in the last 20 years of *nix (context in Issue 499747 and related).
,
May 3 2016
Yup, looks like GN doesn't properly turn that off when enable_profiling=true. Seems easy to fix; trying that now.
,
May 3 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c12f16c40c85acfe5bd1265c21a558f2b5a0de08 commit c12f16c40c85acfe5bd1265c21a558f2b5a0de08 Author: jbroman <jbroman@chromium.org> Date: Tue May 03 18:44:23 2016 Disable custom Android linker when profiling is on (gn build). The gyp build disables the relocation packing linker in this case: https://chromium.googlesource.com/chromium/src/+/2adf3ac9dd/build/java_apk.gypi#452 BUG= 608414 Review-Url: https://codereview.chromium.org/1947633002 Cr-Commit-Position: refs/heads/master@{#391311} [modify] https://crrev.com/c12f16c40c85acfe5bd1265c21a558f2b5a0de08/base/android/linker/config.gni
,
May 4 2016
,
May 5 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6a3ee8ff040e7fd5714eb73b44b58245612f930e commit 6a3ee8ff040e7fd5714eb73b44b58245612f930e Author: catapult-deps-roller <catapult-deps-roller@chromium.org> Date: Thu May 05 03:20:12 2016 Roll src/third_party/catapult/ 6c4d3dbd3..829c320e6 (2 commits). https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/6c4d3dbd3783..829c320e69d4 $ git log 6c4d3dbd3..829c320e6 --date=short --no-merges --format='%ad %ae %s' BUG= 608414 ,609001 TBR=catapult-sheriff@chromium.org Review-Url: https://codereview.chromium.org/1952943002 Cr-Commit-Position: refs/heads/master@{#391741} [modify] https://crrev.com/6a3ee8ff040e7fd5714eb73b44b58245612f930e/DEPS
,
May 5 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0680ab95bd5e7403a5b74c89780eadc6060482d9 commit 0680ab95bd5e7403a5b74c89780eadc6060482d9 Author: catapult-deps-roller <catapult-deps-roller@chromium.org> Date: Thu May 05 21:29:49 2016 Roll src/third_party/catapult/ 5235e17ba..0f4d6256a (1 commit). https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/5235e17baabc..0f4d6256ad5c $ git log 5235e17ba..0f4d6256a --date=short --no-merges --format='%ad %ae %s' BUG= 608414 TBR=catapult-sheriff@chromium.org Review-Url: https://codereview.chromium.org/1949253005 Cr-Commit-Position: refs/heads/master@{#391899} [modify] https://crrev.com/0680ab95bd5e7403a5b74c89780eadc6060482d9/DEPS
,
May 6 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/df8493cd370c1e49b3717c39383b6fba13c4f7e4 commit df8493cd370c1e49b3717c39383b6fba13c4f7e4 Author: catapult-deps-roller <catapult-deps-roller@chromium.org> Date: Fri May 06 01:32:53 2016 Roll src/third_party/catapult/ 5a77ca6c9..925381012 (5 commits). https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/5a77ca6c9cfd..9253810122c8 $ git log 5a77ca6c9..925381012 --date=short --no-merges --format='%ad %ae %s' BUG=609001, 608414 , 608414 TBR=catapult-sheriff@chromium.org Review-Url: https://codereview.chromium.org/1952123005 Cr-Commit-Position: refs/heads/master@{#391981} [modify] https://crrev.com/df8493cd370c1e49b3717c39383b6fba13c4f7e4/DEPS
,
May 10 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/349e3d7cbaf55d319b8c9f2ead66c56033245181 commit 349e3d7cbaf55d319b8c9f2ead66c56033245181 Author: jbroman <jbroman@chromium.org> Date: Tue May 10 19:15:17 2016 Update docs for profiling Content Shell on Android. This needed to be updated to reflect the move from GYP to GN, and to show the more modern alternative to manually invoking perf, which is having Catapult do it for you. BUG= 608414 Review-Url: https://codereview.chromium.org/1950383002 Cr-Commit-Position: refs/heads/master@{#392675} [modify] https://crrev.com/349e3d7cbaf55d319b8c9f2ead66c56033245181/docs/profiling_content_shell_on_android.md
,
May 16 2016
,
May 16 2016
Just to close the loop, what is the last state of things? Is enable_profiling forcing build in arm mode? Did we find the way to make perf work with thumb? Which sequence of -marm, -mthumb, -mapcs made sense at the end?
,
May 16 2016
enable_profiling doesn't force -marm in gn builds; I didn't go back and check what took precedence with profiling==1 on gyp builds. What gn currently spits out does make profiling work, but not stack traces. I tried several combinations of flags, but I wasn't able to get full stack traces to work (mostly I tried to stay in thumb mode, but I think I got a crash when I tried to leave it -- but I didn't investigate further). Opening another bug to get full stack traces on ARM might be useful if you think it's worth pursuing; regrettably I don't have more time to devote to this right now.
,
May 16 2016
np. I think +dskiba is looking at this for "heap" profiling purposes. I guess soon we'll have some king of profiling=true profileing_with_stack_traces=true :)
,
May 16 2016
Yeah, so for heap profiling I exposed arm_use_thumb variable to switch builds to ARM mode (crrev.com/1977533002). So I use both of them: enable_profiling = true # disables relocation packing, enables frame pointers arm_use_thumb = false # switches to ARM mode BTW, there was also a bug with enabling frame pointers in profiling builds - see crrev.com/1965143003
,
May 17 2016
I'll take a look at symbolization in this case.
,
Jun 1 2016
I was able to generate perf trace for Clankium with symbols on Nexus 6 (MRA58R), see https://drive.google.com/file/d/0B_Hmi138MnbJY3JxMkNBYkpHOXc/view?usp=sharing I needed: 1. crrev.com/2031643002 2. GN args: use_goma = true target_os = "android" is_debug = false is_clang = true arm_use_thumb = false enable_profiling = true 3. build/android/adb_profile_chrome --browser build --time 10 --perf Note that I'm compiling with Clank. GCC builds produced far less symbols. I also tried on Nexus 5X, and even though there were some symbols, trace also had many bogus PCs like 0x23. I suspect it has something to do with the fact that Nexus 5X is 64-bit device, and maybe profiling 32-bit Clank doesn't work as well as on 32-bit kernel.
,
Jun 3 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/138c1557c50620eb1bbc567e1938a658909ff061 commit 138c1557c50620eb1bbc567e1938a658909ff061 Author: catapult-deps-roller <catapult-deps-roller@chromium.org> Date: Fri Jun 03 02:24:56 2016 Roll src/third_party/catapult/ 857c0b7e4..3a308b4e6 (12 commits). https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/857c0b7e4db6..3a308b4e6d56 $ git log 857c0b7e4..3a308b4e6 --date=short --no-merges --format='%ad %ae %s' BUG= 352807 ,616832, 608414 , 616483 TBR=catapult-sheriff@chromium.org Review-Url: https://codereview.chromium.org/2033963002 Cr-Commit-Position: refs/heads/master@{#397578} [modify] https://crrev.com/138c1557c50620eb1bbc567e1938a658909ff061/DEPS |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by nedngu...@google.com
, May 2 2016