$ equery-$B w dev-util/perf
/mnt/host/source/src/third_party/chromiumos-overlay/dev-util/perf/perf-3.18-r3.ebuild

perf-3.18.ebuild appears to have four local patches:

1) 3.18-Fix-build-id-matching-on-vmlinux.patch
2) 3.18-Preparation-for-compressed-kernel-modul.patch
3) separate-tools-and-tests.patch
4) 3.18-Fix-hugepage-text.patch

Status of patches WRT upstream:
1) drop - already part of 4.4 kernel.
2) drop - fixed in 3.19 kernel source tree
3) drop - already in 4.4 (128c32ed1866e)
4) forward port to upstream 4.4 source tree?
   (see https://chromium-review.googlesource.com/390488)

Mr Wizard! Help! :)

How do I update the local Manifest to point at the right linux-$V tar ball?

"emerge-$B dev-utils/perf" is still picking up 3.18 and I suspect it's because I need to update the Manifest as was done in "3.14 to 3.18" update:
    https://chromium-review.googlesource.com/319980


So far I have:
...src/third_party/chromiumos-overlay/dev-util/perf $ git status
On branch uprev-perf-ebuild
Your branch is up-to-date with 'cros/master'.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        deleted:    files/3.18-Fix-build-id-matching-on-vmlinux.patch
        deleted:    files/3.18-Preparation-for-compressed-kernel-modul.patch
        renamed:    files/3.18-Fix-hugepage-text.patch -> files/4.4-Fix-hugepage-text.patch
        deleted:    files/separate-tools-and-tests.patch
        deleted:    perf-3.18-r3.ebuild
        new file:   perf-4.4-r1.ebuild
        new file:   perf-4.4.ebuild

ps. yunlian + sonnyrao will have to fixup the (just renamed) 4.4-Fix-hugepage-text.patch once I upload the changes for review. Sorry - just trying to eliminate some of the "grunt" work for you.  I'm wondering why hugepage_text patch was applied to 3.18 while 4.4 branch has been available for more than a month now. As noted elsewhere, perf is expected to be backwards compatible with older kernels and it's considered a bug if it's not.

The patch 3.18-Fix-hugepage-text.patch applies to perf 4.4 cleanly but I did not
do other tests, currently it could not build because of dependency issues.

Thanks yulian! That's good news. :)

Then the review should be "trivial", right? RIGHT?!! :D Somehow I can't convince myself but we'll see. ;)

(sorry s/yulian/yunlian/)

you can run `ebuild ... manifest` inside the sdk to download & generate the Manifest file

Ah ok - Thanks! I also realized I could copy that line from Gentoo :)
   https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-util/perf/Manifest


Next problem:
(cr) (uprev-perf-ebuild) grundler@firesword /mnt/host/source/src/third_party/chromiumos-overlay/dev-util/perf $ emerge-gale dev-util/perf
!!! CONFIG_PROTECT is empty for '/build/gale/'
Calculating dependencies... done!

emerge: there are no ebuilds to satisfy ">=sys-kernel/linux-headers-4.4" for /build/gale/.
(dependency required by "dev-util/perf-4.4-r1::chromiumos" [ebuild])
(dependency required by "dev-util/perf" [argument])


Is something higher level needed here to include linux-headers-4.4 for _all_ boards?
(not just gale)

bumping linux-headers to the latest (like 4.10) shouldn't be a prob i don't think

I've uploaded my current patchset for perf-4.4:
    https://chromium-review.googlesource.com/446668

... and I'm thinking I being too short sighted: should I be moving to perf-4.9. Thoughts?

perf tends to change a lot, so it's good that you finished 4.4
It's worth trying to see if 4.4 -> 4.9 is easy or not and if it is, then by all means go to 4.9, if not, then no worries

1. is this issue still being worked on? Thanks. (see https://chromium-review.googlesource.com/446668 for OOO comment)

2. the patch does not work for the platform this was originally reported on, the issue being a missing ebuild for the 4.4 headers which is similar to what was reported in #8.

"...I'll check to see if we can bump this quickly..." It looks to like the primary engineer for the patch on this issue has gone on vacation for several weeks.  We need support for the originally reported platform. The patches are configured for 'cave' and do not work for our platform.  Thanks.

Sorry Colex for the delay on this one. I will increase priority of this one. 

I'm back from OOO and haven't seen any updates since beginning of March. Should I continue with exist CL that I previously uploaded?
   https://chromium-review.googlesource.com/446668

If yes, please reassign to me.

I've updated this series of CLs:

remote: New Changes:        
remote:   https://chromium-review.googlesource.com/461388 perf: add hugetable patch to -4.4 ebuild        
remote:   https://chromium-review.googlesource.com/461389 perf: only install tools, not tests, etc        
remote:
remote:
remote: Updated Changes:        
remote:   https://chromium-review.googlesource.com/446668 perf: build from 4.4 kernel source

CL 446668 is the main one and "emerge-gale dev-util/perf" is not getting past the "detecting features stage" which fails with:
   No gnu/libc-version.h found

This is actually a lie. I'm pretty sure gnu/libc-version.h is present in the include paths.

But all the "feature detect" tests in 4.4 kernel source tree are failing to compile it seems. (full output appended below)
 
(cr) grundler@firesword /build/gale/tmp/portage/dev-util/perf-4.4-r1/work/linux-4.4/tools/build/feature $ cat test-glibc.make.output
gcc.real: error: unrecognized command line option '-mfpu=neon'
gcc.real: error: unrecognized command line option '-mfloat-abi=hard'
gcc.real: error: unrecognized command line option '-clang-syntax'

CC is set to armv7a-cros-linux-gnueabi-clang. I don't understand why clang isn't happy with these options. Help?

(cr) ((0870b3c...)) grundler@firesword ~/trunk/src/scripts $ emerge-gale dev-util/perf
!!! CONFIG_PROTECT is empty for '/build/gale/'
Calculating dependencies... done!

>>> Emerging (1 of 1) dev-util/perf-4.4-r1::chromiumos for /build/gale/
 * linux-4.4.tar.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                 [ ok ]
 * Running stacked hooks for pre_pkg_setup
 *    sysroot_build_bin_dir ...                                          [ ok ]
 * Determining the location of the kernel source code
 * Found kernel source directory:
 *     /mnt/host/source/src/third_party/kernel/v3.18
 * Found kernel object directory:
 *     /build/gale/usr/src/linux
 * Found sources for kernel version:
 *     3.18.0
 * Checking for suitable kernel configuration options...                 [ ok ]
 * Running stacked hooks for pre_src_unpack
 *    python_multilib_setup ...                                          [ ok ]
>>> Unpacking source...
>>> Unpacking linux-4.4.tar.xz (tools/arch tools/build tools/include tools/lib tools/perf tools/scripts include lib arch/*/lib) to /build/gale/tmp/portage/dev-util/perf-4.4-r1/work
>>> Source unpacked in /build/gale/tmp/portage/dev-util/perf-4.4-r1/work
 * Running stacked hooks for post_src_unpack
 *    asan_init ...                                                      [ ok ]
>>> Preparing source in /build/gale/tmp/portage/dev-util/perf-4.4-r1/work/linux-4.4/tools/perf ...
 * Applying 3.18-Fix-hugepage-text.patch ...                             [ ok ]
>>> Source prepared.
>>> Configuring source in /build/gale/tmp/portage/dev-util/perf-4.4-r1/work/linux-4.4/tools/perf ...
>>> Source configured.
>>> Compiling source in /build/gale/tmp/portage/dev-util/perf-4.4-r1/work/linux-4.4/tools/perf ...
make -j48 V=1 CC=armv7a-cros-linux-gnueabi-clang AR=armv7a-cros-linux-gnueabi-ar LD=armv7a-cros-linux-gnueabi-ld CROSS_COMPILE=armv7a-cros-linux-gnueabi- prefix=/usr bindir_relative=bin 'EXTRA_CFLAGS=-O2 -O2 -pipe -march=armv7-a -mtune=cortex-a7 -mfpu=neon -mfloat-abi=hard -g -fno-exceptions -fno-unwind-tables   -fno-asynchronous-unwind-tables  -clang-syntax' ARCH=arm NO_DEMANGLE=no NO_GTK2=no NO_LIBAUDIT=no NO_LIBPERL=no NO_LIBPYTHON=no NO_LIBUNWIND=no NO_NEWT=no NO_LIBNUMA=no WERROR=0 -f Makefile.perf 

Auto-detecting system features:
...                         dwarf: [ OFF ]
...                         glibc: [ OFF ]
...                          gtk2: [ OFF ]
...                      libaudit: [ OFF ]
...                        libbfd: [ OFF ]
...                        libelf: [ OFF ]
...                       libnuma: [ OFF ]
...        numa_num_possible_cpus: [ OFF ]
...                       libperl: [ OFF ]
...                     libpython: [ OFF ]
...                      libslang: [ OFF ]
...                     libunwind: [ OFF ]
...            libdw-dwarf-unwind: [ OFF ]
...                          zlib: [ OFF ]
...                          lzma: [ OFF ]
...                     get_cpuid: [ OFF ]
...                           bpf: [ OFF ]

config/Makefile:263: *** No gnu/libc-version.h found, please install glibc-dev[el].  Stop.
 * ERROR: dev-util/perf-4.4-r1::chromiumos failed (compile phase):
 *   emake failed
 * 
 * If you need support, post the output of `emerge --info '=dev-util/perf-4.4-r1::chromiumos'`,
 * the complete build log and the output of `emerge -pqv '=dev-util/perf-4.4-r1::chromiumos'`.
 * The complete build log is located at '/build/gale/tmp/portage/logs/dev-util:perf-4.4-r1:20170328-230551.log'.
 * For convenience, a symlink to the build log is located at '/build/gale/tmp/portage/dev-util/perf-4.4-r1/temp/build.log'.
 * The ebuild environment file is located at '/build/gale/tmp/portage/dev-util/perf-4.4-r1/temp/environment'.
 * Working directory: '/build/gale/tmp/portage/dev-util/perf-4.4-r1/work/linux-4.4/tools/perf'
 * S: '/build/gale/tmp/portage/dev-util/perf-4.4-r1/work/linux-4.4/tools/perf'

>>> Failed to emerge dev-util/perf-4.4-r1 for /build/gale/, Log file:

>>>  '/build/gale/tmp/portage/logs/dev-util:perf-4.4-r1:20170328-230551.log'

 * Messages for package dev-util/perf-4.4-r1 merged to /build/gale/:

 * ERROR: dev-util/perf-4.4-r1::chromiumos failed (compile phase):
 *   emake failed
 * 
 * If you need support, post the output of `emerge --info '=dev-util/perf-4.4-r1::chromiumos'`,
 * the complete build log and the output of `emerge -pqv '=dev-util/perf-4.4-r1::chromiumos'`.
 * The complete build log is located at '/build/gale/tmp/portage/logs/dev-util:perf-4.4-r1:20170328-230551.log'.
 * For convenience, a symlink to the build log is located at '/build/gale/tmp/portage/dev-util/perf-4.4-r1/temp/build.log'.
 * The ebuild environment file is located at '/build/gale/tmp/portage/dev-util/perf-4.4-r1/temp/environment'.
 * Working directory: '/build/gale/tmp/portage/dev-util/perf-4.4-r1/work/linux-4.4/tools/perf'
 * S: '/build/gale/tmp/portage/dev-util/perf-4.4-r1/work/linux-4.4/tools/perf'
(cr) ((0870b3c...)) grundler@firesword ~/trunk/src/scripts $ 

the error message says "gcc.real" which implies that some code somewhere is running `gcc`.  which is probably wrong.  you should look into the build and see if you can track that down.

Yeah, I saw the prefix but thought that was just a "label" for compiler output... it's not. :(

$KERNEL_SRC/tools/build/feature/Makefile has gcc hard coded:

 41 CC := $(CROSS_COMPILE)gcc -MD
 42 PKG_CONFIG := $(CROSS_COMPILE)pkg-config

I'll see if adding another patch to change ":=" to "?=" is sufficient.

But I'm not sure what to do about "-MD":
https://gcc.gnu.org/onlinedocs/gcc-6.3.0/gcc/Preprocessor-Options.html#Preprocessor-Options

"-MD is equivalent to -M -MF file, except that -E is not implied....generate a dependency output file as a side-effect of the compilation process."

Not sure why it would be needed here - will omit for now.

Can't omit the -MD: later stages are looking for .d files and completely failing to find them. :(

And I'm also seeing this:
warning: optimization level '-O6' is not supported; using '-O3' instead

I think this is the offending 4.4 Makefile:
   tools/perf/config/Makefile:  CFLAGS += -O6

but I will look more closely at this tomorrow.

you can probably run a sed to change the "$(CROSS_COMPILE)gcc" into "$(CC)"

you should also use sed to delete the -O6 line

Progress:
...
make -j48 V=1 CC=armv7a-cros-linux-gnueabi-clang AR=armv7a-cros-linux-gnueabi-ar LD=armv7a-cros-linux-gnueabi-ld CROSS_COMPILE=armv7a-cros-linux-gnueabi- prefix=/usr bindir_relative=bin 'EXTRA_CFLAGS=-O2 -O2 -pipe -march=armv7-a -mtune=cortex-a7 -mfpu=neon -mfloat-abi=hard -g -fno-exceptions -fno-unwind-tables   -fno-asynchronous-unwind-tables  -clang-syntax' ARCH=arm NO_DEMANGLE=no NO_GTK2=no NO_LIBAUDIT=no NO_LIBPERL=no NO_LIBPYTHON=no NO_LIBUNWIND=no NO_NEWT=no NO_LIBNUMA=no WERROR=0 -f Makefile.perf 

Auto-detecting system features:
...                         dwarf: [ on  ]
...                         glibc: [ on  ]
...                          gtk2: [ OFF ]
...                      libaudit: [ OFF ]
...                        libbfd: [ OFF ]
...                        libelf: [ on  ]
...                       libnuma: [ OFF ]
...        numa_num_possible_cpus: [ OFF ]
...                       libperl: [ OFF ]
...                     libpython: [ OFF ]
...                      libslang: [ OFF ]
...                     libunwind: [ OFF ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ on  ]
...                     get_cpuid: [ OFF ]
...                           bpf: [ OFF ]

config/Makefile:339: BPF API too old. Please install recent kernel headers. BPF support in 'perf record' is disabled.
config/Makefile:470: Python support disabled by user
config/Makefile:651: Your gcc lacks the __get_cpuid() builtin, disables support for auxtrace/Intel PT, please install a newer gcc

Lack get_cpuid() builtin seems like a P0 to me. Is it?


And despite include -MD parameter:
  armv7a-cros-linux-gnueabi-clang -Wp,-MD,fs/.fs.o.d,-MT,fs/fs.o -Wbad-function-cast -Wdeclaration-after-statement -Wformat-security -Wformat-y2k -Winit-self -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wno-system-headers -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-aliasing=3 -Wstrict-prototypes -Wswitch-default -Wswitch-enum -Wundef -Wwrite-strings -Wformat -O2 -O2 -pipe -march=armv7-a -mtune=cortex-a7 -mfpu=neon -mfloat-abi=hard -g -fno-exceptions -fno-unwind-tables   -fno-asynchronous-unwind-tables  -clang-syntax -ggdb3 -Wall -Wextra -std=gnu99  -O3 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fPIC -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D"BUILD_STR(s)=#s"   -c -o fs/fs.o fs/fs.c

I'm still getting missing .d files. :(
cat: fs/.fs.o.d: No such file or directory

(repeated for just about every .c file compiled)

you're building for arm, so the lack of x86 __get_cpuid support makes sense

for the BPF, is that still an issue w/linux-headers-4.4+ ?  those just got added to the tree ...

Doh! True! :) I expected the Makefile(s) to only check this for x86 variant builds though. Despite the itch to fix it, I will ignore for now.

Re BPF: Yes, I waited until the linux-headers-4.4 landed before continuing and the perf-4.4.ebuild DEPENDS    >=sys-kernel/linux-headers-4.4.

The missing *.d is the main problem though. Luis? Yunlian?

are the missing *.d files causing a build error, or just ugliness in the output ?  i've seen build systems that do not handle the *.d generation & inclusion in a single `make` run well.

i doubt this question is really for the toolchain guys either as it tends to be a problem with the build system rather than the preprocessor.

Yes, missing *.d files are causing a build error.

Would building with gcc be useful to rule out clang vs gcc or would there still be concerns about build system?

Doesn't the kernel build also use dependencies? My kernel is building with clang (have local patches). Let me see if there is some special handling done by kernel ebuilds.

I've repo upload again what I currently have. Output of emerge-gale attached.

Deleted: perf-4.4.emerge-output-with-clang.txt 116 KB

perf-4.4.emerge-output-with-clang.txt 116 KB View Download

perf-4.4 is now default:
   https://chromium-review.googlesource.com/c/446668/

I've verified basic functionality works on gale (ARM) and chell(x86) (both use chromeos-3.18 kernels).

Great. Do you know if this will be available in 59? The branch point is today.

R59 hasn't branched yet, so we don't know.  once it's been created, you can check:
https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+refs

I don't know. If it's not, I expect it would be trivial to cherry-pick into 59 branch since all the "hard work" (testing mostly) is done.