Thanks maajid@! Will look into that a bit later once we figure out other (currently more important) coverage-related issues.

I had a similar issue with a fuzzer I'm writing for third_party/WebKit/Source/core/css/parser/CSSParser.cpp:ParseStyleSheet

llvm-profdata warns of 135 mismatched functions and the coverage report doesn't have info on any important files. 
It doesn't even have info for CSSParser.cpp even though I have confirmed I am executing ParseStyleSheet in that file, via print statements and gdb.

I disassembled CSSParser.cpp:ParseStyleSheet and found what I think is the instrumentation from clang coverage

   0x00007fffd335c578 <+72>:    mov    0xbe78999(%rip),%r8        # 0x7fffdf1d4f18
   0x00007fffd335c57f <+79>:    add    $0x1,%r8
   0x00007fffd335c583 <+83>:    mov    %r8,0xbe7898e(%rip)        # 0x7fffdf1d4f18

This is unsurprising since I was also able to confirm that the file is being compiled with the proper flags "-fprofile-instr-generate -fcoverage-mapping"

I found a similar issue in LLVM bug tracker, added our information to it: https://bugs.llvm.org/show_bug.cgi?id=33502

Correction: it is llvm-cov that complains of the mismatched functions, not llvm-profdata

Changed the summary as it doesn't seem to be specific for target_os=chromeos.

Using gdb on llvm-cov, I was able to find that the issue is caused by hash mismatches (https://github.com/llvm-mirror/llvm/blob/master/lib/ProfileData/Coverage/CoverageMapping.cpp#L220) rather than the alternative (unequal region sizes).

Jonathan, please check out the comment by Vedant: https://bugs.llvm.org/show_bug.cgi?id=33502#c2

Let's try it once you are back.

I have built ToT llvm and clang and used them to build my fuzzer and generate the coverage report, and the issue remains.

After using the -dump option with llvm-cov, it complains that 148 functions have mismatched data and then prints 148 lines similar to the one below:

hash-mismatch: No profile record found for '_ZN7logging11CheckGEImplEiiPKc' with hash = 0x0

In all of these cases, the hash was 0x0 and the function name was some kind of mangled name (like the one above) beginning with the prefix "_ZN"

Interestingly one function that I know is hit (ParseSheet) is not in the list of mismatched functions. This is true whether I use the -sparse option with llvm-profdata or not.

When I run the following command:
llvm-profdata show *.profraw -o output -all-functions

The following lines are output (to the "output" file):
...
  _ZN5blink9CSSParser10ParseSheetEPKNS_16CSSParserContextEPNS_18StyleSheetContentsERKN3WTF6StringEb:
    Hash: 0x0000000000000000
    Counters: 1
    Function count: 1065
...


This function count is completely correct (1065 is the number of executions that libFuzzer reports it has executed when running on this corpus and the function is hit once per execution).

I'm going to look into why the hash is reported as 0x0 here and why this function doesn't seem to be included in the merged profdata when its hit count is nonzero and other functions with their hash as 0x0 are included.

Adding one more data point here. This issue is causing the coverage data completely useless for Chrome unit test targets, for example, content_unittests, almost all the important files are missing from the report, so right now, this is a blocking issue for making code coverage useful for Chrome.

I've also tried to get coverage for content_unittests. I've built it with the following GN args:

is_debug = false
use_clang_coverage = true
enable_nacl = false
ffmpeg_branding = "ChromeOS"
use_libfuzzer = true
proprietary_codecs = true
pdf_enable_xfa = true


And disabled almost all tests that have been failing on my desktop when running in a single process mode:



testing/libfuzzer/coverage.py --command "out/coverage/content_unittests --single-process-tests --gtest_filter=-*ServiceWorkerResourceStorageDiskTest*:*SiteInstanceTest*:*PepperGamepadHostTest*:*HostVarTrackerTest*:*V8VarConverterTest*:*WebRtcEventlogHostTest*:*MediaStreamDispatcherTest*:*MediaStreamVideoCapturerSourceTest*:*MediaStreamVideoSourceTest*:*MediaStreamRemoteVideoSourceTest*:*MediaStreamTrackMetricsTest*:*MediaStreamVideoWebRtcSinkTest*:*HTMLAudioElementCapturerSourceTest*:*HTMLVideoElementCapturerSourceTest*:*MediaRecorderHandlerTest*:*PepperToVideoTrackAdapterTest*:*WebContentsVideoCaptureDeviceTest*:*CursorRendererAuraTest*:*SpeechRecognizerImplTest*:*AppCacheRequestHandlerTest*:*AppCacheUpdateJobTest*:*BlobURLRequestJobTest*:*CacheStorageCacheTest*:*CacheStorageManagerTests*:*CacheStorageQuotaClientTests*:*DownloadFileTestWithRename*:*Success/DownloadItemDestinationUpdateRaceTest*:*Failure/DownloadItemDestinationUpdateRaceTest*:*IndexedDBTransactions/IndexedDBTransactionTestMode*:*DetachableResourceHandlerTest*:*MojoAsyncResourceHandlerWithAllocationSizeTest*:*RedirectToFileResourceHandlerTest*:*URLLoaderFactoryImplTest*:*WebContentsAudioInputStreamTest*:*MediaInternalsAudioLogTest*:*GestureEventQueueWithSourceTest*:*MouseWheelEventQueueTest*:*SyntheticGestureControllerTestWithParam*:*MediaDevicesDispatcherHostTest*:*ServiceWorkerContextRecoveryTest*:*ServiceWorkerDataPipeReaderTest*:*SimpleURLLoaderTest*:*NavigationSimulatorTest*:*AudioTrackRecorderTest*" --output content_unittests_report



Unfortunately, it also didn't work, though failed in a different way:

$ llvm-profdata merge -sparse content_unittests_report/coverage.profraw -o content_unittests_report/coverage.profdata
error: content_unittests_report/coverage.profraw: Malformed instrumentation profile data

I don't think "--single-process-tests" is supposed to be used to run tests, as the help message says, it is useful for debugging a specific test in a debugger.

Have you tried ./content_unittests --test-launcher-jobs=1 ?

Nice, that worked! It took about 5-6 hours to generate the report though, but it happened!

maajid@, have you used is_component_build=true in the GN args?

liaoyuke@, same question for you.

Jonathan has discovered that component build was the root cause for his case. We suspect that it might be the cause of your issues as well. Coverage for component builds is quite complicated, see  issue 771403  for more details.


Also, I have coverage report for the content_unittests run: go/chrome-content-unittests-coverage-report

Yuke, could you please check if you can see the files that were missing when you tried doing that locally?

Jonathan has uploaded a CL to prevent coverage being used with component builds: https://chromium-review.googlesource.com/c/chromium/src/+/714279

maajid@ and liaoyuke@, could you please answer my questions in c#20? Thanks!

Looks like the missing files are presented, but I don't remember I ever used is_component_build=true, can you share the GN args your used to run?

I'm using the following GN flags:

is_debug = false
use_clang_coverage = true
is_component_build = false
enable_nacl = false
ffmpeg_branding = "ChromeOS"
use_libfuzzer = true
proprietary_codecs = true
pdf_enable_xfa = true


I assume that different build targets may suppose different default value of is_component_build, this is why it doesn't always work well unless you specify it explicitly as false.

Closing this since neither maajid@ nor liaoyuke@ complain anymore.

Thanks Jonathan for the investigation and for the CL to prevent that in future!

Left a comment on the upstream bug as well: https://bugs.llvm.org/show_bug.cgi?id=33502#c7