Add a code coverage summary export solution in llvm-cov |
|||||
Issue description
To generate code coverage breakdown per directory, we need to get the code coverage summary for files exercised, the desired format should look like following (possibly in JSON format):
{
"file1": {"total_lines": 100, "executed_lines": 50, "total_functions": 5, "executed_functions": 2, ...}
"file2": ...
...
}
The closes three existing solutions are:
1. Using "llvm-cov report" and parse the text output. The problem with this solution is that the format of the report any change any time, and once it does, our script breaks.
2. Using "llvm-cov show --format=html" and parse the index.html file. It has the same problem as in 1, and the format of the html output is even more likely to change.
3. Using "llvm-cov export", this provides what we need, but with tons of extra information, which makes it super slow to run.
Ideally, we should have llvm-cov provide a command to generate code coverage breakdown per directory, but in short term, I think it makes sense to work this around downstream, and then we need a more robust command to generate the information we need, something like "llvm-cov summary" or "llvm-cov export --summary".
,
Dec 11 2017
Sounds good, I'll take a closer look at adding an option to "llvm-cov export" soon.
,
Dec 11 2017
,
Dec 11 2017
Uploaded a CL: https://reviews.llvm.org/D41085 An example of output (I used some online JSON beautifier for local post-processing, the tool still write everything into a single line): $ ~/Projects/llvm/install/bin/llvm-cov export -summary-only -instr-profile=zlib_report/coverage.profdata out/coverage/zlib_uncompress_fuzzer { "version":"2.0.0", "type":"llvm.coverage.json.export", "data":[ { "files":[ { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/adler32.c", "summary":{ "lines":{ "count":100, "covered":62, "percent":62 }, "functions":{ "count":5, "covered":2, "percent":40 }, "instantiations":{ "count":5, "covered":2, "percent":40 }, "regions":{ "count":150, "covered":75, "notcovered":75, "percent":50 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/adler32_simd.c", "summary":{ "lines":{ "count":138, "covered":138, "percent":100 }, "functions":{ "count":1, "covered":1, "percent":100 }, "instantiations":{ "count":1, "covered":1, "percent":100 }, "regions":{ "count":21, "covered":21, "notcovered":0, "percent":100 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/contrib/tests/fuzzers/uncompress_fuzzer.cc", "summary":{ "lines":{ "count":8, "covered":8, "percent":100 }, "functions":{ "count":1, "covered":1, "percent":100 }, "instantiations":{ "count":1, "covered":1, "percent":100 }, "regions":{ "count":7, "covered":7, "notcovered":0, "percent":100 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/crc32.c", "summary":{ "lines":{ "count":178, "covered":0, "percent":0 }, "functions":{ "count":13, "covered":0, "percent":0 }, "instantiations":{ "count":13, "covered":0, "percent":0 }, "regions":{ "count":126, "covered":0, "notcovered":126, "percent":0 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/crc_folding.c", "summary":{ "lines":{ "count":381, "covered":0, "percent":0 }, "functions":{ "count":8, "covered":0, "percent":0 }, "instantiations":{ "count":8, "covered":0, "percent":0 }, "regions":{ "count":58, "covered":0, "notcovered":58, "percent":0 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/inffast.c", "summary":{ "lines":{ "count":222, "covered":170, "percent":76 }, "functions":{ "count":1, "covered":1, "percent":100 }, "instantiations":{ "count":1, "covered":1, "percent":100 }, "regions":{ "count":90, "covered":61, "notcovered":29, "percent":67 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/inffast.h", "summary":{ "lines":{ "count":0, "covered":0, "percent":0 }, "functions":{ "count":0, "covered":0, "percent":0 }, "instantiations":{ "count":0, "covered":0, "percent":0 }, "regions":{ "count":0, "covered":0, "notcovered":0, "percent":0 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/inffixed.h", "summary":{ "lines":{ "count":0, "covered":0, "percent":0 }, "functions":{ "count":0, "covered":0, "percent":0 }, "instantiations":{ "count":0, "covered":0, "percent":0 }, "regions":{ "count":0, "covered":0, "notcovered":0, "percent":0 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/inflate.c", "summary":{ "lines":{ "count":1028, "covered":618, "percent":60 }, "functions":{ "count":22, "covered":10, "percent":45 }, "instantiations":{ "count":22, "covered":10, "percent":45 }, "regions":{ "count":1203, "covered":696, "notcovered":507, "percent":57 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/inftrees.c", "summary":{ "lines":{ "count":266, "covered":264, "percent":99 }, "functions":{ "count":1, "covered":1, "percent":100 }, "instantiations":{ "count":1, "covered":1, "percent":100 }, "regions":{ "count":109, "covered":107, "notcovered":2, "percent":98 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/inftrees.h", "summary":{ "lines":{ "count":0, "covered":0, "percent":0 }, "functions":{ "count":0, "covered":0, "percent":0 }, "instantiations":{ "count":0, "covered":0, "percent":0 }, "regions":{ "count":0, "covered":0, "notcovered":0, "percent":0 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/names.h", "summary":{ "lines":{ "count":0, "covered":0, "percent":0 }, "functions":{ "count":0, "covered":0, "percent":0 }, "instantiations":{ "count":0, "covered":0, "percent":0 }, "regions":{ "count":0, "covered":0, "notcovered":0, "percent":0 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/uncompr.c", "summary":{ "lines":{ "count":56, "covered":50, "percent":89 }, "functions":{ "count":2, "covered":2, "percent":100 }, "instantiations":{ "count":2, "covered":2, "percent":100 }, "regions":{ "count":48, "covered":38, "notcovered":10, "percent":79 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/x86.c", "summary":{ "lines":{ "count":27, "covered":27, "percent":100 }, "functions":{ "count":2, "covered":2, "percent":100 }, "instantiations":{ "count":2, "covered":2, "percent":100 }, "regions":{ "count":8, "covered":8, "notcovered":0, "percent":100 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/zconf.h", "summary":{ "lines":{ "count":0, "covered":0, "percent":0 }, "functions":{ "count":0, "covered":0, "percent":0 }, "instantiations":{ "count":0, "covered":0, "percent":0 }, "regions":{ "count":0, "covered":0, "notcovered":0, "percent":0 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/zlib.h", "summary":{ "lines":{ "count":0, "covered":0, "percent":0 }, "functions":{ "count":0, "covered":0, "percent":0 }, "instantiations":{ "count":0, "covered":0, "percent":0 }, "regions":{ "count":0, "covered":0, "notcovered":0, "percent":0 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/zutil.c", "summary":{ "lines":{ "count":45, "covered":9, "percent":20 }, "functions":{ "count":5, "covered":2, "percent":40 }, "instantiations":{ "count":5, "covered":2, "percent":40 }, "regions":{ "count":32, "covered":3, "notcovered":29, "percent":9 } } }, { "filename":"/usr/local/google/home/mmoroz/Projects/new/chromium/src/third_party/zlib/zutil.h", "summary":{ "lines":{ "count":0, "covered":0, "percent":0 }, "functions":{ "count":0, "covered":0, "percent":0 }, "instantiations":{ "count":0, "covered":0, "percent":0 }, "regions":{ "count":0, "covered":0, "notcovered":0, "percent":0 } } } ], "totals":{ "lines":{ "count":2449, "covered":1346, "percent":54 }, "functions":{ "count":61, "covered":22, "percent":36 }, "instantiations":{ "count":61, "covered":22, "percent":36 }, "regions":{ "count":1852, "covered":1016, "notcovered":836, "percent":54 } } } ] }
,
Dec 11 2017
This is awesome! Thanks Max!
,
Dec 19 2017
This is done, but clang roll in Chromium is still in progress: issue 787920 .
,
Dec 22 2017
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by liaoyuke@chromium.org
, Dec 6 2017