Running code coverage in debug=false mode? |
|||||||
Issue descriptionI tested the running time and binary size of the following different configurations using unit_tests target. 1. coverage_debug_true: is_debug = true use_clang_coverage = true is_component_build = false 2. coverage_debug_false: is_debug = false use_clang_coverage = true is_component_build = false 3. coverage_symbol_level_zero: use_clang_coverage = true is_component_build = false symbol_level = 0 sanitizer_no_symbols = true 4. coverage_debug_false_and_symbol_level_zero: is_debug = false use_clang_coverage = true is_component_build = false symbol_level = 0 sanitizer_no_symbols = true Here is the results on running time: ninja: Entering directory `out/coverage_debug_true/' [38560/38560] LINK ./unit_tests real 88m54.022s user 92m49.868s sys 5m29.280s ninja: Entering directory `out/coverage_debug_false' [38954/38954] LINK ./unit_tests real 147m32.624s user 187m30.288s sys 3m1.904s ninja: Entering directory `out/coverage_symbol_level_zero/' [39836/39836] LINK ./unit_tests real 84m38.285s user 91m8.280s sys 4m12.764s ninja: Entering directory `out/coverage_debug_false_and_symbol_level_zero/' [38592/38592] LINK ./unit_tests real 169m46.107s user 211m21.180s sys 2m59.788s Here is the results on binary size: liaoyuke@liaoyuke2 ~/chromium/src (master) $ du -sh out/coverage_debug_true/unit_tests 5.2G out/coverage_debug_true/unit_tests liaoyuke@liaoyuke2 ~/chromium/src (master) $ du -sh out/coverage_debug_false/unit_tests 2.1G out/coverage_debug_false/unit_tests liaoyuke@liaoyuke2 ~/chromium/src (master) $ du -sh out/coverage_symbol_level_zero/unit_tests 2.6G out/coverage_symbol_level_zero/unit_tests liaoyuke@liaoyuke2 ~/chromium/src (master) $ du -sh out/coverage_debug_false_and_symbol_level_zero/unit_tests 2.1G out/coverage_debug_false_and_symbol_level_zero/unit_tests
,
Feb 20 2018
+dpranke@, I've experimented above configurations using different orders, I get pretty much the same results. There are a few things I don't quite understand. 1. What's the difference between "is_debug = false" and "symbol_level = 0\n sanitizer_no_symbols = true", how much overlap do they have? 2. I thought setting "is_debug = false" will result in smaller binary size (which is true), and it will be faster to link and assemble the final binary, but somehow, it's not the case, do we know why? inferno@, dpranke@, can you share your thoughts before I digging into this?
,
Feb 20 2018
is_debug mostly controls the level of optimization the compiler generates (i.e., the -O flag and a few others), which is separate from the amount of debugging information. There is very little guaranteed about the size of the binaries except that -O3 is normally bigger than -O2, though symbol_level=0 should be the same or smaller than symbol_level=1 and will definitely be smaller than symbol_level=2. I encourage you to be talking to thakis@ and hans@ and the other members of the lexan@ team as well here.
,
Feb 21 2018
hans@, thakis@, can you take a look at #2?
,
Feb 21 2018
I don't have a lot of context here, but replying to the questions in #2: 1) is_debug controls the optimization level. Calling it is_optimized (with the opposite meaning) would probably be a better name, but it's named the way it is for historic reasons. It doesn't affect the debug info (except that optimized debug info is less accurate). symbol_level controls the debug-info generation. 0 means none, 1 means a little, and 2 means full debug info. sanitizer_keep_symbols seems to turn on line-tables-only debug info for sanitizer builds. Maybe that's also relevant for your coverage builds. 2) "is_debug = false" turns on optimization, causing longer compile times and generates a smaller and faster binary. The reason you see different link times is that "is_debug = true" implicitly sets "is_component_build = true", which means Chromium is linked into a number of separate shared libraries. For release builds, it's linked into a single static binary instead, which is slower. It's possible to set "is_component_build = true" together with "is_debug = true" to get faster links.
,
Feb 21 2018
Thanks hans@! I think the numbers make sense now, though it's unfortunately that now is_component_build is not compatible with coverage build. Anyway, I think it's safe to conclude that we should prefer building with "is_debug=true" and "symbol_level=0", I'll bake this preference into the code coverage tool.
,
Feb 21 2018
Yuke, do you mean is_debug=false (not true), and symbol_level=0? If yes, SGTM. As an alternative, we may revisit what's wrong with running component build, maybe we can make it work?
,
Feb 21 2018
Sorry for the confusion, I mean is_debug=true, as hans@ mentioned, it really means is_optimized (with the opposite meaning). I don't think we need optimization, do we? What's more, is_debug=true saves about half of the time to build a coverage target comparing to is_debug=false.
,
Feb 21 2018
I think we need optimizations because: 1) tests run faster 2) binary size is smaller 3) optimized code is more similar to what we ship to users, compared to the non-optimized build 4) optimization doesn't affect code coverage instrumentation, as it's being added before compiler does optimizations We need to check whether is_component_build=true works well with coverage, and if it does, we may try using: is_debug = false is_component_build = true use_clang_coverage = true But I recall there were some issues with generating coverage from a component build :(
,
Feb 22 2018
Right, it currently doesn't work with "ios_component_build=true". I'm fine with enabling optimizations given that build time is not our biggest concern right now, we can revisit our options if it becomes a bottleneck in the future.
,
Apr 12 2018
,
Apr 17 2018
|
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by liaoyuke@chromium.org
, Feb 14 2018