New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 812357 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocked on:
issue 831939



Sign in to add a comment

Running code coverage in debug=false mode?

Project Member Reported by liaoyuke@chromium.org, Feb 14 2018

Issue description

I tested the running time and binary size of the following different configurations using unit_tests target.

1. coverage_debug_true:
is_debug = true
use_clang_coverage = true
is_component_build = false

2. coverage_debug_false:
is_debug = false
use_clang_coverage = true
is_component_build = false

3. coverage_symbol_level_zero:
use_clang_coverage = true
is_component_build = false
symbol_level = 0
sanitizer_no_symbols = true

4. coverage_debug_false_and_symbol_level_zero:
is_debug = false
use_clang_coverage = true
is_component_build = false
symbol_level = 0
sanitizer_no_symbols = true

Here is the results on running time:

ninja: Entering directory `out/coverage_debug_true/'
[38560/38560] LINK ./unit_tests

real	88m54.022s
user	92m49.868s
sys	5m29.280s
ninja: Entering directory `out/coverage_debug_false'
[38954/38954] LINK ./unit_tests

real	147m32.624s
user	187m30.288s
sys	3m1.904s
ninja: Entering directory `out/coverage_symbol_level_zero/'
[39836/39836] LINK ./unit_tests

real	84m38.285s
user	91m8.280s
sys	4m12.764s
ninja: Entering directory `out/coverage_debug_false_and_symbol_level_zero/'
[38592/38592] LINK ./unit_tests

real	169m46.107s
user	211m21.180s
sys	2m59.788s

Here is the results on binary size:

liaoyuke@liaoyuke2 ~/chromium/src (master) $ du -sh out/coverage_debug_true/unit_tests 
5.2G	out/coverage_debug_true/unit_tests
liaoyuke@liaoyuke2 ~/chromium/src (master) $ du -sh out/coverage_debug_false/unit_tests 
2.1G	out/coverage_debug_false/unit_tests
liaoyuke@liaoyuke2 ~/chromium/src (master) $ du -sh out/coverage_symbol_level_zero/unit_tests 
2.6G	out/coverage_symbol_level_zero/unit_tests
liaoyuke@liaoyuke2 ~/chromium/src (master) $ du -sh out/coverage_debug_false_and_symbol_level_zero/unit_tests 
2.1G	out/coverage_debug_false_and_symbol_level_zero/unit_tests
 
So, looks like setting is_debug=false generates the smallest binary size, but the build time is also significantly greater than coverage_symbol_level_zero.

Given that coverage_debug_false only saves 1/4 on binary size comparing to coverage_symbol_level_zero, but the build time is almost doubled, can we conclude that coverage_symbol_level_zero should be the default option to go?

inferno@, mmoroz@, WDYT?
Cc: dpranke@chromium.org
Status: Started (was: Assigned)
+dpranke@, I've experimented above configurations using different orders, I get pretty much the same results. There are a few things I don't quite understand.

1. What's the difference between "is_debug = false" and "symbol_level = 0\n sanitizer_no_symbols = true", how much overlap do they have?

2. I thought setting "is_debug = false" will result in smaller binary size (which is true), and it will be faster to link and assemble the final binary, but somehow, it's not the case, do we know why?

inferno@, dpranke@, can you share your thoughts before I digging into this?
Cc: h...@chromium.org thakis@chromium.org
is_debug mostly controls the level of optimization the compiler generates (i.e., the -O flag and a few others), which is separate from the amount of debugging information.

There is very little guaranteed about the size of the binaries except that -O3 is normally bigger than -O2, though symbol_level=0 should be the same or smaller than symbol_level=1 and will definitely be smaller than symbol_level=2.

I encourage you to be talking to thakis@ and hans@ and the other members of the lexan@ team as well here.
hans@, thakis@, can you take a look at #2?

Comment 5 by h...@chromium.org, Feb 21 2018

I don't have a lot of context here, but replying to the questions in #2:

1)

is_debug controls the optimization level. Calling it is_optimized (with the opposite meaning) would probably be a better name, but it's named the way it is for historic reasons. It doesn't affect the debug info (except that optimized debug info is less accurate).

symbol_level controls the debug-info generation. 0 means none, 1 means a little, and 2 means full debug info.

sanitizer_keep_symbols seems to turn on line-tables-only debug info for sanitizer builds. Maybe that's also relevant for your coverage builds.

2)
"is_debug = false" turns on optimization, causing longer compile times and generates a smaller and faster binary.

The reason you see different link times is that "is_debug = true" implicitly sets "is_component_build = true", which means Chromium is linked into a number of separate shared libraries. For release builds, it's linked into a single static binary instead, which is slower. It's possible to set "is_component_build = true" together with "is_debug = true" to get faster links.
Summary: Running code coverage in debug=false mode? (was: Running code coverage in debug=false mode)
Thanks hans@! I think the numbers make sense now, though it's unfortunately that now is_component_build is not compatible with coverage build.

Anyway, I think it's safe to conclude that we should prefer building with "is_debug=true" and "symbol_level=0", I'll bake this preference into the code coverage tool.

Comment 7 by mmoroz@chromium.org, Feb 21 2018

Yuke, do you mean is_debug=false (not true), and symbol_level=0? If yes, SGTM.

As an alternative, we may revisit what's wrong with running component build, maybe we can make it work?
Sorry for the confusion, I mean is_debug=true, as hans@ mentioned, it really means is_optimized (with the opposite meaning). I don't think we need optimization, do we?

What's more, is_debug=true saves about half of the time to build a coverage target comparing to is_debug=false.

Comment 9 by mmoroz@chromium.org, Feb 21 2018

I think we need optimizations because:

1) tests run faster

2) binary size is smaller

3) optimized code is more similar to what we ship to users, compared to the non-optimized build

4) optimization doesn't affect code coverage instrumentation, as it's being added before compiler does optimizations

We need to check whether is_component_build=true works well with coverage, and if it does, we may try using:
is_debug = false
is_component_build = true
use_clang_coverage = true

But I recall there were some issues with generating coverage from a component build :(

Labels: -Pri-1 Pri-2
Right, it currently doesn't work with "ios_component_build=true". I'm fine with enabling optimizations given that build time is not our biggest concern right now, we can revisit our options if it becomes a bottleneck in the future.
Blockedon: 831939
Status: Fixed (was: Started)

Sign in to add a comment