New issue
Advanced search Search tips

Issue 892651 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Oct 16
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug



Sign in to add a comment

SuperSize: ThinLTO causes .bss symbols v8 embedded builtins to have no object_path

Project Member Reported by agrieve@chromium.org, Oct 5

Issue description

With ThinLTO enabled (now the default again):

>>> Print(size_info.symbols.Filter(lambda s: not s.source_path and s.IsBss()).Sorted())

Showing 10,638 symbols (10,638 unique) with total pss: 0 bytes
Histogram of symbols based on PSS:
     [1,2): 629      [16,32): 238     [256,512): 30     [4096,8192): 12    [65536,131072): 1
     [2,4): 7        [32,64): 279    [512,1024): 16    [8192,16384): 5    [131072,262144): 2
     [4,8): 8925    [64,128): 143   [1024,2048): 24   [16384,32768): 3    [262144,524288): 1
    [8,16): 255    [128,256): 53    [2048,4096): 10   [32768,65536): 5
Sizes: .bss=1.07mb     total=0 bytes
Counts: .bss=12941
Number of unique paths: 28

Section Legend: b=.bss
Index | Running Total | Section@Address | PSS | Path
------------------------------------------------------------
0)          0 (0.0%)  b@0x2f4f3c0  262164         {no path}
             ff_cos_131072
1)          0 (0.0%)  b@0x2f93700  131312         {no path}
             table_4_3_value
2)          0 (0.0%)  b@0x2f068c0  131072         {no path}
             ff_cos_65536
3)          0 (0.0%)  b@0x2ef4440  65536          {no path}
             ff_cos_32768
4)          0 (0.0%)  b@0x2fec164  60000          {no path}
             vr::g_vertices
5)          0 (0.0%)  b@Group      50614          {no path}
             .L_MergedGlobals (count=2044)
6)          0 (0.0%)  b@0x2fb37f0  32828          {no path}
             table_4_3_exp
7)          0 (0.0%)  b@0x2fbb82c  32768          {no path}
             expval_table_fixed
8)          0 (0.0%)  b@0x2eeb200  32768          {no path}
             ff_cos_16384
9)          0 (0.0%)  b@0x2fdba40  22288          {no path}
             about_flags::kFeatureEntries
10)         0 (0.0%)  b@0x2f342d0  18724          {no path}
             v8::internal::compiler::kMachineOperatorGlobalCache
...



>>> Print(size_info.symbols.Filter(lambda s: not s.source_path and not s.object_path and not s.IsBss()).Sorted())

Showing 1,004 symbols (1,004 unique) with total pss: 1907626 bytes
Histogram of symbols based on PSS:
      {0}: 1     [8,16): 154    [64,128): 473    [512,1024): 11      [8192,16384): 2   [1048576,2097152): 1
    [2,4): 3    [16,32): 168   [128,256): 47    [1024,2048): 5    [131072,262144): 1
    [4,8): 61   [32,64): 47    [256,512): 25    [2048,4096): 4    [262144,524288): 1
Sizes: .text=1.30mb     .rodata=36.9kb     .data.rel.ro=25.4kb     .data=49.2kb     .pak.translations=1879 bytes .pak.nontranslated=59 bytes   .other=421kb      total=1.82mb
Counts: .text=5 .rodata=1569 .data.rel.ro=1605 .data=494 .pak.translations=96 .pak.nontranslated=4 .other=2
Number of unique paths: 1

Section Legend: t=.text, r=.rodata, R=.data.rel.ro, d=.data, p=.pak.translations, P=.pak.nontranslated, o=.other
Index | Running Total | Section@Address | PSS | Path
------------------------------------------------------------
0)    1173768 (61.5%) t@0xde56c0   1173768        {no path}
1)    1595813 (83.7%) o@0x0        422045         {no path}
             Overhead: APK file
2)    1781885 (93.4%) t@Group      186072         {no path}
             ** thunk (count=2)
3)    1794984 (94.1%) r@Group      13099          {no path}
             .L.ref.tmp (count=561)
4)    1804510 (94.6%) o@0x0        9526           {no path}
             Overhead: ELF file
5)    1808362 (94.8%) R@Group      3852           {no path}
             .L.ref.tmp (count=449)
6)    1811569 (95.0%) r@Group      3207           {no path}
             .L.ref.tmp.1 (count=149)
7)    1814129 (95.1%) r@Group      2560           {no path}
             .L.ref.tmp.2 (count=118)
8)    1816178 (95.2%) r@0x53ebfc   2049           {no path}
             string literal
9)    1818088 (95.3%) r@Group      1910           {no path}
             .L.ref.tmp.3 (count=85)
10)   1819691 (95.4%) r@Group      1603           {no path}
             .L.ref.tmp.4 (count=69)
11)   1821249 (95.5%) r@Group      1558           {no path}
             .L.ref.tmp.5 (count=68)
12)   1822775 (95.6%) d@Group      1526           {no path}
             .L_MergedGlobals (count=58)
13)   1824135 (95.6%) R@Group      1360           {no path}
             .L.ref.tmp.48 (count=7)
14)   1825158 (95.7%) r@Group      1023           {no path}
             .L.ref.tmp.6 (count=46)
15)   1825886 (95.7%) p@0x0        728            {no path}
             artifacts
16)   1826611 (95.8%) r@Group      725            {no path}
             .L.ref.tmp.7 (count=32)
17)   1827263 (95.8%) R@Group      652            {no path}
             .L.ref.tmp.16 (count=22)
18)   1827895 (95.8%) R@Group      632            {no path}
             .L.ref.tmp.21 (count=13)
19)   1828504 (95.9%) r@Group      609            {no path}
             .L.ref.tmp.8 (count=27)
20)   1829072 (95.9%) R@Group      568            {no path}
             .L.ref.tmp.4 (count=69)
21)   1829640 (95.9%) R@Group      568            {no path}
             .L.ref.tmp.83 (count=2)
22)   1830184 (95.9%) R@Group      544            {no path}
             .L.ref.tmp.55 (count=5)
23)   1830704 (96.0%) R@Group      520            {no path}
             .L.ref.tmp.76 (count=3)
24)   1831218 (96.0%) r@Group      514            {no path}
             .L.ref.tmp.10 (count=24)
25)   1831722 (96.0%) R@Group      504            {no path}
             .L.ref.tmp.46 (count=6)
26)   1832226 (96.0%) R@Group      504            {no path}
             .L.ref.tmp.63 (count=2)
27)   1832709 (96.1%) r@Group      483            {no path}
             .L.ref.tmp.9 (count=22)
28)   1833189 (96.1%) R@Group      480            {no path}
             .L.ref.tmp.6 (count=46)
29)   1833637 (96.1%) R@Group      448            {no path}
             .L.ref.tmp.10 (count=47)
30)   1834077 (96.1%) R@Group      440            {no path}
             .L.ref.tmp.44 (count=6)
31)   1834509 (96.2%) R@Group      432            {no path}
             .L.ref.tmp.8 (count=37)
32)   1834933 (96.2%) R@Group      424            {no path}
             .L.ref.tmp.53 (count=4)
33)   1835347 (96.2%) r@Group      414            {no path}
             .L.ref.tmp.13 (count=18)
34)   1835712 (96.2%) r@Group      365            {no path}
             .L.ref.tmp.12 (count=17)
35)   1836076 (96.2%) r@Group      364            {no path}
             .L.ref.tmp.11 (count=17)
36)   1836436 (96.3%) R@Group      360            {no path}
             .L.ref.tmp.5 (count=38)
37)   1836792 (96.3%) r@Group      356            {no path}
             .L.ref.tmp.14 (count=16)
38)   1837136 (96.3%) R@Group      344            {no path}
             .L.ref.tmp.2 (count=39)
...



For just .text, it's a mere 3 symbols:

>>> Print(size_info.symbols.Filter(lambda s: not s.source_path and not s.object_path and not s.IsBss() and s.section == 't').Sorted())
Showing 4 symbols (4 unique) with total pss: 1359906 bytes
Histogram of symbols based on PSS:
    {0}: 1   [64,128): 1   [131072,262144): 1   [1048576,2097152): 1
Sizes: .text=1.30mb     total=1.30mb
Counts: .text=5
Number of unique paths: 1

Section Legend: t=.text
Index | Running Total | Section@Address | PSS | Path
------------------------------------------------------------
0)    1173768 (86.3%) t@0xde56c0   1173768        {no path}
1)    1359840 (100.0%) t@Group      186072         {no path}
             ** thunk (count=2)
2)    1359906 (100.0%) t@0x86cbb0   66 (size=200)  {no path}
             .udivsi3_skip_div0_test (num_aliases=3)
3)    1359906 (100.0%) t@0xacdd48   0              {no path}


The linker map for the giant symbol shows:
  de56c0   de56c0   11e900    32         thinlto-cache/llvmcache-5573B54B4EDC1E7F3AF7D65CC252AD2E88FA98CA:(.text)
  de56c0   de56c0        0     1                 v8_Default_embedded_blob_
  de8360   de8360        0     1                 Builtins_RecordWrite
  de8680   de8680        0     1                 Builtins_AdaptorWithExitFrame
...

I've filed  bug 892648  to track the fact that we're not giving a proper name to this symbol, but the bigger bug here is that the path is lost. Pre-lto, it knew the symbol was from "gen/v8/embedded.cc"


I'd guess the problem for both is that llvm-nm is not finding the symbols in the .o files (or the names aren't matching up, or for v8's case, the name is empty so we're ignoring it).


 
When I run supersize archive, I also see "no symbols" being printed many times -- this didn't used to happen.
Many of these appear to be purposefully excluded in _AssignNmAliasPathsAndCreatePathAliases():

https://cs.chromium.org/chromium/src/tools/binary_size/libsupersize/archive.py?rcl=37aab329373b49a9753c0481e22c192f21861edd&l=379

Simply removing the symbol.IsBss() check changes the number of BSS symbols without source_path from ~10600 to < 100. Continuing to dig on why this was added, and what would be the proper fix. The line was added to archive.py in:

https://codereview.chromium.org/2851473003




Project Member

Comment 3 by bugdroid1@chromium.org, Oct 13

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f90dc349b7b37a6d268730b1e8a8da56e25f70bd

commit f90dc349b7b37a6d268730b1e8a8da56e25f70bd
Author: Samuel Huang <huangs@chromium.org>
Date: Sat Oct 13 00:46:33 2018

[Supersize] Fix missing object_path for .bss symbols under LLD-LTO (ThinLTO).

Two data sources to get object_path for .bss symbols are:
(1) Linker map file (linker_map_parser)
(2) nm output (archive._AssignNmAliasPathsAndCreatePathAliases()).

Previously, (1) was used but (2) was skipped (by explicitly ignoring
.bss symbols) because it was redundant (true for Gold and LLD).

For LLD-LTO (ThinLTO), however, (1) no longer provides useful info for
object_path. This causes .bss symbols to lose object_path info.

This CL fixes the problem by enabling .bss symbols to get |object_path|
data from (2). Unfortunately, this also causes additional aliasing of
symbols that seem to be off, e.g., in .bss and .rodata symbols that
have the same name become aliased in LLD. This is a less serious problem
compared to the main issue at hand, so we'll fix it as a follow-up.

Also fix 3 missing end-of-line commas in mock_nm.py discovered during
review. The fix prevents this CL from causing spurious test result
changes!

Bug:  892651 
Change-Id: Ia69b2d4beb39e3bfd9af5b7566743e69ed6ae775
Reviewed-on: https://chromium-review.googlesource.com/c/1278959
Reviewed-by: Samuel Huang <huangs@chromium.org>
Reviewed-by: agrieve <agrieve@chromium.org>
Commit-Queue: Samuel Huang <huangs@chromium.org>
Cr-Commit-Position: refs/heads/master@{#599448}
[modify] https://crrev.com/f90dc349b7b37a6d268730b1e8a8da56e25f70bd/tools/binary_size/libsupersize/archive.py
[modify] https://crrev.com/f90dc349b7b37a6d268730b1e8a8da56e25f70bd/tools/binary_size/libsupersize/obj_analyzer.py
[modify] https://crrev.com/f90dc349b7b37a6d268730b1e8a8da56e25f70bd/tools/binary_size/libsupersize/testdata/mock_toolchain/mock_nm.py

Status: Fixed (was: Assigned)
Set to fixed, but a follow-up issue is 895664.

Sign in to add a comment