New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 841513 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: May 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocking:
issue 789981



Sign in to add a comment

pdfium_unittests crashing with use_clang_coverage=true, will affect code coverage launch

Project Member Reported by infe...@chromium.org, May 9 2018

Issue description

Steps to reproduce

gn gen //out/coverage --args='is_clang=true use_libfuzzer=true use_clang_coverage=true is_component_build=false pdf_enable_xfa=true proprietary_codecs=true ffmpeg_branding="ChromeOS" strip_absolute_paths_from_debug_symbols=true is_debug=false'
gclient runhooks
python ./tools/clang/scripts/update.py
python tools/code_coverage/coverage.py -b out/coverage -o out/report -c 'out/coverage/pdf_unittests' pdf_unittests

This crashes and only test out of the other 80. Any crash means we don't show code coverage from pdfium_unittests which is not nice. we do get coverage from pdf_unittests though.

You can see stack with running this
out/coverage/pdf_unittests

Retrying 1 test (retry #1)
[ RUN      ] FindTextTest.FindText
Received signal 11 SEGV_MAPERR 000000000030
#0 0x0000053cc3d0 base::debug::StackTrace::StackTrace()
#1 0x0000053cbe00 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#2 0x7ff7f21160c0 <unknown>
#3 0x0000037a76c6 pp::Resource::~Resource()
#4 0x0000019d2e44 chrome_pdf::FindTextTest_FindText_Test::TestBody()
  r8: 0000000000160a43  r9: 00000000000008ca r10: 00000000000008ca r11: 0000000000000202
 r12: 00007ffd9a1e09f8 r13: 00007ffd9a1e09c0 r14: 0000000000005816 r15: 00007ffd9a1e0af8
  di: 000000000a719fb4  si: 0000000000000000  bp: 00007ffd9a1e0970  bx: 00007ffd9a1e0fd0
  dx: 00007ff7e605f000  ax: 0000000000000000  cx: 000000000006338c  sp: 00007ffd9a1e0960
  ip: 00000000037a76c6 efl: 0000000000010202 cgf: 002b000000000033 erf: 0000000000000004
 trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000030
[end of stack trace]
Calling _exit(1). Core file will not be generated.
[56/56] FindTextTest.FindText (CRASHED)
Retrying 1 test (retry #2)
[ RUN      ] FindTextTest.FindText
Received signal 11 SEGV_MAPERR 000000000030
#0 0x0000053cc3d0 base::debug::StackTrace::StackTrace()
#1 0x0000053cbe00 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#2 0x7fe5197fc0c0 <unknown>
#3 0x0000037a76c6 pp::Resource::~Resource()
#4 0x0000019d2e44 chrome_pdf::FindTextTest_FindText_Test::TestBody()
  r8: 0000000000160a43  r9: 00000000000008ca r10: 00000000000008ca r11: 0000000000000202
 r12: 00007ffce5281b28 r13: 00007ffce5281af0 r14: 0000000000005816 r15: 00007ffce5281c28
  di: 000000000a719fb4  si: 0000000000000000  bp: 00007ffce5281aa0  bx: 00007ffce5282100
  dx: 00007fe50d745000  ax: 0000000000000000  cx: 000000000006338c  sp: 00007ffce5281a90
  ip: 00000000037a76c6 efl: 0000000000010202 cgf: 002b000000000033 erf: 0000000000000004
 trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000030
[end of stack trace]
Calling _exit(1). Core file will not be generated.
[57/57] FindTextTest.FindText (CRASHED)
Retrying 1 test (retry #3)
[ RUN      ] FindTextTest.FindText
Received signal 11 SEGV_MAPERR 000000000030
#0 0x0000053cc3d0 base::debug::StackTrace::StackTrace()
#1 0x0000053cbe00 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#2 0x7f1e1fabd0c0 <unknown>
#3 0x0000037a76c6 pp::Resource::~Resource()
#4 0x0000019d2e44 chrome_pdf::FindTextTest_FindText_Test::TestBody()
  r8: 0000000000160a43  r9: 00000000000008ca r10: 00000000000008ca r11: 0000000000000202
 r12: 00007ffdacc7a158 r13: 00007ffdacc7a120 r14: 0000000000005816 r15: 00007ffdacc7a258
  di: 000000000a719fb4  si: 0000000000000000  bp: 00007ffdacc7a0d0  bx: 00007ffdacc7a730
  dx: 00007f1e13a06000  ax: 0000000000000000  cx: 000000000006338c  sp: 00007ffdacc7a0c0
  ip: 00000000037a76c6 efl: 0000000000010202 cgf: 002b000000000033 erf: 0000000000000004
 trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000030
[end of stack trace]
Calling _exit(1). Core file will not be generated.
[58/58] FindTextTest.FindText (CRASHED)
1 test crashed:
    FindTextTest.FindText (../../pdf/pdfium/findtext_unittest.cc:160)
Tests took 1 seconds.
 
I dont think there is chromium.org/dummy.pdf file

TEST_F(FindTextTest, FindText) {
  pp::URLLoader dummy_loader;
  TestClient client;
  PDFiumEngine engine(&client);
  ASSERT_TRUE(engine.New("https://chromium.org/dummy.pdf", ""));
  ASSERT_TRUE(engine.HandleDocumentLoad(dummy_loader));
Owner: thestig@chromium.org
I added the test.
Blocking: 789981
Sorry i gave the wrong test.

Crash is 
https://chromium-coverage.appspot.com/reports/556999/linux/metadata/pdfium_unittests.log

[----------] 1 test from CPDF_HintTablesTest
[ RUN      ] CPDF_HintTablesTest.Load
UndefinedBehaviorSanitizer:DEADLYSIGNAL
==21623==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address 0x000000000000 (pc 0x0000034ea029 bp 0x7fff2a3149b0 sp 0x7fff2a314990 T21623)
==21623==The signal is caused by a READ memory access.
==21623==Hint: address points to the zero page.
    #0 0x34ea028  (/chromium/src/out/coverage/pdfium_unittests+0x34ea028)
    #1 0x3474746  (/chromium/src/out/coverage/pdfium_unittests+0x3474746)
    #2 0xcde49a  (/chromium/src/out/coverage/pdfium_unittests+0xcde49a)
    #3 0x3004343  (/chromium/src/out/coverage/pdfium_unittests+0x3004343)
    #4 0x30062c5  (/chromium/src/out/coverage/pdfium_unittests+0x30062c5)
    #5 0x3007ab0  (/chromium/src/out/coverage/pdfium_unittests+0x3007ab0)
    #6 0x30256a7  (/chromium/src/out/coverage/pdfium_unittests+0x30256a7)
    #7 0x3024d4f  (/chromium/src/out/coverage/pdfium_unittests+0x3024d4f)
    #8 0x1003af3  (/chromium/src/out/coverage/pdfium_unittests+0x1003af3)
    #9 0x7f827af2782f  (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #10 0xc4c029  (/chromium/src/out/coverage/pdfium_unittests+0xc4c029)

I am trying to see why we didnt symbolize this.
Here is the full stacktrace

[----------] 1 test from CPDF_HintTablesTest
[ RUN      ] CPDF_HintTablesTest.Load
UndefinedBehaviorSanitizer:DEADLYSIGNAL
==11072==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000003926239 bp 0x7ffd4cb89840 sp 0x7ffd4cb89820 T11072)
==11072==The signal is caused by a READ memory access.
==11072==Hint: address points to the zero page.
    #0 0x3926238 in get ../../buildtools/third_party/libc++/trunk/include/memory:2607:19
    #1 0x3926238 in operator-> ../../third_party/pdfium/core/fxcrt/retain_ptr.h:80
    #2 0x3926238 in CPDF_ReadValidator::CPDF_ReadValidator(fxcrt::RetainPtr<IFX_SeekableReadStream> const&, CPDF_DataAvail::FileAvail*) ../../third_party/pdfium/core/fpdfapi/parser/cpdf_read_validator.cpp:51
    #3 0x38b0976 in MakeRetain<CPDF_ReadValidator, const fxcrt::RetainPtr<IFX_SeekableReadStream> &, CPDF_DataAvail::FileAvail *const &> ../../third_party/pdfium/core/fxcrt/retain_ptr.h:125:27
    #4 0x38b0976 in CPDF_DataAvail::CPDF_DataAvail(CPDF_DataAvail::FileAvail*, fxcrt::RetainPtr<IFX_SeekableReadStream> const&, bool) ../../third_party/pdfium/core/fpdfapi/parser/cpdf_data_avail.cpp:85
    #5 0xfbce7a in MakeUnique<CPDF_DataAvail, nullptr_t, fxcrt::RetainPtr<IFX_SeekableReadStream>, bool> ../../third_party/pdfium/third_party/base/ptr_util.h:56:33
    #6 0xfbce7a in MakeDataAvailFromFile ../../third_party/pdfium/core/fpdfapi/parser/cpdf_hint_tables_unittest.cpp:26
    #7 0xfbce7a in CPDF_HintTablesTest_Load_Test::TestBody() ../../third_party/pdfium/core/fpdfapi/parser/cpdf_hint_tables_unittest.cpp:44
    #8 0x33f8123 in testing::Test::Run() ../../third_party/googletest/src/googletest/src/gtest.cc
    #9 0x33fa0a5 in testing::TestInfo::Run() ../../third_party/googletest/src/googletest/src/gtest.cc:2667:11
    #10 0x33fb890 in testing::TestCase::Run() ../../third_party/googletest/src/googletest/src/gtest.cc:2785:28
    #11 0x3419417 in testing::internal::UnitTestImpl::RunAllTests() ../../third_party/googletest/src/googletest/src/gtest.cc:5047:43
    #12 0x3418abf in testing::UnitTest::Run() ../../third_party/googletest/src/googletest/src/gtest.cc
    #13 0x12e25b1 in RUN_ALL_TESTS ../../third_party/googletest/src/googletest/include/gtest/gtest.h:2329:46
    #14 0x12e25b1 in main ../../third_party/pdfium/testing/unit_test_main.cpp:86
    #15 0x7f09902bd2b0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202b0)
    #16 0xf2a029 in _start (/usr/local/google/home/aarya/build/chromium/src/out/coverage/pdfium_unittests+0xf2a029)

UndefinedBehaviorSanitizer can not provide additional info.
==11072==ABORTING
And command is
out/coverage/pdfium_unittests not pdf_unittests, sorry for confusion.
I'll take a look anyway. Do we know why these tests only crash with coverage turned on? They are not crashing when built + ran in a PDFium checkout.
That is the part we are unclear about, so your help is appreciated here.
Hm, interesting. I don't see anything coverage-specific in the stacktrace. My wild guess would be: can it go out of memory, and then end up with a null-deref?

However, that doesn't sound likely to me, as our coverage bots have ~120 GB of RAM each.
I can reproduce the CPDF_HintTablesTest.Load crash in a coverage build. I'll try to answer my own question then.
pdfium_unittests does not work in general in a Chromium checkout, because the directory structure is different. The only bots that run pdfium_unittests are on the PDFium waterfall, which uses a standalone PDFium checkout.
How much work is needed to make it work in chromium checkout OR does it make sense to remove it. Since it is DEPSed in, it will be nice to make the test run so that coverage of those parts of code show up. We won't be be running standalone components from their repos for quite a while, so need a solution from chromium checkout.
Status: Started (was: Assigned)
It's not much work. https://pdfium-review.googlesource.com/32314
Project Member

Comment 14 by bugdroid1@chromium.org, May 10 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chrome/tools/code-coverage/+/c9eb3ff238238aee0f5aed2e2b45d9242dbde5af

commit c9eb3ff238238aee0f5aed2e2b45d9242dbde5af
Author: Abhishek Arya <inferno@chromium.org>
Date: Thu May 10 03:35:57 2018

Project Member

Comment 15 by bugdroid1@chromium.org, May 10 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1c97ea548f8d4cb5cbb285353839ca5128b5af49

commit 1c97ea548f8d4cb5cbb285353839ca5128b5af49
Author: Abhishek Arya <inferno@chromium.org>
Date: Thu May 10 03:53:19 2018

Coverage: Add llvm-symbolizer to path for symbolized stacks

R=mmoroz@chromium.org,liaoyuke@chromium.org

Bug:  784464 ,  841513 
Change-Id: Ia592f2c0145d9117e6c59d14c6b1685d104429c0
Reviewed-on: https://chromium-review.googlesource.com/1053161
Reviewed-by: Max Moroz <mmoroz@chromium.org>
Commit-Queue: Abhishek Arya <inferno@chromium.org>
Cr-Commit-Position: refs/heads/master@{#557446}
[modify] https://crrev.com/1c97ea548f8d4cb5cbb285353839ca5128b5af49/tools/code_coverage/coverage.py

Project Member

Comment 16 by bugdroid1@chromium.org, May 10 2018

The following revision refers to this bug:
  https://pdfium.googlesource.com/pdfium/+/f4f19b51b2d588abe80df8493c23d708ec63f1b7

commit f4f19b51b2d588abe80df8493c23d708ec63f1b7
Author: Lei Zhang <thestig@chromium.org>
Date: Thu May 10 17:33:56 2018

Make GetTestDataDir() work in a non-standalone checkout.

Set the test data dir path correctly if PDFium is living inside another
project as third_party/pdfium.

BUG= chromium:841513 

Change-Id: I565f7d97157e1769be8b7910f3c77d6d00015543
Reviewed-on: https://pdfium-review.googlesource.com/32314
Reviewed-by: dsinclair <dsinclair@chromium.org>
Commit-Queue: Lei Zhang <thestig@chromium.org>

[modify] https://crrev.com/f4f19b51b2d588abe80df8493c23d708ec63f1b7/testing/utils/path_service.h
[modify] https://crrev.com/f4f19b51b2d588abe80df8493c23d708ec63f1b7/testing/utils/path_service.cpp

Status: Fixed (was: Started)
Just waiting for the DEPS roll now.
Project Member

Comment 18 by bugdroid1@chromium.org, May 10 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/511f71f2d113829989df1949d1a9343f1d0019f8

commit 511f71f2d113829989df1949d1a9343f1d0019f8
Author: pdfium-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <pdfium-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Thu May 10 20:13:17 2018

Roll src/third_party/pdfium/ 95061379c..80302c77a (5 commits)

https://pdfium.googlesource.com/pdfium.git/+log/95061379c945..80302c77a854

$ git log 95061379c..80302c77a --date=short --no-merges --format='%ad %ae %s'
2018-05-10 rharrison Use test_dir instead of 'pdfium' for source type
2018-05-10 thestig Add CPDF_Transparency.
2018-05-10 thestig Make GetTestDataDir() work in a non-standalone checkout.
2018-05-10 tsepez Fix destruction order in CPDF_Dibsource.
2018-05-10 npm Remove a completeness check from CJBig2_GRRDProc::DecodeTemplate0Opt

Created with:
  roll-dep src/third_party/pdfium
BUG= chromium:841513 , chromium:840695 , chromium:841200 


The AutoRoll server is located here: https://pdfium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=dsinclair@chromium.org

Change-Id: I579c4a7663af521bb842f5e0f309f2bcd71732f3
Reviewed-on: https://chromium-review.googlesource.com/1054263
Reviewed-by: pdfium-chromium-autoroll <pdfium-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: pdfium-chromium-autoroll <pdfium-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#557647}
[modify] https://crrev.com/511f71f2d113829989df1949d1a9343f1d0019f8/DEPS

Sign in to add a comment