New issue
Advanced search Search tips

Issue 880618 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Dec 4
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: ----

Blocked on:
issue 744567



Sign in to add a comment

cronet_tests failing on chromium.memory/Linux CFI

Project Member Reported by sheriff-...@appspot.gserviceaccount.com, Sep 4

Issue description

Filed by sheriff-o-matic@appspot.gserviceaccount.com on behalf of huangs@chromium.org

cronet_tests failing on chromium.memory/Linux CFI

Builders failed on: 
- Linux CFI: 
  https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20CFI


 
Owner: mef@chromium.org
Reverted: https://chromium-review.googlesource.com/c/chromium/src/+/1205770 . Reassigning to mef@.
Labels: -Sheriff-Chromium
Components: Internals>Network>Library
Labels: OS-Linux
Status: Assigned (was: Available)
This is an accurate summary:

../../components/cronet/native/engine.cc:403:23: note: check failed in /b/s/w/ir/out/Release/./libcronet.so, vtable located in /b/s/w/ir/out/Release/cronet_tests

Will need to think about different / better solution that doesn't involve passing object pointer in void* disguise. 

Here is more of the relevant output:

../../components/cronet/native/engine.cc:403:23: runtime error: control flow integrity check for type 'net::CertVerifier' failed during cast to unrelated type (vtable address 0x0000002695b0)
0x0000002695b0: note: invalid vtable
 00 00 00 00  40 32 76 00 00 00 00 00  80 32 76 00 00 00 00 00  a0 32 76 00 00 00 00 00  70 35 76 00
              ^
../../components/cronet/native/engine.cc:403:23: note: check failed in /b/s/w/ir/out/Release/./libcronet.so, vtable located in /b/s/w/ir/out/Release/cronet_tests
    #0 0x7f15f559db77 in Cronet_Engine_SetMockCertVerifierForTesting ./../../components/cronet/native/engine.cc:403:23
    #1 0x5a6040 in cronet::test::CreateTestEngine(int) ./../../components/cronet/native/test/test_util.cc:59:3
    #2 0x4e58e0 in grpc_support::StartTestStreamEngine(int) ./../../components/cronet/native/test/test_stream_engine.cc:37:21
    #3 0x5d16c1 in grpc_support::BidirectionalStreamTest::SetUp() ./../../components/grpc_support/bidirectional_stream_unittest.cc:40:5
    #4 0x59748c in testing::Test::Run() ./../../third_party/googletest/src/googletest/src/gtest.cc:2518:3
    #5 0x597db0 in testing::TestInfo::Run() ./../../third_party/googletest/src/googletest/src/gtest.cc:2698:11
    #6 0x598561 in testing::TestCase::Run() ./../../third_party/googletest/src/googletest/src/gtest.cc:2816:28
    #7 0x59fa32 in testing::internal::UnitTestImpl::RunAllTests() ./../../third_party/googletest/src/googletest/src/gtest.cc:5182:43
    #8 0x59f67c in testing::UnitTest::Run() ./../../third_party/googletest/src/googletest/src/gtest.cc:4791:10
    #9 0x6b4827 in base::TestSuite::Run() ./../../base/test/test_suite.cc:295:16
    #10 0x4e4f50 in int base::internal::Invoker<base::internal::BindState<int (base::TestSuite::*)(), base::internal::UnretainedWrapper<base::TestSuite> >, int ()>::RunImpl<int (base::TestSuite::*)(), std::__1::tuple<base::internal::UnretainedWrapper<base::TestSuite> >, 0ul>(int (base::TestSuite::*&&)(), std::__1::tuple<base::internal::UnretainedWrapper<base::TestSuite> >&&, std::__1::integer_sequence<unsigned long, 0ul>) ./../../base/bind_internal.h:689:12
    #11 0x6b8f84 in base::OnceCallback<int ()>::Run() && ./../../base/callback.h:99:12
    #12 0x6b7e3f in base::(anonymous namespace)::LaunchUnitTestsInternal(base::OnceCallback<int ()>, unsigned long, int, bool, base::OnceCallback<void ()>) ./../../base/test/launcher/unit_test_launcher.cc:224:38
    #13 0x6b7d07 in base::LaunchUnitTests(int, char**, base::OnceCallback<int ()>) ./../../base/test/launcher/unit_test_launcher.cc:575:10
    #14 0x4e4e0e in main ./../../components/cronet/run_all_unittests.cc:17:10
    #15 0x7f15f2311f44 in __libc_start_main /build/eglibc-ripdx6/eglibc-2.19/csu/libc-start.c:287:0
    #16 0x483029 in _start ??:0:0

Blockedon: 744567
Cc: pauljensen@chromium.org
The problem appears to be with SetMockCertVerifierForTesting taking the MockCertVerifier instantiated in cronet_tests and trying to use it in libcronet.so.

This is not quite legal, so the complaint about "invalid vtable" sounds legit.

I'm planning to remedy the issue by landing https://chromium-review.googlesource.com/c/chromium/src/+/1191464 which will result in cronet linked statically into cronet_tests and MockCertVerifier vtable being shared between Cronet and its test.

We will need to figure another way to allow test certs for testing of apps that use Cronet.
Can you explain what is illegal here?  The vtable pointers will be different, but I'm not sure why this would be a problem.  vtable pointers are different for child classes.  Sounds like we do some extra checks on linux.
My interpretation of 'during cast to unrelated type (vtable address 0x0000002695b0)' is that MockCertVerifier in libcronet.so is expected to implement net::CertVerifier interface defined in libcronet.so.

Passing in MockCertVerifier from cronet_tests that implements net::CertVerifier interface defined in cronet_tests fails the check because those two definitions could be different.

In practice net::CertVerifier interfaces are the same in both binaries, so this implementation works fine on most normal tests.

My wild guess is that Linux.CFI is specifically setup to detect this condition, possibly by adding some id to vtable of base class and checking that passed child class includes that id?
Here's what the CFI in the bot's name indicates:
https://www.chromium.org/developers/testing/control-flow-integrity

Since the testing of Cronet's libcronet.so intentionally involves swapping equivalent vtables, we could disable these tests for CFI using GN or C++ preprocessor flags.
Status: Fixed (was: Assigned)
This issue was fixed by changing cronet_tests to statically link with cronet and avoid duplicate class definitions.

Sign in to add a comment