One more:

Received signal 11 SEGV_MAPERR 000000000010
#0 0x000000e63328 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#1 0x7fe5e9315cb0 <unknown>
#2 0x00000213c290 remoting::protocol::KeyEvent::Clear()
#3 0x00000213f058 remoting::protocol::KeyEvent::CopyFrom()
#4 0x000000ffd8f5 remoting::NormalizingInputFilterCros::ProcessKeyDown()
#5 0x000000568264 remoting::NormalizingInputFilterCrosTest_FunctionKey_Test::TestBody()
#6 0x000000f770df testing::Test::Run()
#7 0x000000f79a3e testing::TestInfo::Run()
#8 0x000000f7b7e3 testing::TestCase::Run()
#9 0x000000f94ea8 testing::internal::UnitTestImpl::RunAllTests()
#10 0x000000f94127 testing::internal::HandleExceptionsInMethodIfSupported<>()
#11 0x000000f93be5 testing::UnitTest::Run()
#12 0x000000ec237d base::TestSuite::Run()
#13 0x000000eab239 base::(anonymous namespace)::LaunchUnitTestsInternal()
#14 0x000000eab04e base::LaunchUnitTests()
#15 0x00000050256b main
#16 0x7fe5e8d5376d __libc_start_main
#17 0x0000004ae9b9 <unknown>
  r8: 0000000008000000  r9: 0000000004ea1ac0 r10: 4e09e3306b7c3288 r11: 0000337bce626bb8
 r12: 801b4f4d92a23198 r13: 801b4f4d92a23198 r14: 0000000004ea1ac0 r15: 9ddfea08eb382d69
  di: 0000337bce626bb8  si: 0000000000001671  bp: 00007fff90a63590  bx: 0000337bce626bb8
  dx: 0000000004ea1ac0  ax: 00007fe565348000  cx: 000000000213c44f  sp: 00007fff90a63560
  ip: 000000000213c290 efl: 0000000000010202 cgf: 0000000000000033 erf: 0000000000000004
 trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000010
[end of stack trace]

I've reproduced one of the failures in cc_unittests:

gn gen //out/gn '--args=is_ubsan_vptr=true is_debug=false' --check
ninja -C out/gn cc_unittests
gdb --args ./out/gn/cc_unittests --gtest_filter=LayerProtoConverterTest.RecursivePropertiesSerialization

[ RUN      ] LayerProtoConverterTest.RecursivePropertiesSerialization
[New Thread 0x7fffede37700 (LWP 46639)]

Program received signal SIGSEGV, Segmentation fault.
0x000000000243694d in Clear () at gen/cc/proto/layer.pb.cc:1608
1608        ZR_(background_color_, safe_opaque_background_color_);
(gdb) bt
#0  0x000000000243694d in Clear () at gen/cc/proto/layer.pb.cc:1608
#1  0x0000000002436574 in Clear () at gen/cc/proto/layer.pb.cc:1050
#2  0x0000000002434594 in Clear () at ../../third_party/protobuf/src/google/protobuf/repeated_field.h:564
#3  Clear<google::protobuf::RepeatedPtrField<cc::proto::LayerProperties>::TypeHandler> () at ../../third_party/protobuf/src/google/protobuf/repeated_field.h:1447
#4  Clear () at ../../third_party/protobuf/src/google/protobuf/repeated_field.h:1922
#5  Clear () at gen/cc/proto/layer.pb.cc:793
#6  0x00000000008b7c2f in TestBody () at ../../cc/layers/layer_proto_converter_unittest.cc:230
#7  0x0000000002750147 in testing::Test::Run() () at ../../testing/gtest/src/gtest.cc:2474
#8  0x00000000027517da in Run () at ../../testing/gtest/src/gtest.cc:2656
#9  0x0000000002752722 in Run () at ../../testing/gtest/src/gtest.cc:2774
#10 0x000000000275f7e8 in RunAllTests () at ../../testing/gtest/src/gtest.cc:4647
#11 0x000000000275ebe9 in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> () at ../../testing/gtest/src/gtest.cc:2458
#12 0x000000000275e9dd in Run () at ../../testing/gtest/src/gtest.cc:4255
#13 0x0000000001cf2e83 in RUN_ALL_TESTS () at ../../testing/gtest/include/gtest/gtest.h:2237
#14 Run () at ../../base/test/test_suite.cc:230
#15 0x0000000001d038f4 in Run () at ../../base/callback.h:397
#16 LaunchUnitTestsInternal () at ../../base/test/launcher/unit_test_launcher.cc:206
#17 0x0000000001d037a9 in LaunchUnitTests () at ../../base/test/launcher/unit_test_launcher.cc:445
#18 0x000000000193f813 in main () at ../../cc/test/run_all_unittests.cc:12

And this is the second stack trace kind:

Starting program: /usr/local/google/home/krasin/chr25/src/out/gn/cc_unittests --gtest_filter=RendererPixelTest/0.EnlargedRenderPassTexture
...
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffede37700 (LWP 64311)]
0x00007ffff2a2b153 in __dynamic_cast () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0  0x00007ffff2a2b153 in __dynamic_cast () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x000000000058f73d in __ubsan::checkDynamicType(void*, void*, unsigned long) ()
#2  0x000000000058e822 in HandleDynamicTypeCacheMiss(__ubsan::DynamicTypeCacheMissData*, unsigned long, unsigned long, __ubsan::ReportOptions) ()
#3  0x000000000058e7f2 in __ubsan_handle_dynamic_type_cache_miss ()
#4  0x00007fffee124d8d in _mesa_ast_to_hir () at ../../third_party/mesa/src/src/glsl/ast_to_hir.cpp:90
#5  0x00007fffee05a182 in _mesa_glsl_compile_shader () at ../../third_party/mesa/src/src/mesa/program/ir_to_mesa.cpp:3081
#6  0x00007fffee001abe in compile_shader () at ../../third_party/mesa/src/src/mesa/main/shaderapi.c:733
#7  _mesa_CompileShaderARB () at ../../third_party/mesa/src/src/mesa/main/shaderapi.c:1031
#8  0x000000000285f64b in DoCompile () at ../../gpu/command_buffer/service/shader_manager.cc:85
#9  0x00000000028438a0 in CompileAttachedShaders () at ../../gpu/command_buffer/service/program_manager.cc:1490
#10 0x0000000002842be6 in Link () at ../../gpu/command_buffer/service/program_manager.cc:1052
#11 0x00000000027cef96 in DoLinkProgram () at ../../gpu/command_buffer/service/gles2_cmd_decoder.cc:7044
#12 0x00000000027610a8 in gpu::gles2::GLES2DecoderImpl::HandleLinkProgram(unsigned int, void const*) () at ../../gpu/command_buffer/service/gles2_cmd_decoder_autogen.h:2345
#13 0x00000000027b3d0a in DoCommandsImpl<false> () at ../../gpu/command_buffer/service/gles2_cmd_decoder.cc:4471
#14 0x00000000027133bf in gpu::CommandParser::ProcessCommands(int) () at ../../gpu/command_buffer/service/cmd_parser.cc:53
#15 0x00000000027161ae in PutChanged () at ../../gpu/command_buffer/service/command_executor.cc:61
#16 0x0000000002823bfd in Run<> () at ../../base/bind_internal.h:181
#17 MakeItSo<base::WeakPtr<gpu::InProcessCommandBuffer>> () at ../../base/bind_internal.h:334
#18 Run () at ../../base/bind_internal.h:372
#19 0x000000000281d352 in FlushOnGpuThread () at ../../gpu/command_buffer/service/in_process_command_buffer.cc:525
#20 0x0000000002823fa8 in Run () at ../../base/bind_internal.h:372
#21 0x0000000002a34423 in Run () at ../../base/callback.h:397
#22 RunTask () at ../../base/debug/task_annotator.cc:51
#23 0x00000000029b6de2 in RunTask () at ../../base/message_loop/message_loop.cc:479
#24 0x00000000029b7759 in DeferOrRunPendingTask () at ../../base/message_loop/message_loop.cc:488
#25 0x00000000029b7dc4 in DoWork () at ../../base/message_loop/message_loop.cc:600
#26 0x00000000029bb38c in Run () at ../../base/message_loop/message_pump_default.cc:33
#27 0x00000000029b6361 in RunHandler () at ../../base/message_loop/message_loop.cc:443
#28 0x00000000029e0d39 in base::RunLoop::Run() () at ../../base/run_loop.cc:35
#29 0x00000000029b5749 in Run () at ../../base/message_loop/message_loop.cc:295
#30 0x00000000029f960f in ThreadMain () at ../../base/threading/thread.cc:254
#31 0x00000000029f25a7 in ThreadFunc () at ../../base/threading/platform_thread_posix.cc:70
#32 0x00007ffff27b8182 in start_thread (arg=0x7fffede37700) at pthread_create.c:312
#33 0x00007ffff22cf47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

At this moment, I completely don't understand what's wrong. But it more looks like a ubsan / clang issue than a code issue.

Interesting. It turns out that with GN I see more failures, and they are likely not real. Filed  https://crbug.com/606003 

So, after figuring out the difference between GYP and GN for ubsan_vptr (the fix is under review), and ran a GYP-based bisect. The Nico's guess about libprotobuf3 is correct:

7cf8148f570a1072928fa9ede6e0760edc137cbd is the first bad commit
commit 7cf8148f570a1072928fa9ede6e0760edc137cbd
Author: xyzzyz <xyzzyz@chromium.org>
Date:   Wed Apr 20 16:50:23 2016 -0700

    Update //third_party/protobuf to version 3.
    
    For context, see https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/zJZPkxEFVxY
    
    For the things to look out for while reviewing this, see the doc https://docs.google.com/document/d/1qjSBV2ioi0mykAT1eSXuMaNqO0BIS-j1PeYfxT-TbW4/edit?usp=sharing
    
    BUG= 597321 
    
    Review URL: https://codereview.chromium.org/1842653006
    
    Cr-Commit-Position: refs/heads/master@{#388596}


I will try to take another look at this on Monday.

There're two things fundamentally broken about protobuf + ubsan_vptr, but both are about the same:

1) GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET macro that tries to emulate __builtin_offsetof and makes are reinterpret_casts on an invalid pointer (16):
https://code.google.com/p/chromium/codesearch#chromium/src/third_party/protobuf/src/google/protobuf/generated_message_reflection.h&q=GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET&sq=package:chromium&type=cs&l=590

This macro is the reason for the 3 lines in ubsan/vptr_blacklist.txt:
https://code.google.com/p/chromium/codesearch#chromium/src/tools/ubsan/vptr_blacklist.txt&q=vptr_blacklist&sq=package:chromium&type=cs&l=76

2) ZR_HELPER that appears in the generated files:

#define ZR_HELPER_(f) reinterpret_cast<char*>(\
  &reinterpret_cast<BaseLayerProperties*>(16)->f)

Some issue: invalid reinterpret_cast.

The first issue might be fixed by defining GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET as __builtin_offsetof and suppressing -Winvalid-offsetof.

The second issue is more complicated and I need to think more about this.

The immediate reason for the tests to be broken is ZR_HELPER. As a short-term solution to make the bot green would be to blacklist the following files:

out/Release/gen/protoc_out/cc/proto/point3f.pb.cc
out/Release/gen/protoc_out/cc/proto/size.pb.cc
out/Release/gen/protoc_out/cc/proto/layer_position_constraint.pb.cc
out/Release/gen/protoc_out/cc/proto/vector2d.pb.cc
out/Release/gen/protoc_out/cc/proto/vector2df.pb.cc
out/Release/gen/protoc_out/cc/proto/point.pb.cc
out/Release/gen/protoc_out/cc/proto/sizef.pb.cc
out/Release/gen/protoc_out/cc/proto/scroll_offset.pb.cc
out/Release/gen/protoc_out/cc/proto/renderer_settings.pb.cc
out/Release/gen/protoc_out/cc/proto/recording_source.pb.cc
out/Release/gen/protoc_out/cc/proto/managed_memory_policy.pb.cc
out/Release/gen/protoc_out/net/quic/proto/cached_network_parameters.pb.cc
out/Release/gen/protoc_out/cc/proto/pointf.pb.cc
out/Release/gen/protoc_out/cc/proto/layer_tree_debug_state.pb.cc
out/Release/gen/protoc_out/cc/proto/layer_selection_bound.pb.cc
out/Release/gen/protoc_out/cc/proto/begin_main_frame_and_commit_state.pb.cc
out/Release/gen/protoc_out/cc/proto/layer_tree_host.pb.cc
out/Release/gen/protoc_out/cc/proto/layer_tree_settings.pb.cc
out/Release/gen/protoc_out/cc/proto/display_item.pb.cc
out/Release/gen/protoc_out/gpu/command_buffer/service/disk_cache_proto.pb.cc
out/Release/gen/protoc_out/cc/proto/layer.pb.cc
out/Release/gen/protoc_out/cc/proto/property_tree.pb.cc

In the long term, this is insane, we need to find a better way.

The for the generated code should be in third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message.cc:

The following macro needs to be changed:
+  const char* macros =
+      "#define ZR_HELPER_(f) __builtin_offsetof($classname$, f)\n\n"
+      "#define ZR_(first, last) do {\\\n"
+      "  ::memset(&first, 0,\\\n"
+      "           ZR_HELPER_(last) - ZR_HELPER_(first) + sizeof(last));\\\n"
+      "} while (0)\n\n";

But again, that does not address the problem of a warning. A pragma will need to be used. Also, it might be a good idea to only use __builtin_offsetof for Clang.

xyzzyz, can you work with krasin to bring this problem to the attention of upstream protobuf?

I created an issue: https://github.com/google/protobuf/issues/1450

Isn't replacing it with offsetof() on non-POD type still undefined behaviour, though?

Thank you, Adam, for filing an upstream bug. I've commented there and hope to participate in the discussion.

Is there someone particular from the protobuf maintainers we should add to the bug?

I usually got responses from Feng Xiao pretty quickly, and looks like he already responded there.

Adam,

Feng approved the changes to the internal protobuf codebase, and github will get my changes in 2-3 weeks.

Since we have 2 buildbots broken at this time, I would like to submit a change into third_party/protobuf ahead of the official fixes. Once github gets my fixes, I will revert the local patch and will import the new protobuf version.

Does it sound good to you?

Yes, this sounds fine -- we already carry some local patches ahead of upstream release.

In your CL, please also add a patch to third_party/protobuf/patches/ and update the README.chromium with your patch description.

Sure, will do.

Btw, how do I update  src/google/protobuf/descriptor.pb.cc and other generated files?

I tried:

cd third_party/protobuf
./autogen.sh
./configure
./generate_descriptor_proto.sh

but while that has succeeded, it generated diff that includes more changes that I've made, which suggests that either I hold it wrong, or the previous change didn't have these scripts running.

Please, guide me.

Let me make sure I understand you correctly -- protobuf release ships some generated C++ files, and you want to update them so that they incorporate your changes, right? If so, I think you need to ask Feng or some other protobuf maintainers, as I've never had to do it.

Ah, if Chrome does not do that, it's fine. I've already regenerated these files in the internal Google repo, and my temporary fix may live w/o them.

The bot will be green even with these changes:
https://codereview.chromium.org/1929193002

The bots should be green because your change only touches protobuf_full, which we don't use in Chromium (it's only used by the protoc compiler itself). I don't think it's worth regenerating, as there's no semantic difference, and it will get updated in the next release anyway.

yes, exactly my thoughts! :)
Filed  https://crbug.com/607751  to update //third_party/protobuf later.

Thank you for your collaboration on the issue. It really helped that you brought the issue to the upstream bug tracker.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7809c2d9948352fb73d50d3b88b45e42bcfed5cc

commit 7809c2d9948352fb73d50d3b88b45e42bcfed5cc
Author: krasin <krasin@google.com>
Date: Fri Apr 29 01:50:59 2016

Use offsetof in ZR_HELPER_ for Clang.

As an immediate reason for the change is that making an invalid
reinterpret_cast(16) causes Control Flow Integrity and UBSan vptr
sanitizers to crash on reading the vtable pointer. Ignoring this
problem (by suppressing a check) would mask real failures, some
of which have security consequences.

Theoretically, a similar change could be made for GCC, but there's
no immediate pressure to do that, as GCC does not have any type-checking
subsystem for reinterpret_casts.

NOTE: this patch will need to be reverted, when the github repo gets the official fix submitted by me to the internal Google repo.

BUG= 605933 

Review-Url: https://codereview.chromium.org/1929193002
Cr-Commit-Position: refs/heads/master@{#390568}

[modify] https://crrev.com/7809c2d9948352fb73d50d3b88b45e42bcfed5cc/third_party/protobuf/README.chromium
[add] https://crrev.com/7809c2d9948352fb73d50d3b88b45e42bcfed5cc/third_party/protobuf/patches/0011-use-offsetof-for-clang.patch
[modify] https://crrev.com/7809c2d9948352fb73d50d3b88b45e42bcfed5cc/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message.cc
[modify] https://crrev.com/7809c2d9948352fb73d50d3b88b45e42bcfed5cc/third_party/protobuf/src/google/protobuf/generated_message_reflection.h

'ClangToTLinuxUBSanVptr tester' bot is green:

https://build.chromium.org/p/chromium.fyi/builders/ClangToTLinuxUBSanVptr%20tester/builds/439

Thanks!