cross built 64-bit chrome.exe does not use v8 snapshot |
|||||||||||||
Issue descriptionReproduce steps: 1. Cross build chrome on Linux.(https://chromium.googlesource.com/chromium/src/+/lkcr/docs/win_cross.md) 2. Copy out/Release folder to window box. (To avoid too many files, I removed gen objs and clang_x86 in out/Release folder) 3. Try run chrome.ese on Windows The chrome windows shows up, but seems the renderer crashes.
,
Jan 18 2018
,
Jan 18 2018
I cross built a bench of unittests(aura, views, base, content, components, net, sql, cc, gpu etc). I found most failed unittests are related to V8. For example: ProxyResolverV8Test.* in net_unittests
,
Jan 19 2018
I tried build chrome with v8_use_snapshot = false and workaround several gn issues. The cross built chrome can browser web now. So the issue is related to v8 snapshot.
,
Jan 20 2018
Is this reproduced with v8_use_snapahot=true use_v8_context_snapshot=false ? I ask this question to distinguish this issue is on V8's snapshot or on Blink's snapshot.
,
Jan 20 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a3f0ece5deee50fceadd4ee1dcaeea0103d08385 commit a3f0ece5deee50fceadd4ee1dcaeea0103d08385 Author: Peng Huang <penghuang@chromium.org> Date: Sat Jan 20 01:22:46 2018 Fix v8_use_snapshot = false issue with windows cross build Bug: 803591 Change-Id: Ic531a5a561fdca2dd7eca4acb94bb068530ee716 Reviewed-on: https://chromium-review.googlesource.com/876726 Commit-Queue: Peng Huang <penghuang@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Cr-Commit-Position: refs/heads/master@{#530714} [modify] https://crrev.com/a3f0ece5deee50fceadd4ee1dcaeea0103d08385/BUILD.gn [modify] https://crrev.com/a3f0ece5deee50fceadd4ee1dcaeea0103d08385/chrome/installer/mini_installer/BUILD.gn
,
Jan 22 2018
v8_use_snapahot=true use_v8_context_snapshot=false This configure doesn't work, because the below error: Waiting for editor on "/usr/local/google/home/penghuang/sources/chromium/src/out_win/Release/args.gn"... Generating files... ERROR Input to target not generated by a dependency. The file: //out_win/Release/v8_context_snapshot.bin is listed as an input or source for the target: //chrome/installer/mini_installer:mini_installer_archive but no targets in the build generate that file.
,
Jan 30 2018
,
Jan 31 2018
Now we can build Chromium for Windows on 32bit MacOS w/o "v8_use_snapshot=false". My CL(*) made it not to create the snapshot for Blink. It uses a snapshot for V8 as we did in past / we are doing on Android/ChromeOS. (*) https://chromium-review.googlesource.com/c/chromium/src/+/882624 The key issue is that we cannot build Blink for 32bit MacOSX. Some code is written in assembly only for x64. e.g. https://cs.chromium.org/chromium/src/base/mac/call_with_eh_frame_asm.S I think enabling V8 context snapshot on such builds is a different issue, so let me close this.
,
Jan 31 2018
You meant to close Issue 794838 , not this, right?
,
Jan 31 2018
Ooops, you are right. I'm sorry to be confusing.
,
Jan 31 2018
I currently don't have the means to reproduce this. But I have a suspicion why this happens. When we create the snapshot, we embed V8's version as a string into a 64 byte area of the snapshot. When we deserialize from the snapshot, we create the version string again and compare that to the 64 bytes in the snapshot. The version string is created via snprintf. I suspect that Windows implements it differently than Linux. Maybe it is terminated differently?
,
Jan 31 2018
You are right. The string parts are same, but the rest part of the 64 bytes are different. BTW I tried skipping the check, but v8 still crashes in other place. Probably the similar issue exists in other places as well.
,
Jan 31 2018
Can you maybe verify whether this patch helps? https://chromium-review.googlesource.com/#/c/v8/v8/+/894523
,
Jan 31 2018
Hm.... Now that I think about this, I don't think this can work. V8's snapshot includes code objects. These code objects are platform-dependent, as Windows and Linux have different calling conventions. Code with Linux calling convention called on Windows will cause crashes. I am not aware of cross-compile support in V8 to build on Linux on Windows. And this is the first time I heard that Chrome offers one. The document describing cross compilation says "It's possible to build most parts of the codebase on a Linux or Mac host while targeting Windows.". The examples it shows are build targets that are not related to V8, such as base_unittests. I don't think cross compilation between Windows and Linux ever worked with V8.
,
Jan 31 2018
How difficult to fix the calling convention? Does compiler have flags to control it? Or could we upload prebuild V8 snapshot binary to somewhere, and download it during cross build on non-Windows platforms? At least, we can modify gn file, to make sure v8_use_snapshot is not enabled for cross build.
,
Jan 31 2018
I'd say pretty difficult. The way a snapshot is created is that we initialize an instance of V8 when we run mksnapshot, and serialize this instance of V8. At runtime, we deserialize that instance from the snapshot. For cross-compiled platforms like ARM, V8 has a simulator that executes ARM instructions. The mksnapshot on the cross-compile host initializes and runs V8 on that ARM simulator to create the snapshot. In the target platform we can then use that snapshot because it has been created using a simulator. This does not apply for Linux/Windows cross-compile. In fact, V8 does not offer a x64 simulator at all. You could use pre-built V8 snapshots, but those are tied to specific V8 versions and like you've seen, platform. It's probably simpler if you cross-compile mksnapshot, and run it on the target platform to create the snapshot on the target platform.
,
Jan 31 2018
I want to fix this somehow, so let's reopen and give it to me, and I'll think of something.
,
Jan 31 2018
(For example, build the mksnapshot entry point with itanium calling conventions and then the rest with windows calling conventions, and make the compiler translate at system function call sites -- that way the snapshot might work.)
,
Jan 31 2018
Not sure I understand what you mean. I'm talking about code that V8 generates itself. There are some differences in calling convention that V8 hard codes into these generated code objects. So unless you run mksnapshot on the target platform, I don't see an easy solution.
,
Jan 31 2018
Oh! But isn't it then enough to teach v8 mksnapshot that host platform and target platform aren't necessarily the same thing, and then tell it to generate windows abi type code when targeting windows even though it's running on a linux host? win abi type code can run fine on linux as long as you use itanium calling conventions and struct layout for calling system functions.
,
Feb 1 2018
If mksnapshot built for Windows calling convention can run fine on Linux (both for code generated by C++ compiler and by V8), then I guess this could work. Like I mentioned previously, we already do this for non-Intel platforms. We run mksnapshot on the host (either ia32 or x64 depending on the target platform's pointer size) with the target platform's simulator. So the concept that host and target is not the same platform already exists in V8's build configs. Probably not for Win/Linux though.
,
Feb 2 2018
The following revision refers to this bug: https://chromium.googlesource.com/v8/v8.git/+/b8059eb51188e36c342eaab71e1bdd5d3cedf8bd commit b8059eb51188e36c342eaab71e1bdd5d3cedf8bd Author: Yang Guo <yangguo@chromium.org> Date: Fri Feb 02 08:41:40 2018 [snapshot] use strncmp to compare version string. R=jgruber@chromium.org Bug: chromium:803591/ Change-Id: I3f2d1126df4362c2f434551a28953a1bdebfeff5 Reviewed-on: https://chromium-review.googlesource.com/894523 Reviewed-by: Jakob Gruber <jgruber@chromium.org> Commit-Queue: Yang Guo <yangguo@chromium.org> Cr-Commit-Position: refs/heads/master@{#51050} [modify] https://crrev.com/b8059eb51188e36c342eaab71e1bdd5d3cedf8bd/src/snapshot/snapshot-common.cc
,
Feb 22 2018
ExternalReference in assembler.h
,
Feb 25 2018
penghuang: Can you confirm that you did a 64-bit build? Things seem better in 32-bit as far as I can tell (at least one v8-using net_unittest runs fine on swarming in a 32-bit build).
,
Feb 26 2018
The following revision refers to this bug: https://chromium.googlesource.com/v8/v8.git/+/bd7204998acbfae83d2af529c2e7bf3e16a4fca6 commit bd7204998acbfae83d2af529c2e7bf3e16a4fca6 Author: Nico Weber <thakis@chromium.org> Date: Mon Feb 26 10:13:49 2018 Disable snapshots in 64-bit win/cross builds for now. Snapshots don't yet work in 64-bit win/cross builds, so disable them until they do. No behavior change in builds that aren't 64-bit win/cross builds. Bug: chromium:803591 Change-Id: I7a04c7e01a58a95a2bfb78c7d2593c7c5c5041cf Reviewed-on: https://chromium-review.googlesource.com/936668 Reviewed-by: Yang Guo <yangguo@chromium.org> Commit-Queue: Nico Weber <thakis@chromium.org> Cr-Commit-Position: refs/heads/master@{#51554} [modify] https://crrev.com/bd7204998acbfae83d2af529c2e7bf3e16a4fca6/gni/v8.gni
,
Feb 26 2018
I am not sure, but looks like the addresses in the crash log are in 64 bits. FYI below is args I used. target_os = "win" v8_use_snapshot = false # use_v8_context_snapshot = false is_debug = false # use_goma = true dcheck_always_on = true
,
Feb 26 2018
The following revision refers to this bug: https://pdfium.googlesource.com/pdfium/+/611431decd9a5b7dd456355fe763b2dee9c2a1a1 commit 611431decd9a5b7dd456355fe763b2dee9c2a1a1 Author: Nico Weber <thakis@chromium.org> Date: Mon Feb 26 20:36:16 2018 Fix -Wunused-variable warning if building with V8 snapshots disabled. Bug: chromium:803591 Change-Id: I4dbbe52ca296e7232ad3eb225a2bdd84dcd0fa81 Reviewed-on: https://pdfium-review.googlesource.com/27730 Commit-Queue: Nico Weber <thakis@chromium.org> Commit-Queue: dsinclair <dsinclair@chromium.org> Reviewed-by: dsinclair <dsinclair@chromium.org> [modify] https://crrev.com/611431decd9a5b7dd456355fe763b2dee9c2a1a1/testing/unit_test_main.cpp
,
Mar 1 2018
Current status: Snapshots are disabled by default in 64-bit builds, so renderers should no longer crash but they start up slower instead. hans said he's interested in looking into a real fix (idea: pass BUILD_WIN on linux host, tag runtime functions with __attribute__((ms_abi))), so giving this to him for now.
,
Mar 20 2018
Dumping some thoughts.. It seems the fundamental problem is we want to build all V8 generated code with the MS ABI. The code calling into that generated code must call it using that ABI. Calls from the generated code will use the MS ABI, so functions in the runtime need to be compiled using that ABI. Should be able to use __attribute__((ms_abi)) for this. There's a V8_TARGET_ARCH_X64 macro, and then _WIN64 is used to decide whether to target MS or Itanium. We'd want to introduce a V8_TARGET_ARCH_WIN64 macro for cross-compilation. Runtime functions called from generated code can be found because they get the function pointers via ExternalReference. But to get the attribute onto those functions, we'd want to tag them, maybe with something like V8_RUNTIME_ENTRYPOINT. Calls into generated code go via function pointers created by FUNCTION_CAST. We'd need to get the abi attribute on those pointers somehow.
,
Mar 20 2018
How does this work with snapshots for ARM and the simulator? Calls from generated code to external references go through redirect functions (ExternalReference::Redirect) Maybe we could use that too? This scheme seems to put special bits in the generated code that's handled by the simulator(?). Could we instead use Redirect() to insert thunks for external functions for the "win on linux" scenario? Looking at FUNCTION_CAST for finding calls to generated code as I said in #33 doesn't work. I think instead the place to look at is GeneratedCode::Call().
,
Mar 27 2018
Work-in-progress patch: https://chromium-review.googlesource.com/c/v8/v8/+/980758 thakis: I'm confused about where to hook in and set the V8_TARGET_WIN64 macro from gn. It's a special situation since we're not really cross-compiling mksnapshot, but still want to convey that while it's built for the host it should have a different target.. yangguo: Do you think something like this could be acceptable to V8 (with some polish of course)?
,
Mar 29 2018
> thakis: I'm confused about where left a comment on your patch
,
Apr 3 2018
(retitling for what's left to do)
,
Apr 3 2018
Redirects on simulator builds are a bit of a hack. Every Redirector instance allocates a piece of memory that acts as a piece of machine code with a special instruction. So the simulator executed the call by actually calling into the Redirector code. The special instruction tells the simulator to stop simulating, and instead call the function address stored on the Redirector instance. The Redirector is not allocated in executable memory, because the simulator treads code as data. For this case of cross-compilation this won't work though. There is also no simulator, obviously. I think the easiest way to do this is, in mksnapshot, 1) Initialize the V8 isolate/heap but generate the CEntryStub for Linux. 2) Execute code to bootstrap V8 context. 3) Replace the CEntryStub with one targeting Windows. 4) During serialization, rewrite references to the old CEntryStub to the new one. We are currently moving code objects off V8's heap into the executable binary. This includes the CEntryStub, which may make step 4 a lot easier or even obsolete. Please coordinate with jgruber on this. This actually also affect a few other pieces of generated code such as the RegExpExecStub or JSEntryStub.
,
Apr 3 2018
Btw how do you intend to test this? V8 doesn't currently have a cross-compile bot for linux to windows. Also, why do we need this?
,
Apr 3 2018
why do we need this: we want cross builds to be identical to non-cross builds in the binary they produce.
,
Apr 3 2018
I meant, I'm not familiar with the background why we need cross-compilation in the first place.
,
Apr 3 2018
Because most devs are on linux and most users aren't :-) People seem to find it very useful now that it exists. |
|||||||||||||
►
Sign in to add a comment |
|||||||||||||
Comment 1 by penghuang@chromium.org
, Jan 18 2018The crash stack: # # Fatal error in ../../v8/src/snapshot/snapshot-common.cc, line 317 # Version mismatch between V8 binary and snapshot. # V8 binary version: 6.5.202 # Snapshot version: 6.5.202 # The snapshot consists of 3851036 bytes and contains 3 context(s). # Backtrace: base::debug::StackTrace::StackTrace [0x000001C56F2DBBBB+107] (\usr\local\google\home\penghuang\sources\chromium\src\base\debug\stack_trace_win.cc:286) base::debug::StackTrace::StackTrace [0x000001C56F2DABEF+31] (\usr\local\google\home\penghuang\sources\chromium\src\base\debug\stack_trace.cc:199) gin::`anonymous namespace'::PrintStackTrace [0x000001C50BE68554+36] (\usr\local\google\home\penghuang\sources\chromium\src\gin\v8_platform.cc:30) V8_Fatal [0x000001C519C0125B+123] (\usr\local\google\home\penghuang\sources\chromium\src\v8\src\base\logging.cc:135) v8::internal::Snapshot::CheckVersion [0x000001C50CC0ED95+293] (\usr\local\google\home\penghuang\sources\chromium\src\v8\src\snapshot\snapshot-common.cc:317) v8::internal::Snapshot::Initialize [0x000001C50CC0EA00+144] (\usr\local\google\home\penghuang\sources\chromium\src\v8\src\snapshot\snapshot-common.cc:44) v8::IsolateNewImpl [0x000001C50C0AC246+454] (\usr\local\google\home\penghuang\sources\chromium\src\v8\src\api.cc:8366) gin::IsolateHolder::IsolateHolder [0x000001C50BE3C9EE+734] (\usr\local\google\home\penghuang\sources\chromium\src\gin\isolate_holder.cc:71) blink::V8PerIsolateData::V8PerIsolateData [0x000001C510EABC81+273] (\usr\local\google\home\penghuang\sources\chromium\src\third_party\WebKit\Source\platform\bindings\V8PerIsolateData.cpp:79) blink::V8PerIsolateData::Initialize [0x000001C510EAC835+181] (\usr\local\google\home\penghuang\sources\chromium\src\third_party\WebKit\Source\platform\bindings\V8PerIsolateData.cpp:130) blink::V8Initializer::InitializeMainThread [0x000001C50D80FDE8+728] (\usr\local\google\home\penghuang\sources\chromium\src\third_party\WebKit\Source\bindings\core\v8\V8Initializer.cpp:645) blink::Initialize [0x000001C5196BC0CC+204] (\usr\local\google\home\penghuang\sources\chromium\src\third_party\WebKit\Source\controller\BlinkInitializer.cpp:108) content::RenderThreadImpl::InitializeWebKit [0x000001C505011284+516] (\usr\local\google\home\penghuang\sources\chromium\src\content\renderer\render_thread_impl.cc:1244) content::RenderThreadImpl::Init [0x000001C50500E2A5+2309] (\usr\local\google\home\penghuang\sources\chromium\src\content\renderer\render_thread_impl.cc:718) content::RenderThreadImpl::RenderThreadImpl [0x000001C50500CBAF+2975] (\usr\local\google\home\penghuang\sources\chromium\src\content\renderer\render_thread_impl.cc:663) content::RenderThreadImpl::Create [0x000001C50500BF3F+415] (\usr\local\google\home\penghuang\sources\chromium\src\content\renderer\render_thread_impl.cc:575) content::RendererMain [0x000001C5050CCE82+866] (\usr\local\google\home\penghuang\sources\chromium\src\content\renderer\renderer_main.cc:201) content::RunNamedProcessTypeMain [0x000001C505747B7F+223] (\usr\local\google\home\penghuang\sources\chromium\src\content\app\content_main_runner.cc:426) content::ContentMainRunnerImpl::Run [0x000001C5057486E1+593] (\usr\local\google\home\penghuang\sources\chromium\src\content\app\content_main_runner.cc:720) content::ContentServiceManagerMainDelegate::RunEmbedderProcess [0x000001C50574479A+42] (\usr\local\google\home\penghuang\sources\chromium\src\content\app\content_service_manager_main_delegate.cc:51) service_manager::Main [0x000001C514802CD3+1299] (\usr\local\google\home\penghuang\sources\chromium\src\services\service_manager\embedder\main.cc:456) content::ContentMain [0x000001C505747A15+85] (\usr\local\google\home\penghuang\sources\chromium\src\content\app\content_main.cc:19) ChromeMain [0x000001C5747692D6+422] (\usr\local\google\home\penghuang\sources\chromium\src\chrome\app\chrome_main.cc:129) MainDllLoader::Launch [0x0000000140088FC7+1127] (\usr\local\google\home\penghuang\sources\chromium\src\chrome\app\main_dll_loader_win.cc:199) wWinMain [0x000000014007E46A+1130] (\usr\local\google\home\penghuang\sources\chromium\src\chrome\app\chrome_exe_main_win.cc:230) invoke_main [0x00000001402AE0CD+45] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:123) __scrt_common_main_seh [0x00000001402AE207+295] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:283) __scrt_common_main [0x00000001402AE28E+14] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:326) wWinMainCRTStartup [0x00000001402AE2A9+9] (f:\dd\vctools\crt\vcstartup\src\startup\exe_wwinmain.cpp:17) BaseThreadInitThunk [0x00007FF94BC11FE4+20] RtlUserThreadStart [0x00007FF94C94EFB1+33] Received fatal exception EXCEPTION_ACCESS_VIOLATION Backtrace: base::win::`anonymous namespace'::ForceCrashOnSigAbort [0x000001C56F64EE65+5] (\usr\local\google\home\penghuang\sources\chromium\src\base\win\win_util.cc:89) raise [0x00007FF9219AFE21+1089] v8::base::OS::Abort [0x000001C519C0F1FF+15] (\usr\local\google\home\penghuang\sources\chromium\src\v8\src\base\platform\platform-win32.cc:872) V8_Fatal [0x000001C519C01274+148] (\usr\local\google\home\penghuang\sources\chromium\src\v8\src\base\logging.cc:136) v8::internal::Snapshot::CheckVersion [0x000001C50CC0ED95+293] (\usr\local\google\home\penghuang\sources\chromium\src\v8\src\snapshot\snapshot-common.cc:317) v8::internal::Snapshot::Initialize [0x000001C50CC0EA00+144] (\usr\local\google\home\penghuang\sources\chromium\src\v8\src\snapshot\snapshot-common.cc:44) v8::IsolateNewImpl [0x000001C50C0AC246+454] (\usr\local\google\home\penghuang\sources\chromium\src\v8\src\api.cc:8366) gin::IsolateHolder::IsolateHolder [0x000001C50BE3C9EE+734] (\usr\local\google\home\penghuang\sources\chromium\src\gin\isolate_holder.cc:71) blink::V8PerIsolateData::V8PerIsolateData [0x000001C510EABC81+273] (\usr\local\google\home\penghuang\sources\chromium\src\third_party\WebKit\Source\platform\bindings\V8PerIsolateData.cpp:79) blink::V8PerIsolateData::Initialize [0x000001C510EAC835+181] (\usr\local\google\home\penghuang\sources\chromium\src\third_party\WebKit\Source\platform\bindings\V8PerIsolateData.cpp:130) blink::V8Initializer::InitializeMainThread [0x000001C50D80FDE8+728] (\usr\local\google\home\penghuang\sources\chromium\src\third_party\WebKit\Source\bindings\core\v8\V8Initializer.cpp:645) blink::Initialize [0x000001C5196BC0CC+204] (\usr\local\google\home\penghuang\sources\chromium\src\third_party\WebKit\Source\controller\BlinkInitializer.cpp:108) content::RenderThreadImpl::InitializeWebKit [0x000001C505011284+516] (\usr\local\google\home\penghuang\sources\chromium\src\content\renderer\render_thread_impl.cc:1244) content::RenderThreadImpl::Init [0x000001C50500E2A5+2309] (\usr\local\google\home\penghuang\sources\chromium\src\content\renderer\render_thread_impl.cc:718) content::RenderThreadImpl::RenderThreadImpl [0x000001C50500CBAF+2975] (\usr\local\google\home\penghuang\sources\chromium\src\content\renderer\render_thread_impl.cc:663) content::RenderThreadImpl::Create [0x000001C50500BF3F+415] (\usr\local\google\home\penghuang\sources\chromium\src\content\renderer\render_thread_impl.cc:575) content::RendererMain [0x000001C5050CCE82+866] (\usr\local\google\home\penghuang\sources\chromium\src\content\renderer\renderer_main.cc:201) content::RunNamedProcessTypeMain [0x000001C505747B7F+223] (\usr\local\google\home\penghuang\sources\chromium\src\content\app\content_main_runner.cc:426) content::ContentMainRunnerImpl::Run [0x000001C5057486E1+593] (\usr\local\google\home\penghuang\sources\chromium\src\content\app\content_main_runner.cc:720) content::ContentServiceManagerMainDelegate::RunEmbedderProcess [0x000001C50574479A+42] (\usr\local\google\home\penghuang\sources\chromium\src\content\app\content_service_manager_main_delegate.cc:51) service_manager::Main [0x000001C514802CD3+1299] (\usr\local\google\home\penghuang\sources\chromium\src\services\service_manager\embedder\main.cc:456) content::ContentMain [0x000001C505747A15+85] (\usr\local\google\home\penghuang\sources\chromium\src\content\app\content_main.cc:19) ChromeMain [0x000001C5747692D6+422] (\usr\local\google\home\penghuang\sources\chromium\src\chrome\app\chrome_main.cc:129) MainDllLoader::Launch [0x0000000140088FC7+1127] (\usr\local\google\home\penghuang\sources\chromium\src\chrome\app\main_dll_loader_win.cc:199) wWinMain [0x000000014007E46A+1130] (\usr\local\google\home\penghuang\sources\chromium\src\chrome\app\chrome_exe_main_win.cc:230) invoke_main [0x00000001402AE0CD+45] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:123) __scrt_common_main_seh [0x00000001402AE207+295] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:283) __scrt_common_main [0x00000001402AE28E+14] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:326) wWinMainCRTStartup [0x00000001402AE2A9+9] (f:\dd\vctools\crt\vcstartup\src\startup\exe_wwinmain.cpp:17) BaseThreadInitThunk [0x00007FF94BC11FE4+20] RtlUserThreadStart [0x00007FF94C94EFB1+33]