Measure binary size impact of Mojo vs. Chrome-IPC. |
||||||||||||||
Issue descriptionMojo does a lot of code generation for serialisation, and defining interfaces. So does Chrome-IPC, albeit via macros. We need to compare the binary size impact of converting IPCs to Mojo.
,
Apr 14 2016
Copy-paste some posts: amistry: One thing some of us have been wondering is what the effect of Mojo interfaces on binary size is. So, as a quick experiment, I put together a short change (https://codereview.chromium.org/1841743003/) and compared the before/after binary size on win/mac/linux. It's not a very good experiment since it doesn't do a apples-to-apples comparison of any particular IPC. There's also page-size granularity, so the number are terrible. But it shows some indication of the effect. Here are the results: Linux (stripped): GN config: is_component_build = false is_debug = false use_goma = true enable_nacl = false is_official_build = true GYP_DEFINES: chromeos=0 component=static_library use_goma=1 disable_nacl=1 asan=0 tsan=0 lsan=0 buildtype=Official GN GYP Base 112502440 112141848 With image_decoder.mojom 112514728 (+12288) 112150040 (+8192) OSX: GYP_DEFINES: component=static_library use_goma=1 disable_nacl=1 buildtype=Official mac_strip_release=1 Base 113730660 With image_decoder.mojom 113738916 (+8256) Windows: GYP_DEFINES: buildtype=Official chrome.dll chrome_child.dll Base 36073984 44631552 With image_decoder.mojom 36078080 (+4096) 44632064 (+512) ===================================================================================================== amistry: Following up, I looked at a pair of real IPC conversions. I have the the utility process image decoder (https://codereview.chromium.org/1844103004/) and JSON parser (https://codereview.chromium.org/1861573002/) and looked at the before and after size of chrome.dll and chrome_child.dll on windows. Like before, this was an "Official" buildtype, so it should gain a reduction in binary size due to LTO and identical-code-folding. file sizes: chrome.dll chrome_child.dll before 36126720 44521472 after 36129792 (+3072) 44526080 (+4608) code size: chrome.dll chrome_child.dll before 29023744 33872384 after 29026816 (+3072) 33876992 (+4608) (using objdump -x | grep SizeOfCode) Firstly, Windows PE sections appear to be rounded to 512 byte "pages", so take these with a grain of salt. But at a first approximation, almost all of the increase in binary size is due to the extra generated code ===================================================================================================== jam: Thanks, that's very helpful data. These two changes converted 8 IPCs, with roughly 7680 extra bytes. Say each binary had an empty half page, so that's 7K extra. To get a rough estimate of number of IPCs (avoiding chromecast/android/content_shell messages, but counting the small number of messages that are platform-specific), searching the following directories: content\common,chrome,components,ppapi,extensions;gpu for IPC_MESSAGE_CONTROL, IPC_MESSAGE_ROUTED, IPC_SYNC_MESSAGE_CONTROL, IPC_SYNC_MESSAGE_ROUTED gives 2155 messages. So in worst case scenario the binaries will grow by nearly 2MB. That's not excessive to slow down this effort, but we should understand why it's taking more. Can we tweak the binding generator to take out various pieces of the code to isolate this further? e.g.. remove the version checking code, which we don't really need. I suspect that one is small, but just as an example. ===================================================================================================== pasko: Please include Android into binary size considerations. This is probably the most important platform to care about binary size. I gave it a try with image_decoder.mojom patch with Android/ARM/32bit official build. Adds 8K to the current ~40MiB binary. So it looks that our projected number for Android is about the same - 2MiB (unless we have less IPCs on Android?). This is huge :(
,
Apr 14 2016
,
May 17 2016
,
Jul 26 2016
,
Jul 26 2016
This roughly projects to be 2MiB binary size regression on Android as a result of migrating to Mojo IPC. Changing P3 -> P2.
,
Jul 26 2016
,
Jul 26 2016
These are great findings! Thanks for digging through this earlier rather than later. Certainly sounds to me that we'll need to optimize the generated code more before continuing with the mojo migration.
,
Jul 26 2016
I ran Anand's Safe JSON change (https://codereview.chromium.org/1861573002/, checked in as ab6f6ae93d12cf5913efbb7f1f1f6c34e2cab39b) through tools/binary_size/explain_binary_size_delta.py and here's the top 18 symbols by size increase (I've attached the complete report): Total change: +6828 bytes ========================= 138 added, totalling +10254 bytes across 39 sources 95 removed, totalling -4497 bytes across 29 sources 4 grown, for a net change of +1215 bytes (944 bytes before, 2159 bytes after) across 2 sources 77 shrunk, for a net change of -144 bytes (686 bytes before, 542 bytes after) across 4 sources 386624 unchanged, totalling 84143956 bytes Source stats: 19453 sources encountered. 3 completely new. 3 removed completely. 59 partially changed. 19388 completely unchanged. Per-source Analysis: ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- +3547 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/out/size/gen/components/safe_json/public/interfaces/safe_json.mojom.cc - (gained 3547, lost 0) ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- New symbols: +784: safe_json::mojom::SafeJsonParserStub::AcceptWithResponder(mojo::Message*, mojo::MessageReceiverWithStatus*) type=t, size=784 bytes +544: safe_json::mojom::SafeJsonParser_Parse_ProxyToResponder::Run(base::ListValue const&, mojo::String const&) type=t, size=544 bytes +481: safe_json::mojom::SafeJsonParserResponseValidator::Accept(mojo::Message*) type=t, size=481 bytes +444: safe_json::mojom::SafeJsonParserProxy::Parse(mojo::String const&, mojo::Callback<void (base::ListValue const&, mojo::String)> const&) type=t, size=444 bytes +426: safe_json::mojom::SafeJsonParserRequestValidator::Accept(mojo::Message*) type=t, size=426 bytes +381: safe_json::mojom::SafeJsonParser_Parse_ForwardToCallback::Accept(mojo::Message*) type=t, size=381 bytes ----------------------------------------------------------------------------------------------------------------------------------------------- +980 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/components/safe_json/safe_json_parser_impl.cc - (gained 1600, lost 620) ----------------------------------------------------------------------------------------------------------------------------------------------- New symbols: +358: safe_json::SafeJsonParserImpl::~SafeJsonParserImpl() type=t, size=358 bytes Grown symbols: +952: safe_json::SafeJsonParserImpl::StartWorkOnIOThread() type=t, (was 360 bytes, now 1312 bytes) ---------------------------------------------------------------------------------------------------------------------------------------- +723 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/mojo/public/cpp/bindings/strong_binding.h - (gained 723, lost 0) ---------------------------------------------------------------------------------------------------------------------------------------- New symbols: +334: mojo::StrongBinding<safe_json::mojom::SafeJsonParser>::Bind(mojo::InterfaceRequest<safe_json::mojom::SafeJsonParser>) type=t, size=334 bytes +217: mojo::StrongBinding<safe_json::mojom::SafeJsonParser>::~StrongBinding() type=t, size=217 bytes ----------------------------------------------------------------------------------------------------------------------- +690 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/ipc/ipc_message_utils.cc - (gained 690, lost 0) ----------------------------------------------------------------------------------------------------------------------- New symbols: +683: IPC::(anonymous namespace)::GetValueSize(base::PickleSizer*, base::Value const*, int) type=t, size=683 bytes --------------------------------------------------------------------------------------------------------------------------------------------------------- +688 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/components/safe_json/utility/safe_json_parser_mojo_impl.cc - (gained 688, lost 0) --------------------------------------------------------------------------------------------------------------------------------------------------------- New symbols: +561: safe_json::SafeJsonParserMojoImpl::Parse(mojo::String const&, mojo::Callback<void (base::ListValue const&, mojo::String)> const&) type=t, size=561 bytes ------------------------------------------------------------------------------------------------------------------------------------------- +575 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/mojo/public/cpp/bindings/lib/binding_state.h - (gained 575, lost 0) ------------------------------------------------------------------------------------------------------------------------------------------- New symbols: +572: mojo::internal::BindingState<safe_json::mojom::SafeJsonParser, false>::Bind(mojo::ScopedHandleBase<mojo::MessagePipeHandle>, scoped_refptr<base::SingleThreadTaskRunner>) type=t, size=572 bytes ------------------------------------------------------------------------------------------------------------------------------------------------- +546 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/mojo/public/cpp/bindings/lib/interface_ptr_state.h - (gained 546, lost 0) ------------------------------------------------------------------------------------------------------------------------------------------------- New symbols: +429: mojo::internal::InterfacePtrState<safe_json::mojom::SafeJsonParser, false>::ConfigureProxyIfNecessary() type=t, size=429 bytes --------------------------------------------------------------------------------------------------------------------------------------------------------------- +373 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/out/size/gen/blink/bindings/modules/v8/V8USBInTransferResult.cpp - (gained 373, lost 0) --------------------------------------------------------------------------------------------------------------------------------------------------------------- New symbols: +373: blink::USBInTransferResultV8Internal::dataAttributeGetterCallback(v8::FunctionCallbackInfo<v8::Value> const&) type=t, size=373 bytes -------------------------------------------------------------------------------------------------------------------------------------------- +361 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/third_party/pdfium/fpdfsdk/fxedit/fxet_ap.cpp - (gained 361, lost 0) -------------------------------------------------------------------------------------------------------------------------------------------- New symbols: +361: CPVT_GenerateAP::GetFontSetString(IPVT_FontMap*, int, float) type=t, size=361 bytes -------------------------------------------------------------------------------------------------------------------------------------------------------------------- +290 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/out/size/gen/blink/bindings/modules/v8/V8CanvasRenderingContext2D.cpp - (gained 290, lost 0) -------------------------------------------------------------------------------------------------------------------------------------------------------------------- New symbols: +290: blink::OffscreenCanvasRenderingContext2DV8Internal::lineCapAttributeSetterCallback(v8::FunctionCallbackInfo<v8::Value> const&) type=t, size=290 bytes --------------------------------------------------------------------------------------------------------------------------------------- +235 - Source: /usr/local/google/home/tibell/src/chromium/src/out/size/content/public/common/service_registry.h - (gained 235, lost 0) --------------------------------------------------------------------------------------------------------------------------------------- New symbols: +233: void content::ServiceRegistry::AddService<safe_json::mojom::SafeJsonParser>(base::Callback<void (mojo::InterfaceRequest<safe_json::mojom::SafeJsonParser>), (base::internal::CopyMode)1>) type=t, size=233 bytes
,
Jul 27 2016
The above was built on Linux using: is_component_build = false is_debug = false use_goma = true enable_nacl = false is_official_build = true
,
Aug 1 2016
I looked through all the assembly for SafeJsonParserResponseValidator::Accept, the 3rd largest function in the generated .mojom.cc file and its instructions breaks down roughly as: * 33% destructors (mostly of std::string) * 33% deserialization (of mojo::String and base::ListValue) * 33% misc (move arguments in place for callback, function prologue/epilogue, report errors, etc) There are no obvious savings, except that if std::string wasn't refcounted we'd generate less destructor code.
,
Aug 1 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/029b671b558428d07b0054e83451253e3a5db225 commit 029b671b558428d07b0054e83451253e3a5db225 Author: tibell <tibell@chromium.org> Date: Mon Aug 01 02:57:11 2016 Move generated Mojo C++ error reporting code out of line Also switch from scoped_refptr to unique_ptr, as we only have one owner. Savings: SafeJsonParserStub::AcceptWithResponder: 42 bytes SafeJsonParser_Parse_ForwardToCallback: 44 bytes We lose a few bytes (but come out positive in the end) due to fixing the incorrect Ninja template expansions in the error messages. BUG= 597125 Review-Url: https://codereview.chromium.org/2188143002 Cr-Commit-Position: refs/heads/master@{#408901} [modify] https://crrev.com/029b671b558428d07b0054e83451253e3a5db225/mojo/public/cpp/bindings/lib/validation_errors.cc [modify] https://crrev.com/029b671b558428d07b0054e83451253e3a5db225/mojo/public/cpp/bindings/lib/validation_errors.h [modify] https://crrev.com/029b671b558428d07b0054e83451253e3a5db225/mojo/public/tools/bindings/generators/cpp_templates/interface_definition.tmpl
,
Aug 1 2016
I also looked into removing some strings in e.g. validation error messages when building an optimized ("official") build. The saving is <100 bytes and complicates the code a bit with #ifdefs, so I don't know if it's worth it.
,
Aug 1 2016
I tried to roughly break down where the +6828 bytes growth in switching the safe JSON parser to Mojo comes from. This is mostly from either template expansion (e.g. InterfacePtr functions) or code generation (the .mojom.cc file). Connection management: +2500 bytes * InterfacePtr * InterfaceRegistry * (Strong)Binding Callback/bind: +1400 bytes * We use more callbacks in Mojo than Chrome IPC (e.g. for returning a response) New classes/abstractions: +2700 bytes * These is the the generated proxies, proxy adapters, etc in the .mojom.cc file, minus the parts already included above and adjusted for e.g. Chrome IPC message dispatch code it displaces. * This part is a bit hard to measure due to code inlined (e.g. from callbacks) into functions in the .mojom.cc file functions. These numbers are again from Linux.
,
Aug 1 2016
(I should also add that these numbers don't include any LTO that may or may not happen. However, generating less code is probably a better approach than hoping for LTO to remove it.)
,
Aug 4 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ad0c0b3bab4a19dfafb51bda1dad56024c2dc9e2 commit ad0c0b3bab4a19dfafb51bda1dad56024c2dc9e2 Author: tibell <tibell@chromium.org> Date: Thu Aug 04 01:13:30 2016 Reduce code size of BindingState BindingState is a templated class that gets expanded for every Mojo impl, causing code bloat. By moving the functions that don't depend on the template parameter out of line we save roughly 2.3k bytes in safe_json.mojom (on Linux). I expect similar gains for every .mojom. BUG= 597125 Review-Url: https://codereview.chromium.org/2203243002 Cr-Commit-Position: refs/heads/master@{#409691} [modify] https://crrev.com/ad0c0b3bab4a19dfafb51bda1dad56024c2dc9e2/mojo/public/cpp/bindings/BUILD.gn [add] https://crrev.com/ad0c0b3bab4a19dfafb51bda1dad56024c2dc9e2/mojo/public/cpp/bindings/lib/binding_state.cc [modify] https://crrev.com/ad0c0b3bab4a19dfafb51bda1dad56024c2dc9e2/mojo/public/cpp/bindings/lib/binding_state.h [modify] https://crrev.com/ad0c0b3bab4a19dfafb51bda1dad56024c2dc9e2/mojo/public/cpp/bindings/lib/filter_chain.cc [modify] https://crrev.com/ad0c0b3bab4a19dfafb51bda1dad56024c2dc9e2/mojo/public/cpp/bindings/lib/filter_chain.h
,
Aug 11 2016
I've spent quite some time trying to reduce the increase on the client side of things. The main growth here is in InterfacePtrState::ConfigureProxyIfNecessary, which is used internally by each template instantiation of InterfacePtr. There's also some growth due to the use of callbacks to return the result. You can see my fledgling attempts here: https://codereview.chromium.org/2205193002/. My approach was to try to share common code in ConfigureProxyIfNecessary by moving it to a .cc file, which had quite a lot of knock-on effects in other parts of the code base. Note that ConfigureProxyIfNecessary does contain some true per-interface variation in terms of the ResponseValidator_ so we can't hope to remove all the generated code here. After several different approaches (one of them in the CL above) I'm convinced that there's not much, if anything to be gained here. The compiler does a good job already of sharing most of the generated templated code between different instantiation. Any size gains likely will have to come from the server side of things.
,
Aug 16 2016
> Any size gains likely will have to come from the server side of things. what do you mean by server side of things? > Note that ConfigureProxyIfNecessary does contain some true per-interface > variation in terms of the ResponseValidator_ so we can't hope to remove all > the generated code here. Mojo noob question: is this some sort of complex validation of what was sent on the the other side? If so, I do not see why it should be used in production code at any cost in binary size. Did Chrome IPC provide all the same validation guarantees?
,
Aug 16 2016
Sorry for the confusing terminology. By the "server side" I mean the side that implements the interface and its methods. Someone else will have to confirm, since I don't have much experience with the old implementation, but I believe we do more sender-side validation than we used to.
,
Aug 17 2016
> Note that ConfigureProxyIfNecessary does contain some true per-interface > variation in terms of the ResponseValidator_ so we can't hope to remove all > the generated code here. > Mojo noob question: is this some sort of complex validation of what was sent on > the the other side? > If so, I do not see why it should be used in production code at any cost in > binary size. Did Chrome IPC provide all the same validation guarantees? Here we are talking about InterfacePtr, which is the "client side" of a mojo interface. The incoming messages are responses to requests made with this interface ptr. Because the "service side" of a mojo interface could be in an untrusted process, we need to verify that those responses are valid: 1) a response should correspond to a previous request; 2) the data it carries should conform to the mojom definition so we can deserialize it safely. The old Chrome IPC doesn't know about request/response (except for sync messages), so it doesn't do (1), the application should manually do that with a request ID if necessary. It does check whether it can safely read data out of the received message, which is similar to (2).
,
Aug 18 2016
,
Sep 28 2016
Is it possible to know when the "service side" lives in a trusted process? And if so, could we strip the sanity checks in this case?
,
Nov 3 2016
As another data point, the following change converted App Banners from IPC -> Mojo. https://codereview.chromium.org/2393513004 It made the libchrome.so on Android 8kb larger (according to https://chromeperf.appspot.com/report?sid=f8be4459943df7093c2c0cdfa83b9350c742c4da55093717a3f0633ac95b5e2a&num_points=1500)
,
Nov 3 2016
RE #22: The idea of skipping validation at run time for messages coming from trusted processes has been proposed before. It is not easy to get right because message pipe handles could be transferred across processes. I assume you are proposing to remove generated validation code altogether so it is not exactly the same. That is an interesting idea. It would require a lot of careful thinking to ensure security, too. Maybe we could measure how much space validation takes up before seriously considering such approach. Thanks for the input!
,
Nov 3 2016
Random idea: add a compile-time flag somewhere to say that the pipe can be used only between trusted endpoints, with this flag it would DCHECK on sending the handle cross-process Flexibility of passing pipes around is nice, but the costs are pretty high. We grew the native binary size by 6MiB in the last 6 months (I believe part of that is due to Mojo) which makes users disable Chrome updates in emerging markets, where 6MiB of mobile traffic comes at non-trivial price compared to income.
,
Nov 3 2016
Thanks for reply! Not suggesting that we shouldn't consider this. Just think that it would be nice to get some data first and see how much this could gain us. Validation code is just part of the auto-generated bindings, which also do serialization/deserialization, message dispatching, callback adapter, etc. Also, we can only skip validation for some of the interfaces (and usually just one direction of those interfaces). That is why I feel like to see some data first. :)
,
Nov 8 2016
Another 24kb regression from: Port messages sent by WebIDBDatabaseImpl to Mojo. https://codereview.chromium.org/2449953008/
,
Nov 8 2016
Trust is not a binary property. There aren't simply "trusted" or "untrusted" endpoints, at least not in the limit. Even if we were to deem certain interfaces as trustworthy and mark them as such to avoid generating validation code, this would only affect a small minority of the total interfaces defined in the system. Why don't we first try building with all validation code omitted, just to see how it affects binary size?
,
Nov 8 2016
Also if we do a lot of sender-side validation, it should be changed to only build when DCHECK_IS_ON. IIRC some (most?) of it is already in the form of DCHECKs though.
,
Nov 8 2016
#28: sounds good to omit all validation code and see what the upper bound is. I don't have access to my machine now, could try it out tomorrow. #29: I think I have been careful when I introduced sender-sider validation which should be optimized away when DCHECK is off. And we have very little sender-side validation: fixed-array size mismatch, unexpected null pointer, associated endpoints sent on wrong message pipe.
,
Nov 9 2016
Here comes the data: I tried removing all validation of method request/response, struct and union. (CL: crbug.com/2491583004). I then built chrome with/without the CL on Linux with the following gn args: is_component_build = false is_debug = false use_goma = true enable_nacl = false is_official_build = true Results: With validation: 120429384 Without validation: 119974728 The saving is ~454K. Please note that this is a pretty optimistic upper bound. We can only omit validation for a subset of interfaces, and usually only one direction (either client->service messages or the other way around). And if a struct/union could be used in both trusted/untrusted messages, its validation cannot be omitted.
,
Nov 10 2016
Thanks for the data! Awesome! 2rockot on: > Even if we were to deem certain interfaces as trustworthy and mark them as > such to avoid generating validation code, this would only affect a small > minority of the total interfaces defined in the system. Can you help me gain the same intuition regarding the fraction of trusted interfaces and total interfaces? 450K for validation code would be a lot, do you have a lower bound? How many interfaces / mojo pipes is this 450K coming from? How many more pipes are there to build? Perhaps there are slower but more code-compact ways to validate? Then we could do it on non-performance-critical interfaces.
,
Jan 9 2017
,
Feb 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/83392e9f4312a231f60c5d2b8f0bfa4f80aa92cd commit 83392e9f4312a231f60c5d2b8f0bfa4f80aa92cd Author: scottmg <scottmg@chromium.org> Date: Thu Feb 09 18:46:45 2017 Reduce size of mojo AssociatedBinding template expansions In particular for AssociatedBinding::Bind(), reduces expansion to 54% of original size on Windows. This only saves about 4K on the final libchrome.so size on Android, so not a huge win, but: 1) it doesn't add much complexity to the implementation, and 2) I think it'll be useful to do more followups that de-type bindings code. R=yzshen@chromium.org BUG= 597125 ,689690 Review-Url: https://codereview.chromium.org/2680243003 Cr-Commit-Position: refs/heads/master@{#449364} [modify] https://crrev.com/83392e9f4312a231f60c5d2b8f0bfa4f80aa92cd/mojo/public/cpp/bindings/BUILD.gn [modify] https://crrev.com/83392e9f4312a231f60c5d2b8f0bfa4f80aa92cd/mojo/public/cpp/bindings/associated_binding.h [add] https://crrev.com/83392e9f4312a231f60c5d2b8f0bfa4f80aa92cd/mojo/public/cpp/bindings/lib/associated_binding.cc
,
Apr 3 2017
Another 20kb increase from: https://codereview.chromium.org/2744963002 "Introduce InterfaceEndpointClient(IEC), InterfaceEndpointHandle and PipeControlMessage Handler/Proxy"
,
Apr 11 2017
One effort to try and reduce overhead: bug 689690
,
Apr 13 2017
Another 20kb increase: bug 710967
,
May 9 2017
,
May 29 2017
Just updating bug statuses so that "Available" means available. If you don't want to own this anymore, please revert it to Available and remove the owner value. Thanks!
,
May 29 2017
,
May 29 2018
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue. Sorry for the inconvenience if the bug really should have been left as Available. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jul 19
Issue 865076 : App Service causes a 28KiB regression. This is for a Service with just a tiny 2 method interface, 2 mojo enums, and a mojo struct.
,
Nov 6
|
||||||||||||||
►
Sign in to add a comment |
||||||||||||||
Comment 1 by amistry@chromium.org
, Apr 14 2016