While I agree that the switch to mojo addresses this issue, it's going to
be quite a long time before that completes.

Maybe all that is needed is a well defined way of restoring all macros to their "normal" values.

So today the generator code works by using this sequence:

#include "ipc/ipc_message_macros.h"  // "Dummy" include
#include "ipc/struct_constructor_macros.h"  // For instance
#include "ipc/ipc_channel_proxy_unittest_messages.h"  // Example message file

The message file will then include "ipc/ipc_message_macros.h" again but since it's already has been included, that is a nop and the macros in "ipc/struct_constructor_macros.h" are still active.

Making it so that "ipc/ipc_message_macros.h" always replaces existing macros will in other words not work unless other restructuring is done as well. 

An alternative is to remove the include of "ipc/ipc_message_macros.h" from inside the message files and put the burden on the includer to have included the right macros before the message file. This could be a really large change which breaks easily.

A third alternative is to split all message files into an inner file with the actual message data, and an outer file with the include of "ipc/ipc_message_macros.h" and an include of the inner file. Generator code would include the inner file. 

This would be robust but an even larger change. The best long term I think.

For jumbo in particular, there are workarounds. 

A fourth (possible but unwelcome) alternative is to jumbo_exclude_sources all generator files and a fifth alternative is to move generator file into their own targets so jumbo won't combine them with code using the messages.

Just taking a step back for a second: between this and the recent
namespaces-in-tests issue - and probably others I'm overlooking in the set
of unfortunate contortions entertained on behalf of jumbo builds - it seems
to me that we may not be taking the cleanest approach to addressing build
time issues.

Has the complexity grown beyond what was originally anticipated? How much
further do we have to go? Have we considered changing clang in a way that
allows us to get the behavior we want without imposing otherwise
unnecessary (and frankly hard to rationalize) constraints on our own source
code?

If you are talking about jumbo/unity builds, that is plan C and there is no plan D because this is the big hammer after all small hammers failed to have any impact. Compilation times now, on "ordinary" hardware, is 3 hours for a full build (possibly more because it increases by about 1% per week) but for you (I assume you work at Google), goma swallows most of it and high end hardware blunts the rest.

There are alternatives, such as making goma available to external contributors and that has been explored but is a dead end for several reasons. We have also explored building the unity build functionality into ninja but that would require forking ninja and would insert magic into the build system. Jumbo is after all 100% reproducible and predictable.

Making clang faster is not especially important to the clang maintainers so it's actually getting slower, not faster, and there is no way to make it dramatically faster.

The potential for jumbo is to make builds ~8x faster. Currently on master full builds with jumbo is 2-3x faster (for normal hardware, the effect lessens the more parallel hardware you have). My local Work-In-Progress is 4.3x faster with many areas untouched. Nothing else that anyone has proposed is even close.

About known complexities in adapting the source to work well in jumbo builds there are 2 non-trivial problems that I've been pondering the last half year:

1. IPC.
2. The duplicate code in tests

That I after half a year decided to take a serious look at IPC at the same time as content/tests is more random or possible an unconscious attempt at avoiding Christmas.

Unless I think of something better than what I documented above I will not touch IPC, but I was hoping for some rather clean change that would leave the code in a better/more robust place than when I started.

Thanks. I certainly didn't mean to imply that build times aren't terrible
or important outside of the goma case. I am at Google but I do somewhat
regularly end up building Chrome on cheaper hardware and without goma. It
is not fun.

I was mostly wondering if there were ways we could convince clang to
partially simulate separate compilation units while still processing
unified source blobs (e.g. separate concatenated files with some new
special #pragma). That could address the test namespacing issue in a
cleaner way than imposing explicit namespacing on developers, but I guess
it wouldn't really help with IPC message macro situation.

Maybe generated IPC code is small enough that we can tolerate nothing
changing here for now? And of course we will eventually get mojo
conversions done.

I'm happy you know what I'm talking about! Or well, my condolences. :-)

The idea with an clang feature that create "file local" namespaces out of anonymous namespaces was mentioned on chromium-dev but there are some downsides there make me think it's not the best path. For instance, there are still people using other compilers for various purposes (compiling the whole or part of the code) and those would be excluded. It would also still require code changes and there are places where anonymous namespaces cooperate across multiple files (the cc file and other included files) so it wouldn't work fully by itself anyway. Then there is the risk of tooling suddenly not working because of non-standard extensions. Maybe if I had been more involved and familiar with the clang project, I would see a path through such a project but not so at the moment.

> Maybe generated IPC code is small enough that we can tolerate nothing
changing here for now? 

There is certainly not that many IPC generator files around. The main difference is between not having any special cased files at all, and having one or two. I've tried to avoid all special casing to keep the build files clean and to prevent them from being an obvious and tempting way out that slowly eroded jumbo builds.

And also, one reason I filed this issue was to have something to refer to when creating such a jumbo_excluded_sources block.

My first experience with jumbo (though a different implementation) was for Presto at Opera. There were people that thought 15 minute compiles were ludicrous and they implemented jumbo and sent me patches to rename some macros and stuff. I was quite indifferent but after getting used to it, it just was and everyone used it and thought of it as one of the things that gave our code base an edge. I hope the same will be true for this, but I will settle for much less, like being able to ask people to upstream fixes without implicitly asking them to spend a full work day on it.

(Today a much larger Presto of 2+ million lines of code compiles in about 3 minutes on a normal desktop computer)

Oh, I see now what you mean about #pragmas. Something to clear out some namespaces and make new ones. That seems better than what I imagined at first. Still concerns but it addresses some of them.