goma should ensure the server is started when trying to build |
|
Issue descriptionIf goma is enabled, and you try to do a build but the server isn't running, then you will get inscrutable errors from ninja. It seems like we should be able to figure out a way to have goma reliably auto-start itself. One obvious way would be to do this in autoninja.py, by calling `goma_ctl.py ensure_start`, but we don't require everyone to use autoninja, and it's not obvious how we could require that (or if it would really be desirable). A second (probably preferable) way would be for the goma client wrapper itself to call ensure_start. I think we've tried this in the past and seen issues, but I'm not sure if we know for sure that we *can't* make this work reliably. Any other thoughts / options?
,
Jul 29
Goma used to do that at some point. I think it printed "goma not running; starting" from all processes and made them block until startup was done. That wasn't an awesome experience. I don't have an opinion on putting it in autoninja, but I'm reasonably confident that most people don't use that. imho nice to have, but not super important, given that we tried to do it, it didn't reallyw ork out, and things seem to be ok. *shrug*
,
Jul 30
I think you can try it with GOMA_START_COMPILER_PROXY=true for gomacc. it was not so great experience as thakis@ stated. I think it would be nice to put it in autoninja (but not so many people use it...), or add support in ninja to run some command at startup?
,
Jul 30
#3 I think it can be done to add such function in current ninja wrapper. https://cs.chromium.org/chromium/tools/depot_tools/ninja
,
Jul 30
why we don't put autoninja feature in the wrapper, then?
,
Jul 30
From my point of view, I enforce autoninja's feature including NINJA_SUMMARIZE_BUILD to users and let them know how much time is taken in the build. But that is too radical?
,
Jul 30
I don't think we should do this in ninja or ninja's wrapper. In the limit that leads to running build/install-build-deps.sh there too -- it seems like confusion of responsibilities to me.
,
Jul 30
> imho nice to have, but not super important, given that we tried to do it, > it didn't reallyw ork out, and things seem to be ok. *shrug* I'd probably describe this as "maybe tolerable", not "ok". This causes really confusing errors for the user, and I believe lots of users hit this. I'd like to understand the "didn't really work out" part better. This doesn't seem like an unsolvable problem, and I'm not sure what the downside of blocking the individual goma processes is, assuming the check to see if goma is started or not can be done fast enough (I'm guessing this could be done on the order of time it takes to stat() something). I agree we shouldn't try to push this into ninja itself.
,
Jul 31
I think it would not be good idea to put such feature in ninja wrapper in depot_tools. it would be confusing there is difference between native ninja vs depot_tools ninja. maybe, we should promote using autoninja in chromium (update docs to use autoninja instead of ninja, etc) and fix autoninja to start compiler_proxy if it is not started, or add goma specific feature? for "didn't really work out", https://chrome-internal-review.googlesource.com/c/goma/client/+/260255 hmm no BUG= in CL description. we should invoke goma_ctl.py instead of directly run compiler_proxy?
,
Jul 31
I understand that we might be starting 10s or 1000s of goma processes in a small amount of time, all of which would then try to stat the file (or whatever) to start things up, but I still feel like this should be a relatively solvable problem? I guess I don't know exactly what we saw before, but it looks like --start_compiler_proxy should still work, so I could test it out and see?
,
Aug 2
Re: #9 Just FYI, I searched our mailing list and found something related. PSA on stop supporting automatic start compiler_proxy from gomacc. https://goto.google.com/ziueb Several reasons why we recommend to start compiler_proxy with goma_ctl. https://goto.google.com/zhdgx How past gomacc behavior was ugly? https://goto.google.com/wijxq One of the issues we decided to stop auto start feature. b/28142555 List of things what other people do: - ChromeOS simple chrome automatically download & starts compiler_proxy when cros command is executed. - Android make automatically execute "goma_ctl.py ensure_start" if USE_GOMA is true. - fuchsia silently execute "goma_ctl.py ensure_start" when fx set is executed.
,
Aug 2
Executing "goma_ctl.py ensure_start" in autoninja seems to be something similar to what other folks do, and I believe it works well if people don't try to execute autoninja from two or more different locations at once.
,
Aug 2
Thanks for the context in #c11. I still don't really understand why this seems hard to make work properly, but I'm happy to change autoninja.py for now and assign this as a low priority thing for myself to look at some day, rather than taking up any more of the goma team's time on it. |
|
►
Sign in to add a comment |
|
Comment 1 by brucedaw...@chromium.org
, Jul 27