goma build with a high -j value spawns hundreds of python processes for bindings |
||||
Issue description
This is on my Mac Pro 2013 (Trashcan).
What steps will reproduce the problem?
(1) ninja -C out/Release content_shell -j600
What is the expected output?
Should only start {number of cores} local jobs, in my case 24.
What do you see instead?
600 python jobs start, this makes the bindings step take ages (5-10m) by being bottle necked on the disk throughput. If I do ninja -C ... -j 24 for the bindings step it only takes a few seconds instead of many minutes.
,
Oct 3 2016
peria@: Is this related to your change that has increased # of aggregated files?
,
Oct 3 2016
better to specify pool for binding rules in ninja? https://ninja-build.org/manual.html#ref_pool
,
Oct 3 2016
Re #2; Yes, http://crrev.com/417935, which started to compile each generated file in parallel, could trigger this issue, but the root issue is in ninja and goma, as commented in #0. Ukai-san, can't goma limit the number of local-fallback processes?
,
Oct 3 2016
I think binding generator doesn't run under gomacc, so it's out of control of goma.
,
Oct 3 2016
Ah, I see. This issue is not in compiling C++ files, but in generating them. Then, my CL in #4 is not related, and this issue should be existing for long.
,
Oct 20 2016
,
Mar 2 2017
|
||||
►
Sign in to add a comment |
||||
Comment 1 by esprehn@chromium.org
, Sep 30 2016