chromium.win Win Builder flaky because of goma |
||||
Issue descriptionTwo recent builds on https://build.chromium.org/p/chromium.win/builders/Win%20Builder have failed the compile step because goma flaked out. https://build.chromium.org/p/chromium.win/builders/Win%20Builder/builds/41774 https://build.chromium.org/p/chromium.win/builders/Win%20Builder/builds/41776 Sample build output: [17190/17869] CXX obj/chrome/test/browser_tests/captive_portal_browsertest.obj FAILED: obj/chrome/test/browser_tests/captive_portal_browsertest.obj ninja -t msvc -e environment.x86 -- C:\b\c\goma_client/gomacc.exe "c:\b\c\win_toolchain\vs_files\f53e4598951162bad6330f7a167486c7ae5db1e5\vc\bin\amd64_x86/cl.exe" /nologo /showIncludes @obj/chrome/test/browser_tests/captive_portal_browsertest.obj.rsp /c ../../chrome/browser/captive_portal/captive_portal_browsertest.cc /Foobj/chrome/test/browser_tests/captive_portal_browsertest.obj /Fd"obj/chrome/test/browser_tests_cc.pdb" E0712 13:00:07.033089 15808 named_pipe_client_win.cc:45] Timed-out to create new pipe:\\.\pipe\goma.ipc compiler proxy isn't running. Exiting. In 41774, there were many such messages right around each other. I believe the timeout error, but I don’t know if I believe “compiler proxy isn’t running” because some compilations succeed again after a batch of failures. There’s a nuisance tree closure every time this happens.
,
Jul 13 2017
Sorry for inconvenience. Let me increase timeout for namedpipe.
,
Jul 13 2017
,
Jul 13 2017
Increasing the timeout will help, but it won't necessarily solve the problem and it will make things worse in the case where compiler_proxy.exe actually isn't running. One possible solution would be an initial timeout that is the current length, then a check for the existence of compiler_proxy.exe before waiting again. Getting entirely away from timeouts would also be nice - they are a bit of a hack - but I don't know if that is practical.
,
Jul 14 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/goma/client/+/50f5b57eba0928b9cb2f06a80cba067bb51d2a36 commit 50f5b57eba0928b9cb2f06a80cba067bb51d2a36 Author: Takuto Ikuta <tikuta@google.com> Date: Fri Jul 14 02:27:56 2017
,
Jul 14 2017
I left some notes on the code as checked in at https://chrome-internal-review.googlesource.com/c/410328#message-ef34c7dff8647ac3187bbb843f01614130b9a8b8, and some on the broader concept of using named pipes on Windows for IPC at https://chrome-internal-review.googlesource.com/c/410328#message-f73304ae15641fc4fb691374aea30c3a154bbf79.
,
Jul 19 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/goma/client/+/f0e5aeb27f8004406297a1d84181a23122442230 commit f0e5aeb27f8004406297a1d84181a23122442230 Author: Takuto Ikuta <tikuta@google.com> Date: Wed Jul 19 04:49:59 2017
,
Jun 2 2018
Is this still happening?
,
Jun 5 2018
I think goma client has mitigation of this issue but the same issue may occur when people increase -j. Discussion thread is b/70640154.
,
Jun 6 2018
it doesn't happen on builder now (as we tune -j appropriately) ? close this, or change bug title?
,
Jul 26
I am seeing this issue locally. Is there anything I can do to get around this?
,
Jul 27
We are tracking the issue in b/70640154 But currently only the way to prevent this issue is to use smaller -j. |
||||
►
Sign in to add a comment |
||||
Comment 1 by hinoka@chromium.org
, Jul 12 2017