New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 717348 link

Starred by 3 users

Issue metadata

Status: Archived
Owner:
Last visit > 30 days ago
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocking:
issue 717347



Sign in to add a comment

When running on Swarming on Windows, Layout Tests leaves processes + temp files around

Project Member Reported by tansell@chromium.org, May 2 2017

Issue description

You see the following output at the end of the run;
----------------------------
Failed to delete e:\b\s\w\ir (11 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 2 seconds.
Failed to delete e:\b\s\w\ir (11 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 4 seconds.
Failed to delete e:\b\s\w\ir. The following files remain:
- \\?\e:\b\s\w\ir\out\Release\AHEM____.TTF
- \\?\e:\b\s\w\ir\out\Release\content_shell.exe
- \\?\e:\b\s\w\ir\out\Release\content_shell.log
- \\?\e:\b\s\w\ir\out\Release\content_shell.pak
- \\?\e:\b\s\w\ir\out\Release\icudtl.dat
- \\?\e:\b\s\w\ir\out\Release\natives_blob.bin
- \\?\e:\b\s\w\ir\out\Release\osmesa.dll
- \\?\e:\b\s\w\ir\out\Release\snapshot_blob.bin
- \\?\e:\b\s\w\ir\out\Release
- \\?\e:\b\s\w\ir\out
- \\?\e:\b\s\w\ir
Enumerating processes:
- pid 3636; Handles: 221; Exe: e:\b\s\w\ir\out\Release\content_shell.exe; Cmd: "e:\b\s\w\ir\out\Release\content_shell.exe" --type=renderer --disable-display-list-2d-canvas --disable-threaded-compositing --enable-experimental-web-platform-features --enable-inband-text-tracks --enable-logging --enable-precise-memory-info --force-device-scale-factor=1.0 --ignore-autoplay-restrictions --register-pepper-plugins="e:\b\s\w\ir\out\Release\blink_test_plugin.dll#Blink Test Plugin#Interesting description.#0.8;application/x-blink-test-plugin#blinktestplugin,e:\b\s\w\ir\out\Release\blink_deprecated_test_plugin.dll#Blink Deprecated Test Plugin#Description#0.1;application/x-blink-deprecated-test-plugin#blinkdeprecatedtestplugin" --touch-events=enabled --use-gl=osmesa --disable-threaded-animation --enable-gpu-benchmarking --service-pipe-token=F7ABFBE0897FA7854D001DFBE69899BE --lang=en-US --run-layout-test --enable-crash-reporter --crash-dumps-dir="e:\b\s\w\ir\out\Release\crash-dumps" --register-font-files="e:\b\s\w\ir\out\Release\/AHEM____.TTF" --enable-pinch --device-scale-factor=1 --num-raster-threads=4 --enable-main-frame-before-activation --content-image-texture-target=0,0,3553;0,1,3553;0,2,3553;0,3,3553;0,4,3553;0,5,3553;0,6,3553;0,7,3553;0,8,3553;0,9,3553;0,10,3553;0,11,3553;0,12,3553;0,13,3553;0,14,3553;0,15,3553;0,16,3553;1,0,3553;1,1,3553;1,2,3553;1,3,3553;1,4,3553;1,5,3553;1,6,3553;1,7,3553;1,8,3553;1,9,3553;1,10,3553;1,11,3553;1,12,3553;1,13,3553;1,14,3553;1,15,3553;1,16,3553;2,0,3553;2,1,3553;2,2,3553;2,3,3553;2,4,3553;2,5,3553;2,6,3553;2,7,3553;2,8,3553;2,9,3553;2,10,3553;2,11,3553;2,12,3553;2,13,3553;2,14,3553;2,15,3553;2,16,3553;3,0,3553;3,1,3553;3,2,3553;3,3,3553;3,4,3553;3,5,3553;3,6,3553;3,7,3553;3,8,3553;3,9,3553;3,10,3553;3,11,3553;3,12,3553;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --service-request-channel-token=F7ABFBE0897FA7854D001DFBE69899BE --renderer-client-id=3 --mojo-platform-channel-handle=1544 /prefetch:1
- pid 3768; Handles: 473; Exe: e:\b\s\w\ir\out\Release\content_shell.exe; Cmd: e:\b\s\w\ir\out\Release\content_shell.exe --run-layout-test --enable-direct-write --enable-crash-reporter --crash-dumps-dir=e:\b\s\w\ir\out\Release\crash-dumps -
- pid 4188; Handles: 165; Exe: e:\b\s\w\ir\out\Release\content_shell.exe; Cmd: "e:\b\s\w\ir\out\Release\content_shell.exe" --type=gpu-process --disable-gpu-rasterization --enable-logging --run-layout-test --enable-crash-reporter --crash-dumps-dir="e:\b\s\w\ir\out\Release\crash-dumps" --register-font-files="e:\b\s\w\ir\out\Release\/AHEM____.TTF" --use-gl=osmesa --supports-dual-gpus=false --gpu-vendor-id=0x0000 --gpu-device-id=0x0000 --gpu-driver-vendor --gpu-driver-version --gpu-driver-date --run-layout-test --enable-crash-reporter --crash-dumps-dir="e:\b\s\w\ir\out\Release\crash-dumps" --register-font-files="e:\b\s\w\ir\out\Release\/AHEM____.TTF" --enable-logging --service-request-channel-token=58C133DE3FAD29979A8B1999A1A1A4DA --mojo-platform-channel-handle=1112 --ignored=" --type=renderer " /prefetch:2
- pid 5892; Handles: 256; Exe: e:\b\s\w\ir\out\Release\content_shell.exe; Cmd: "e:\b\s\w\ir\out\Release\content_shell.exe" --type=renderer --disable-display-list-2d-canvas --disable-threaded-compositing --enable-experimental-web-platform-features --enable-inband-text-tracks --enable-logging --enable-precise-memory-info --force-device-scale-factor=1.0 --ignore-autoplay-restrictions --register-pepper-plugins="e:\b\s\w\ir\out\Release\blink_test_plugin.dll#Blink Test Plugin#Interesting description.#0.8;application/x-blink-test-plugin#blinktestplugin,e:\b\s\w\ir\out\Release\blink_deprecated_test_plugin.dll#Blink Deprecated Test Plugin#Description#0.1;application/x-blink-deprecated-test-plugin#blinkdeprecatedtestplugin" --touch-events=enabled --use-gl=osmesa --disable-threaded-animation --enable-gpu-benchmarking --service-pipe-token=04C00583A81FFA1951DB14EBA671C821 --lang=en-US --run-layout-test --enable-crash-reporter --crash-dumps-dir="e:\b\s\w\ir\out\Release\crash-dumps" --register-font-files="e:\b\s\w\ir\out\Release\/AHEM____.TTF" --enable-pinch --device-scale-factor=1 --num-raster-threads=4 --enable-main-frame-before-activation --content-image-texture-target=0,0,3553;0,1,3553;0,2,3553;0,3,3553;0,4,3553;0,5,3553;0,6,3553;0,7,3553;0,8,3553;0,9,3553;0,10,3553;0,11,3553;0,12,3553;0,13,3553;0,14,3553;0,15,3553;0,16,3553;1,0,3553;1,1,3553;1,2,3553;1,3,3553;1,4,3553;1,5,3553;1,6,3553;1,7,3553;1,8,3553;1,9,3553;1,10,3553;1,11,3553;1,12,3553;1,13,3553;1,14,3553;1,15,3553;1,16,3553;2,0,3553;2,1,3553;2,2,3553;2,3,3553;2,4,3553;2,5,3553;2,6,3553;2,7,3553;2,8,3553;2,9,3553;2,10,3553;2,11,3553;2,12,3553;2,13,3553;2,14,3553;2,15,3553;2,16,3553;3,0,3553;3,1,3553;3,2,3553;3,3,3553;3,4,3553;3,5,3553;3,6,3553;3,7,3553;3,8,3553;3,9,3553;3,10,3553;3,11,3553;3,12,3553;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --service-request-channel-token=04C00583A81FFA1951DB14EBA671C821 --renderer-client-id=5 --mojo-platform-channel-handle=1556 /prefetch:1
- pid 6156; Handles: 650; Exe: e:\b\s\w\ir\out\Release\content_shell.exe; Cmd: e:\b\s\w\ir\out\Release\content_shell.exe --run-layout-test --enable-direct-write --enable-crash-reporter --crash-dumps-dir=e:\b\s\w\ir\out\Release\crash-dumps -
- pid 6560; Handles: 165; Exe: e:\b\s\w\ir\out\Release\content_shell.exe; Cmd: "e:\b\s\w\ir\out\Release\content_shell.exe" --type=gpu-process --disable-gpu-rasterization --enable-logging --run-layout-test --enable-crash-reporter --crash-dumps-dir="e:\b\s\w\ir\out\Release\crash-dumps" --register-font-files="e:\b\s\w\ir\out\Release\/AHEM____.TTF" --use-gl=osmesa --supports-dual-gpus=false --gpu-vendor-id=0x0000 --gpu-device-id=0x0000 --gpu-driver-vendor --gpu-driver-version --gpu-driver-date --run-layout-test --enable-crash-reporter --crash-dumps-dir="e:\b\s\w\ir\out\Release\crash-dumps" --register-font-files="e:\b\s\w\ir\out\Release\/AHEM____.TTF" --enable-logging --service-request-channel-token=FD9A7DC59A8171B341DD42F794CC440B --mojo-platform-channel-handle=1132 --ignored=" --type=renderer " /prefetch:2
Terminating 6 processes.
- 3636 killed
- 3768 killed
- 4188 killed
- 5892 killed
- 6156 killed
- 6560 killed
Enumerating processes:
Failed to delete the run directory, forcibly failing
the task because of it. No zombie process can outlive a
successful task run and still be marked as successful.
Fix your stuff.
----------------------------

Looks like a bunch of content shell processes have not been cleaned up?
 
Cc: tansell@chromium.org
This problem seems to be occuring when the test exit early.

When the test succeeds it seems to clean up properly?
Not just windows; also seen on ChromeOS here:
https://uberchromegw.corp.google.com/i/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%281%29/builds/38292

Swarming results:
https://chromium-swarm.appspot.com/task?id=36073009a5929c10&refresh=10&show_raw=1

Failure:
SUCCESS: all tests passed.
Failed to delete /b/s/w/ir (3 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 2 seconds.
Failed to delete /b/s/w/itXMuAJ2 (3 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 2 seconds.
Failed to delete /b/s/w/itXMuAJ2 (2 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 4 seconds.
Failed to delete /b/s/w/itXMuAJ2. The following files remain:
- /b/s/w/itXMuAJ2
- /b/s/w/itXMuAJ2
23927 2017-05-09 18:09:53.268 E: Failure with [Errno 2] No such file or directory: '/b/s/w/itXMuAJ2'
Failed to delete the temporary directory, forcibly failing
the task because of it. No zombie process can outlive a
successful task run and still be marked as successful.
Fix your stuff.
The ChromeOS thing you are linking to is browser_tests, not layout tests. Please file a separate bug for that.
I had thought it was only occuring when test failed. However, I just found a case where the layout tests succeeded but still left a process running;

--------------
21:50:44.951 7372   crypto/subtle/aes-key-algorithm-mismatch.html took 0.0 seconds
21:50:44.951 7372   virtual/sharedarraybuffer/crypto/subtle/hkdf/importKey-failures.html took 0.0 seconds
21:50:44.951 7372   virtual/sharedarraybuffer/crypto/subtle/rsa-pss/verify.html took 0.0 seconds
21:50:44.951 7372   virtual/sharedarraybuffer/crypto/subtle/unwrapKey-badParameters.html took 0.0 seconds
21:50:44.952 7372   crypto/subtle/hkdf/exportKey.html took 0.0 seconds
21:50:44.952 7372   virtual/sharedarraybuffer/crypto/subtle/hmac/generateKey-failures.html took 0.0 seconds
21:50:44.952 7372   virtual/sharedarraybuffer/crypto/subtle/sign-verify-badParameters.html took 0.0 seconds
21:50:44.952 7372   virtual/threaded/fast/scroll-behavior/scroll-element-without-overflow.html took 0.0 seconds
21:50:44.952 7372   external/wpt/encrypted-media/drm-mp4-setmediakeys-multiple-times-with-the-same-mediakeys.html took 0.0 seconds
21:50:44.952 7372   virtual/threaded/inspector/tracing/timeline-paint/layer-tree.html took 0.0 seconds
21:50:44.952 7372   external/wpt/encrypted-media/drm-mp4-waiting-for-a-key.html took 0.0 seconds
21:50:44.952 7372   external/wpt/encrypted-media/drm-events.html took 0.0 seconds
21:50:44.952 7372   external/wpt/encrypted-media/drm-keystatuses.html took 0.0 seconds
21:50:44.952 7372   external/wpt/encrypted-media/drm-mp4-playback-temporary-multikey.html took 0.0 seconds
21:50:44.952 7372   external/wpt/encrypted-media/drm-mp4-playback-temporary-setMediaKeys-after-src.html took 0.0 seconds
21:50:44.952 7372 
21:50:44.952 7372 Tests that timed out or crashed:
21:50:44.952 7372   external/wpt/editing/run/multitest.html took 30.1 seconds
21:50:44.952 7372   virtual/mojo-loading/http/tests/inspector-protocol/network-fetch-content-with-error-status-code.html took 6.1 seconds
21:50:44.952 7372   external/wpt/workers/semantics/structured-clone/dedicated.html took 6.1 seconds
21:50:44.952 7372   external/wpt/content-security-policy/connect-src/connect-src-eventsource-blocked.sub.html took 6.1 seconds
21:50:44.952 7372   external/wpt/uievents/interface/dblclick-event-manual.htm took 6.1 seconds
21:50:44.952 7372   http/tests/permissions/chromium/test-request-multiple-worker.html took 6.1 seconds
21:50:44.952 7372   external/wpt/orientati=> Results: 6915/7014 tests passed (98.6%)

=> Tests to be fixed (1192):
     24 text-only failures       ( 2.0%)
     54 image-only failures      ( 4.5%)
      6 image and text failures  ( 0.5%)
     15 timeouts                 ( 1.3%)

=> Tests that will only be fixed if they crash (WONTFIX) (62):


Expected to timeout, but passed: (3)
  external/wpt/streams/piping/multiple-propagation.sharedworker.html
  external/wpt/streams/readable-streams/count-queuing-strategy-integration.sharedworker.html
  external/wpt/streams/readable-streams/pipe-through.sharedworker.html

Expected to fail, but passed: (1)
  fast/multicol/span/pseudo-before-after-in-content.html


------------------------------------------------------------------------------
on-event/t022-manual.html took 6.1 seconds
21:50:44.953 7372   external/wpt/service-workers/service-worker/clients-matchall-include-uncontrolled.https.html took 6.1 seconds
21:50:44.953 7372   external/wpt/mixed-content/blockable/no-opt-in/cross-origin-http/form-tag/top-level/no-redirect/no-opt-in-blocks.https.html took 6.1 seconds
21:50:44.953 7372   external/wpt/mixed-content/blockable/http-csp/same-host-http/form-tag/top-level/no-redirect/opt-in-blocks.https.html took 6.1 seconds
21:50:44.953 7372   external/wpt/mixed-content/blockable/no-opt-in/cross-origin-http/form-tag/top-level/keep-scheme-redirect/no-opt-in-blocks.https.html took 6.1 seconds
21:50:44.953 7372   external/wpt/html/browsers/history/the-location-interface/reload_post_1.html took 6.1 seconds
21:50:44.953 7372   virtual/mojo-loading/http/tests/security/xss-DENIED-cross-origin-stack-overflow.html took 6.0 seconds
21:50:44.953 7372   external/wpt/mixed-content/blockable/http-csp/cross-origin-http/form-tag/top-level/swap-scheme-redirect/opt-in-blocks.https.html took 6.0 seconds
21:50:44.953 7372   external/wpt/html/semantics/interactive-elements/context-menus/contextmenu-event-manual.htm took 6.0 seconds
21:50:44.953 7372 
21:50:44.963 7372 Time to process slowest subdirectories:
21:50:44.963 7372   external/wpt/XMLHttpRequest took 34.9 seconds to run 17 tests.
21:50:44.963 7372   external/wpt/css/CSS2/normal-flow took 17.0 seconds to run 70 tests.
21:50:44.963 7372   external/wpt/editing/run took 35.7 seconds to run 5 tests.
21:50:44.963 7372   locked_shard_1 took 119.9 seconds to run 376 tests.
21:50:44.963 7372   locked_shard_2 took 176.6 seconds to run 376 tests.
21:50:44.963 7372   media took 21.1 seconds to run 25 tests.
21:50:44.963 7372   virtual/disable-spinvalidation/paint/invalidation took 18.7 seconds to run 60 tests.
21:50:44.963 7372   virtual/gpu-rasterization/images took 29.3 seconds to run 21 tests.
21:50:44.963 7372   virtual/threaded/animations took 16.9 seconds to run 34 tests.
21:50:44.963 7372 
21:50:44.963 7372 
21:50:44.963 7372 5849 tests ran as expected (5756 passed, 93 didn't), 10 didn't:
21:50:44.963 7372 
21:50:44.995 7372 Generating dashboard...
21:50:45.000 7372 Dashboard generated.
21:50:45.000 7372 
21:50:45.000 7372 Testing completed, Exit status: 0
Failed to delete e:\b\s\w\ir (4 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 2 seconds.
Failed to delete e:\b\s\w\ir (4 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 4 seconds.
Failed to delete e:\b\s\w\ir. The following files remain:
- \\?\e:\b\s\w\ir\out\Release\content_shell.exe
- \\?\e:\b\s\w\ir\out\Release
- \\?\e:\b\s\w\ir\out
- \\?\e:\b\s\w\ir
Enumerating processes:
- pid 5724; Handles: 3; Exe: None; Cmd: "e:\b\s\w\ir\out\Release\content_shell.exe" --type=gpu-process --disable-gpu-rasterization --enable-logging --run-layout-test --enable-crash-reporter --crash-dumps-dir="e:\b\s\w\ir\out\Release\crash-dumps" --register-font-files="e:\b\s\w\ir\out\Release\/AHEM____.TTF" --use-gl=osmesa --supports-dual-gpus=false --gpu-vendor-id=0x0000 --gpu-device-id=0x0000 --gpu-driver-vendor --gpu-driver-version=9.0.3 --gpu-driver-date --run-layout-test --enable-crash-reporter --crash-dumps-dir="e:\b\s\w\ir\out\Release\crash-dumps" --register-font-files="e:\b\s\w\ir\out\Release\/AHEM____.TTF" --enable-logging --service-request-channel-token=328ECF4896E35D5F946F41ABA5805E55 --mojo-platform-channel-handle=1980 --ignored=" --type=renderer " /prefetch:2
Terminating 1 processes.
- 5724 killed
Enumerating processes:
Failed to delete the run directory, forcibly failing
the task because of it. No zombie process can outlive a
successful task run and still be marked as successful.
Fix your stuff.

--------------


This appears to occasionally happen even when the run was a success;
==============================================================================
Expected to fail, but passed: (5)
  external/wpt/css/css-display-3/display-contents-computed-style.html
  external/wpt/css/css-text-3/i18n/css3-text-line-break-opclns-005.html
  external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/writing-modes-3/text-combine-upright-compression-006a.html
  virtual/layout_ng/external/wpt/css/CSS2/floats-clear/clear-applies-to-012.xht
  virtual/layout_ng/external/wpt/css/CSS2/normal-flow/block-formatting-contexts-015.xht


------------------------------------------------------------------------------
econds to run 97 tests.
11:14:56.163 1768   locked_shard_1 took 152.6 seconds to run 381 tests.
11:14:56.163 1768   locked_shard_2 took 149.9 seconds to run 441 tests.
11:14:56.163 1768   virtual/disable-spinvalidation/paint/invalidation took 20.1 seconds to run 70 tests.
11:14:56.163 1768   virtual/exotic-color-space/images took 17.0 seconds to run 25 tests.
11:14:56.163 1768   virtual/layout_ng/external/wpt/css/CSS2/normal-flow took 20.0 seconds to run 71 tests.
11:14:56.163 1768   virtual/layout_ng/external/wpt/css/CSS2/positioning took 15.9 seconds to run 45 tests.
11:14:56.163 1768 
11:14:56.163 1768 
11:14:56.163 1768 5874 tests ran as expected (5724 passed, 150 didn't), 20 didn't:
11:14:56.163 1768 
11:14:56.193 1768 Generating dashboard...
11:14:56.199 1768 Dashboard generated.
11:14:56.199 1768 
11:14:56.200 1768 Testing completed, Exit status: 0
Failed to delete e:\b\s\w\ir (4 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 2 seconds.
Failed to delete e:\b\s\w\ir (4 files remaining).
  Maybe the test has a subprocess outliving it.
  Sleeping 4 seconds.
Failed to delete e:\b\s\w\ir. The following files remain:
- \\?\e:\b\s\w\ir\out\Release\content_shell.exe
- \\?\e:\b\s\w\ir\out\Release
- \\?\e:\b\s\w\ir\out
- \\?\e:\b\s\w\ir
Enumerating processes:
- pid 3256; Handles: 3; Exe: None; Cmd: "e:\b\s\w\ir\out\Release\content_shell.exe" --type=gpu-process --disable-gpu-rasterization --enable-color-correct-rendering --enable-logging --run-layout-test --enable-crash-reporter --crash-dumps-dir="e:\b\s\w\ir\out\Release\crash-dumps" --register-font-files="e:\b\s\w\ir\out\Release\/AHEM____.TTF" --use-gl=osmesa --supports-dual-gpus=false --gpu-vendor-id=0x0000 --gpu-device-id=0x0000 --gpu-driver-vendor --gpu-driver-version=9.0.3 --gpu-driver-date --run-layout-test --enable-crash-reporter --crash-dumps-dir="e:\b\s\w\ir\out\Release\crash-dumps" --register-font-files="e:\b\s\w\ir\out\Release\/AHEM____.TTF" --enable-logging --service-request-channel-token=1CC2F955698CB5DF581E69E0613A84F4 --mojo-platform-channel-handle=2308 --ignored=" --type=renderer " /prefetch:2
Terminating 1 processes.
- 3256 killed
Enumerating processes:
*** Swarming tried multiple times to delete the run directory and failed ***
*** Hard failing the task ***

Swarming detected that your testing script ran an executable, which may have
started a child executable, and the main script returned early, leaving the
children executables playing around unguided.

You don't want to leave children processes outliving the task on the Swarming
bot, do you? The Swarming bot doesn't.

How to fix?
- For any process that starts children processes, make sure all children
  processes terminated properly before each parent process exits. This is
  especially important in very deep process trees.
  - This must be done properly both in normal successful task and in case of
    task failure. Cleanup is very important.
- The Swarming bot sends a SIGTERM in case of timeout.
  - You have 30.0 seconds to comply after the signal was sent to the process
    before the process is forcibly killed.
- To achieve not leaking children processes in case of signals on timeout, you
  MUST handle signals in each executable / python script and propagate them to
  children processes.
  - When your test script (python or binary) receives a signal like SIGTERM or
    CTRL_BREAK_EVENT on Windows), send it to all children processes and wait for
    them to terminate before quitting.

See
https://github.com/luci/luci-py/blob/master/appengine/swarming/doc/Bot.md#graceful-termination-aka-the-sigterm-and-sigkill-dance
for more information.

*** May the SIGKILL force be with you ***
==============================================================================
Owner: tansell@chromium.org
Status: Assigned (was: Untriaged)
maruel@ and I spent a bit of time looking into this last week. We agreed that we can probably change swarming as needed to make sure we're not blocking switching Windows to swarming.

However, we also couldn't find any recent examples where it looked like this was still happening. Given that the last update here is from almost two months ago, maybe it isn't?
 Issue 694326  has been merged into this issue.
Project Member

Comment 9 by bugdroid1@chromium.org, Aug 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/aa374e133dee3b25a8f1df7b3a43dcb27e328878

commit aa374e133dee3b25a8f1df7b3a43dcb27e328878
Author: Tim 'mithro' Ansell <tansell@chromium.org>
Date: Tue Aug 22 09:52:25 2017

buildbot: Deploying LayoutTests on Swarming on Webkit FYI Windows bots.

We are deploying on the FYI bots first (rather than on the trybots
first) to try and get more data on the layout tests leaving processes
running after finishing (see  https://crbug.com/717348 ).

BUG= 717348 , 524758 
TBR=qyearsley@chromium.org,dpranke@chromium.org
CC=​​​mcgreevy@chromium.org,mikelawther@chromium.org,maruel@chromium.org

Change-Id: I74b1085f12160e78272ef80cd6adceb66382a2ad
Reviewed-on: https://chromium-review.googlesource.com/625539
Commit-Queue: Tim 'mithro' Ansell <tansell@chromium.org>
Reviewed-by: Tim 'mithro' Ansell <tansell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#496264}
[modify] https://crrev.com/aa374e133dee3b25a8f1df7b3a43dcb27e328878/testing/buildbot/chromium.webkit.json

Project Member

Comment 10 by bugdroid1@chromium.org, Aug 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/be2db396fe48b8990e08032ff7cd13a35e3fd520

commit be2db396fe48b8990e08032ff7cd13a35e3fd520
Author: Tim 'mithro' Ansell <tansell@chromium.org>
Date: Wed Aug 23 03:10:17 2017

chromium_tests: Remove locally run layout tests on Windows.

src side change was landed at https://crrev.com/625539

We are deploying on the FYI bots first (rather than on the trybots
first) to try and get more data on the layout tests leaving processes
running after finishing (see  https://crbug.com/717348 ).

Note: You must revert the other CL as well as this one.

BUG= 717348 , 524758 
TBR=qyearsley@chromium.org,dpranke@chromium.org
CC=mcgreevy@chromium.org,mikelawther@chromium.org,maruel@chromium.org,hbos@chromium.org

Change-Id: Ic9a0f47ad2f426b8ff9d2e3aee3251b4f9f9a298
Reviewed-on: https://chromium-review.googlesource.com/625860
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Tim 'mithro' Ansell <tansell@chromium.org>
Commit-Queue: Tim 'mithro' Ansell <tansell@chromium.org>

[delete] https://crrev.com/6c6d14b2c9a9224da7822f10fecbba913b665d00/scripts/slave/recipes/chromium_trybot.expected/non_cq_blink_tryjob.json
[modify] https://crrev.com/be2db396fe48b8990e08032ff7cd13a35e3fd520/scripts/slave/recipe_modules/chromium_tests/chromium_webkit.py
[modify] https://crrev.com/be2db396fe48b8990e08032ff7cd13a35e3fd520/scripts/slave/recipes/chromium.expected/chromium_webkit_parent_revision_webkit.json
[modify] https://crrev.com/be2db396fe48b8990e08032ff7cd13a35e3fd520/scripts/slave/recipes/chromium_trybot.py
[modify] https://crrev.com/be2db396fe48b8990e08032ff7cd13a35e3fd520/scripts/slave/recipes/chromium.expected/chromium_webkit_parent_revision_chromium.json

Project Member

Comment 11 by bugdroid1@chromium.org, Aug 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fb027fe65919314bbbdfb18acb35b8e41f0cf6d9

commit fb027fe65919314bbbdfb18acb35b8e41f0cf6d9
Author: Tim 'mithro' Ansell <tansell@chromium.org>
Date: Wed Aug 23 05:48:37 2017

buildbot: Adding --debug to WebKit Win7 (dbg) bot.

Should fix;
------------------------------------
22:11:25.948 2648 Found 13285 tests (total 66425); running 11471, skipping 1814.
22:11:25.948 2648 Checking build ...
22:11:25.948 2648 Unable to find test driver
22:11:25.949 2648     at e:\b\swarm_slave\w\ir\out\Release\content_shell.exe
22:11:25.949 2648 Unable to find content_shell_crash_service.exe
22:11:25.949 2648     at e:\b\swarm_slave\w\ir\out\Release\content_shell_crash_service.exe
22:11:25.949 2648     Could not find crash service, unexpected crashes won't be symbolized.
22:11:25.949 2648     Did you build the target blink_tests?
22:11:25.949 2648
22:11:25.949 2648 For complete Windows build requirements, please see:
22:11:25.949 2648
22:11:25.949 2648     https://chromium.googlesource.com/chromium/src/+/master/docs/windows_build_instructions.md
22:11:25.949 2648 Build check failed
22:11:25.952 2648
22:11:25.952 2648 Testing completed. Exit status: 255
------------------------------------

BUG= 717348 , 524758 ,757955
TBR=tapted@google.com
CC=​qyearsley@chromium.org,dpranke@chromium.org
NOTRY=true

Change-Id: I27047cca345ec367d70a7bfa486f459f596a2f90
Reviewed-on: https://chromium-review.googlesource.com/627551
Reviewed-by: Tim 'mithro' Ansell <tansell@chromium.org>
Commit-Queue: Tim 'mithro' Ansell <tansell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#496592}
[modify] https://crrev.com/fb027fe65919314bbbdfb18acb35b8e41f0cf6d9/testing/buildbot/chromium.webkit.json

Project Member

Comment 12 by bugdroid1@chromium.org, Aug 24 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8c8f1fa98b08869a848fe9a8565c51eb1a86cb93

commit 8c8f1fa98b08869a848fe9a8565c51eb1a86cb93
Author: Tim 'mithro' Ansell <tansell@chromium.org>
Date: Thu Aug 24 06:03:29 2017

LayoutTest: Increase timeout on Win 7 debug.

Increase timeout on debug bots on Windows because it's a bit slower.
Increase the number of shards so it doesn't increase the cycle time.

BUG= 717348 , 524758 
TBR=yoichio@chromium.org,vitaliii@chromium.org,tschumann@chromium.org,dschuyler@chromium.org
CC=​qyearsley@chromium.org,dpranke@chromium.org,tapted@chromium.org
NOTRY=true

Change-Id: Ia1d104b232b07a9126fb063ad89bbb39607fdaff
Reviewed-on: https://chromium-review.googlesource.com/631458
Commit-Queue: Tim 'mithro' Ansell <tansell@chromium.org>
Reviewed-by: Tim 'mithro' Ansell <tansell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#496964}
[modify] https://crrev.com/8c8f1fa98b08869a848fe9a8565c51eb1a86cb93/testing/buildbot/chromium.webkit.json

Status: Archived (was: Assigned)
This problem has gone away. If it comes back we will reopen the bug.

Sign in to add a comment