unit_tests failing on chromium.win/Win10 Tests x64 |
|||
Issue descriptionFiled by sheriff-o-matic@appspot.gserviceaccount.com on behalf of vitaliii@google.com unit_tests failing on chromium.win/Win10 Tests x64 Builders failed on: - Win10 Tests x64: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20Tests%20x64
,
May 9 2018
Presumably relevant bit of log ============== SUCCESS: all tests passed. Tests took 485 seconds. Failed to delete C:\b\s\w\ir (20 files remaining). Maybe the test has a subprocess outliving it. Sleeping 2 seconds. Failed to delete C:\b\s\w\ir (20 files remaining). Maybe the test has a subprocess outliving it. Sleeping 4 seconds. Failed to delete C:\b\s\w\ir. The following files remain: - \\?\C:\b\s\w\ir - \\?\C:\b\s\w\ir\out - \\?\C:\b\s\w\ir\out\Release_x64 - \\?\C:\b\s\w\ir\out\Release_x64\chrome.dll - \\?\C:\b\s\w\ir\out\Release_x64\chrome.exe - \\?\C:\b\s\w\ir\out\Release_x64\chrome_100_percent.pak - \\?\C:\b\s\w\ir\out\Release_x64\chrome_200_percent.pak - \\?\C:\b\s\w\ir\out\Release_x64\chrome_child.dll - \\?\C:\b\s\w\ir\out\Release_x64\chrome_elf.dll - \\?\C:\b\s\w\ir\out\Release_x64\d3dcompiler_47.dll - \\?\C:\b\s\w\ir\out\Release_x64\icudtl.dat - \\?\C:\b\s\w\ir\out\Release_x64\initialexe - \\?\C:\b\s\w\ir\out\Release_x64\initialexe\chrome.exe - \\?\C:\b\s\w\ir\out\Release_x64\libEGL.dll - \\?\C:\b\s\w\ir\out\Release_x64\libGLESv2.dll - \\?\C:\b\s\w\ir\out\Release_x64\locales - \\?\C:\b\s\w\ir\out\Release_x64\locales\en-US.pak - \\?\C:\b\s\w\ir\out\Release_x64\natives_blob.bin - \\?\C:\b\s\w\ir\out\Release_x64\resources.pak - \\?\C:\b\s\w\ir\out\Release_x64\v8_context_snapshot.bin Enumerating processes: - pid 688; Handles: 137; Exe: C:\b\s\w\ir\out\Release_x64\chrome.exe; Cmd: C:\b\s\w\ir\out\Release_x64\chrome.exe --type=crashpad-handler "--user-data-dir=C:\Users\chrome-bot\AppData\Local\Chromium\User Data" /prefetch:7 --monitor-self-annotation=ptype=crashpad-handler "--database=C:\Users\chrome-bot\AppData\Local\Chromium\User Data\Crashpad" "--metrics-dir=C:\Users\chrome-bot\AppData\Local\Chromium\User Data" --annotation=plat=Win64 --annotation=prod=Chromium --annotation=ver=68.0.3425.0-devel --initial-client-data=0x190,0x194,0x198,0x18c,0x19c,0x7ffd137b6250,0x7ffd137b6260,0x7ffd137b6270 - pid 1944; Handles: 420; Exe: C:\b\s\w\ir\out\Release_x64\chrome.exe; Cmd: "C:\b\s\w\ir\out\Release_x64\chrome.exe" --type=gpu-process --field-trial-handle=1632,8640685463012438446,17892973344504919221,131072 --noerrdialogs --start-stack-profiler --gpu-preferences=KAAAAAAAAACAAwBgAQAAAAAAAAAAAGAAEAAAAAAAAAAAAAAAAAAAACgAAAAEAAAAIAAAAAAAAAAoAAAAAAAAADAAAAAAAAAAOAAAAAAAAAAQAAAAAAAAAAAAAAAKAAAAEAAAAAAAAAAAAAAACwAAABAAAAAAAAAAAQAAAAoAAAAQAAAAAAAAAAEAAAALAAAA --noerrdialogs --start-stack-profiler --service-request-channel-token=545469409D897F62EB81F47A43B5465B --mojo-platform-channel-handle=1652 --ignored=" --type=renderer " /prefetch:2 - pid 7748; Handles: 244; Exe: C:\b\s\w\ir\out\Release_x64\chrome.exe; Cmd: "C:\b\s\w\ir\out\Release_x64\chrome.exe" --type=renderer --field-trial-handle=1632,8640685463012438446,17892973344504919221,131072 --service-pipe-token=782ADE8CD2A0094D6A4DFB8B6F6538CC --lang=en-US --noerrdialogs --enable-offline-auto-reload --enable-offline-auto-reload-visible-only --blink-settings=cssExternalScannerNoPreload=false,cssExternalScannerPreload=true --device-scale-factor=1 --num-raster-threads=4 --enable-main-frame-before-activation --service-request-channel-token=782ADE8CD2A0094D6A4DFB8B6F6538CC --renderer-client-id=4 --mojo-platform-channel-handle=2428 /prefetch:1 - pid 7924; Handles: 969; Exe: C:\b\s\w\ir\out\Release_x64\chrome.exe; Cmd: "C:\b\s\w\ir\out\Release_x64\chrome.exe" --notification-launch-id=0|0|Default|0|https://example.com/|notification_id Terminating 4 processes: - 688 killed - 1944 killed - 7748 killed - 7924 killed *** Swarming tried multiple times to delete the run directory and failed *** *** Hard failing the task *** Swarming detected that your testing script ran an executable, which may have started a child executable, and the main script returned early, leaving the children executables playing around unguided. You don't want to leave children processes outliving the task on the Swarming bot, do you? The Swarming bot doesn't. How to fix? - For any process that starts children processes, make sure all children processes terminated properly before each parent process exits. This is especially important in very deep process trees. - This must be done properly both in normal successful task and in case of task failure. Cleanup is very important. - The Swarming bot sends a SIGTERM in case of timeout. - You have 30.0 seconds to comply after the signal was sent to the process before the process is forcibly killed. - To achieve not leaking children processes in case of signals on timeout, you MUST handle signals in each executable / python script and propagate them to children processes. - When your test script (python or binary) receives a signal like SIGTERM or CTRL_BREAK_EVENT on Windows), send it to all children processes and wait for them to terminate before quitting. See https://github.com/luci/luci-py/blob/master/appengine/swarming/doc/Bot.md#graceful-termination-aka-the-sigterm-and-sigkill-dance for more information. *** May the SIGKILL force be with you *** +--------------------------------------------------------------------------------+ | End of shard 0 | | Pending: 0.4s Duration: 487.0s Bot: win10-0af2fd95-us-west1-b-959b Exit: 1 | +--------------------------------------------------------------------------------+ ==============
,
May 9 2018
All of them seem to fail in the same way.
,
May 9 2018
There are also tests which fail first time, but pass on the retry.
e.g.
==========
[ RUN ] ChromeDownloadManagerDelegateTest.LastSavePath
unknown file: error:
Unexpected mock function call - returning directly.
Function call: RequestConfirmation(000001B0395CEC40, @000001B03718B3E0 C:\Users\chrome-bot\Downloads\foo.txt, 4-byte object <02-00 00-00>, @000000A417FFE370 8-byte object <40-B8 5C-39 B0-01 00-00>)
Google Mock tried the following 1 expectation, but it didn't match:
../../chrome/browser/download/chrome_download_manager_delegate_unittest.cc(432): EXPECT_CALL(*delegate(), RequestConfirmation(save_as_download.get(), expected_prompt_path, _, _))...
Expected arg #1: is equal to C:\Users\CHROME~1\AppData\Local\Temp\scoped_dir8996_14319\foo.txt
Actual: C:\Users\chrome-bot\Downloads\foo.txt
Expected: to be called once
Actual: never called - unsatisfied and active
Stack trace:
Backtrace:
testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop [0x00007FF7692DAA87+87]
testing::internal::AssertHelper::operator= [0x00007FF7692DA60E+78]
testing::internal::GoogleTestFailureReporter::ReportFailure [0x00007FF7692CBEED+109]
testing::internal::UntypedFunctionMockerBase::UntypedInvokeWith [0x00007FF7692CE84F+1023]
testing::internal::FunctionMockerBase<void __cdecl(base::OnceCallback<void __cdecl(std::basic_string<char,std::char_traits<char>,std::allocator<char> > const & __ptr64)> & __ptr64)>::InvokeWith [0x00007FF768867DBA+10]
testing::internal::FunctionMockerBase<enum safe_browsing::AsyncMatch __cdecl(GURL const & __ptr64,safe_browsing::SafeBrowsingDatabaseManager::Client * __ptr64)>::InvokeWith [0x00007FF7682F1AF2+786]
DownloadTargetDeterminer::DoRequestConfirmation [0x00007FF76B37C639+217]
DownloadTargetDeterminer::DoLoop [0x00007FF76B37BE23+83]
base::internal::Invoker<base::internal::BindState<base::RepeatingCallback<void __cdecl(int,std::basic_string<char,std::char_traits<char>,std::allocator<char> > const & __ptr64)>,int,std::basic_string<char,std::char_traits<char>,std::allocator<char> > >,vo [0x00007FF76C2DAC29+57]
base::debug::TaskAnnotator::RunTask [0x00007FF76B1BEC25+309]
base::MessageLoop::RunTask [0x00007FF76B1589AC+572]
base::MessageLoop::DoWork [0x00007FF76B158F08+408]
base::MessagePumpForUI::DoRunLoop [0x00007FF76B1A71F9+169]
base::MessagePumpWin::Run [0x00007FF76B1A6D58+104]
base::RunLoop::Run [0x00007FF76B13BAD1+49]
testing::Matcher<base::FilePath const & __ptr64>::Matcher<base::FilePath const & __ptr64> [0x00007FF7682EC800+608]
ChromeDownloadManagerDelegateTest_LastSavePath_Test::TestBody [0x00007FF7682EAC8F+2095]
[8839/8842] ChromeDownloadManagerDelegateTest.LastSavePath (TIMED OUT)
======
This seems to happen pretty consistently.
,
May 9 2018
====== [3340:7664:0508/153715.297:630171:ERROR:unit_test_launcher.cc(338)] no test result for SyncInternalsMessageHandlerTest.WriteUserEventBlank [3340:7664:0508/153715.297:630171:ERROR:unit_test_launcher.cc(338)] no test result for SyncInternalsMessageHandlerTest.WriteUserEventZero [3340:7664:0508/153715.297:630171:ERROR:unit_test_launcher.cc(338)] no test result for WebDialogWebContentsDelegateTest.DoNothingMethodsTest [3340:7664:0508/153715.297:630171:ERROR:unit_test_launcher.cc(338)] no test result for WebDialogWebContentsDelegateTest.OpenURLFromTabTest [3340:7664:0508/153715.297:630171:ERROR:unit_test_launcher.cc(338)] no test result for WebDialogWebContentsDelegateTest.AddNewContentsForegroundTabTest [3340:7664:0508/153715.297:630171:ERROR:unit_test_launcher.cc(338)] no test result for WebDialogWebContentsDelegateTest.DetachTest [3340:7664:0508/153715.297:630171:ERROR:unit_test_launcher.cc(338)] no test result for WindowSizerTestCommon.PersistedWindowOffscreenWithNonAggressiveRepositioning [3340:7664:0508/153715.297:630171:ERROR:unit_test_launcher.cc(338)] no test result for WindowSizerTestCommon.AdjustFitSize [3340:7664:0508/153715.297:630171:ERROR:unit_test_launcher.cc(338)] no test result for WindowSizerTest.DefaultSizeCase [3801/8842] SyncInternalsMessageHandlerTest.WriteUserEventBadParse (18 ms) [ RUN ] SyncInternalsMessageHandlerTest.WriteUserEventBlank [2560:6604:0508/153714.172:629046:FATAL:sequenced_task_runner_handle.cc(33)] Check failed: ThreadTaskRunnerHandle::IsSet(). Error: This caller requires a sequenced context (i.e. the current task needs to run from a SequencedTaskRunner). Backtrace: base::debug::StackTrace::StackTrace [0x00007FF6B180F424+36] logging::LogMessage::~LogMessage [0x00007FF6B17915A5+101] base::SequencedTaskRunnerHandle::Get [0x00007FF6B17B46CB+219] mojo::internal::GetTaskRunnerToUseFromUserProvidedTaskRunner [0x00007FF6B11B7CC0+48] mojo::internal::InterfacePtrStateBase::Bind [0x00007FF6B11AE458+104] mojo::internal::InterfacePtrState<bluetooth::mojom::Adapter>::Bind [0x00007FF6B44A2405+69] mojo::InterfacePtr<bluetooth::mojom::Adapter>::Bind [0x00007FF6B2395CA9+89] mojo::MakeRequest<bluetooth::mojom::Adapter> [0x00007FF6B0F5F97E+110] resource_coordinator::PageSignalReceiver::AddObserver [0x00007FF6B2F06E26+134] resource_coordinator::TabManager::ResourceCoordinatorSignalObserver::ResourceCoordinatorSignalObserver [0x00007FF6B2F351D7+39] resource_coordinator::TabManager::TabManager [0x00007FF6B2F30D25+437] TestingBrowserProcess::GetTabManager [0x00007FF6B1134B5E+46] ChromeUnitTestSuite::`scalar deleting destructor' [0x00007FF6B11365AB+411] testing::internal::TestEventRepeater::OnTestStart [0x00007FF6AF93035C+44] testing::TestInfo::Run [0x00007FF6AF92E4BE+94] testing::TestCase::Run [0x00007FF6AF92E9E2+258] testing::internal::UnitTestImpl::RunAllTests [0x00007FF6AF935B3E+638] testing::UnitTest::Run [0x00007FF6AF9357B5+165] base::TestSuite::Run [0x00007FF6B113F966+118] base::LaunchUnitTests [0x00007FF6B114094A+554] base::LaunchUnitTests [0x00007FF6B11407C8+168] main [0x00007FF6B40A3471+269] __scrt_common_main_seh [0x00007FF6B4B72E39+285] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:283) BaseThreadInitThunk [0x00007FFBA6D72774+20] RtlUserThreadStart [0x00007FFBA7190D51+33] [3802/8842] SyncInternalsMessageHandlerTest.WriteUserEventBlank (UNKNOWN) ======
,
May 9 2018
Suspects: https://chromium-review.googlesource.com/c/chromium/src/+/1049984 I am not sure what it is doing, but "SEE_MASK_NOCLOSEPROCESS" seems related :) https://chromium-review.googlesource.com/c/chromium/src/+/1038226 ICU rollout, but I have no idea what ICU is.
,
May 9 2018
The failures are a result of a launch of chrome.exe with --notification-launch-id, which certainly is related to native notifications. I think it's due to a handle leak. The revert in r557125 should clear it up.
,
May 9 2018
,
May 9 2018
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20Tests%20x64/23296 is green. |
|||
►
Sign in to add a comment |
|||
Comment 1 by vitaliii@chromium.org
, May 9 2018Owner: vitaliii@chromium.org
Status: Started (was: Available)