browser_tests failing on chromium.memory/Linux ASan LSan Tests (1) |
||||||||
Issue descriptionbrowser_tests failing on chromium.memory/Linux ASan LSan Tests (1) Builders failed on: - Linux ASan LSan Tests (1): https://build.chromium.org/p/chromium.memory/builders/Linux%20ASan%20LSan%20Tests%20%281%29 First failing build: https://build.chromium.org/p/chromium.memory/builders/Linux%20ASan%20LSan%20Tests%20%281%29/builds/37629 Failing test: SSLUITest.TestWSSInvalidCertAndClose Error message: [ RUN ] SSLUITest.TestWSSInvalidCertAndClose Xlib: extension "RANDR" missing on display ":99". [8463:8463:0730/205311.087448:WARNING:password_store_factory.cc(255)] Using basic (unencrypted) store for password storage. See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_password_storage.md for more information about password storage options. Trying to start websocket server on wss://127.0.0.1:0... WebSocket server started on wss://127.0.0.1:52726... sending server_data: {"host": "127.0.0.1", "port": 52726} (36 bytes) [8463:8501:0730/205314.282347:WARNING:simple_synchronous_entry.cc(1101)] Could not open platform files for entry. [8463:8471:0730/205314.730694:ERROR:cert_verify_proc_nss.cc(902)] CERT_PKIXVerifyCert for 127.0.0.1 failed err=-8181 [8463:8531:0730/205315.273684:WARNING:embedded_test_server.cc(219)] Request not handled. [snip] Returning 404: /favicon.ico [8463:8463:0730/205315.617950:INFO:CONSOLE(15)] "WebSocket connection to 'wss://127.0.0.1:52726/' failed: Error in connection establishment: net::ERR_INSECURE_RESPONSE", source: http://127.0.0.1:60677/ssl/wss_close_slave.html?52726 (15) [8463:8463:0730/205315.620308:INFO:CONSOLE(15)] "WebSocket connection to 'wss://127.0.0.1:52726/' failed: Error in connection establishment: net::ERR_INSECURE_RESPONSE", source: http://127.0.0.1:60677/ssl/wss_close_slave.html?52726 (15) [8463:8463:0730/205315.622153:INFO:CONSOLE(16)] "WebSocket connection to 'wss://127.0.0.1:52726/' failed: Error in connection establishment: net::ERR_INSECURE_RESPONSE", source: http://127.0.0.1:60677/ssl/wss_close.html?52726 (16) [8463:8463:0730/205315.663177:INFO:CONSOLE(15)] "WebSocket connection to 'wss://127.0.0.1:52726/' failed: Error in connection establishment: net::ERR_INSECURE_RESPONSE", source: [snip] http://127.0.0.1:60677/ssl/wss_close_slave.html?52726 (15) ----------------------------------------------------- Suppressions used: count bytes template 2 288 libfontconfig ----------------------------------------------------- [8463:8463:0730/205325.085177:ERROR:event_page_request_manager.cc(147)] An error encountered while waking the event page. [8463:8463:0730/205325.085343:ERROR:event_page_request_manager.cc(89)] Draining request queue. (queue-length=2) ==8603:21195==ERROR: AddressSanitizer failed to allocate 0xb09000 (11571200) bytes of FakeStack (error code: 12) ERROR: Failed to mmap
,
Jul 31 2017
That change only changed TestExpectations, so it can't be that. The test has been flaky for some time: https://build.chromium.org/p/chromium.memory/builders/Linux%20ASan%20LSan%20Tests%20%281%29?numbuilds=200 https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=SSLUITest.TestWSSInvalidCertAndClose So it's probably something around July 28.
,
Jul 31 2017
Hmm, perhaps it's a memory issue? "AddressSanitizer failed to allocate 0xb09000 (11571200) bytes of FakeStack (error code: 12)" There's a memory-infra CL in range: https://chromium-review.googlesource.com/c/565406/ hjd@, could you PTAL?
,
Aug 3 2017
Hmm, it would be exciting if that CL caused this flake. The error seems like ASAN is OOMing (similar to an error error in this thread [1]). My CL does allocate 500kb more memory but it shouldn't be ran in the course of SSLUITest.TestWSSInvalidCertAndClose and so shouldn't affect the amount of memory we use for this test. It is possible we happened to recently cross a threshold of memory use where this starts happening due to other CLs though. +glider Does that diagnosis seem correct? [1]: https://github.com/google/sanitizers/issues/697 ^ This thread includes the great comment: "Note that ASan's allocator may mmap N times more than your app requested."
,
Aug 3 2017
> It is possible we happened to recently cross a threshold of memory use where this starts happening due to other CLs though. Yes, this sounds likely. If the bug reproduces locally, running it with ASAN_OPTIONS=verbosity=2 may shed some light onto what's going on. Kostya, am I understanding right that FakeStack::Destroy never unmaps the memory, so in the case we create and destroy many threads (which might be happening in the tests) we may consume all the available memory?
,
Aug 3 2017
> FakeStack::Destroy never unmaps the memory
Why do you think so?
void FakeStack::Destroy(int tid) {
...
FlushUnneededASanShadowMemory(reinterpret_cast<uptr>(this), size);
UnmapOrDie(this, size);
}
,
Aug 7 2017
> Why do you think so? Sorry, I've misinterpreted the code. Of course it does, and there are no leaks.
,
Aug 10 2017
,
Aug 1
,
Jan 11
Setting defect without priority to Pri-2. |
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by meade@chromium.org
, Jul 31 2017Status: Assigned (was: Available)