Crash on Stable 8530.96.0 => shill::HTTPRequest::ReadFromServer-87be9517 |
|||||||||||||||
Issue descriptionReport IDe669f59b00000000 Product, version -> ChromeOS, 8530.96.0 Magic Signature__memmove_ssse3edit bugs&comments Stable Signatureshill::HTTPRequest::ReadFromServer-87be9517edit bugs&comments Report TimeTue, 11 Oct 2016 16:41:12 GMT Client IDB42237E76EC94BC187FF8043EAA4093B Thread 0 CRASHED [SIGSEGV @ 0x00000000 ] MAGIC SIGNATURE THREAD 0x00007f1637368208 (libc-2.19.so + 0x00143208 ) __memmove_ssse3 0x00007f16384cd7a3 (shill -stl_algobase.h:378 ) shill::HTTPRequest::ReadFromServer 0x00007f16384ce251 (shill -bind_internal.h:186 ) base::internal::Invoker<base::IndexSequence<0ul>, base::internal::BindState<base::internal::RunnableAdapter<void (shill::HTTPRequest::*)(shill::InputData*)>, void(shill::HTTPRequest*, shill::InputData*), base::WeakPtr<shill::HTTPRequest> >, base::internal::InvokeHelper<true, void, base::internal::RunnableAdapter<void (shill::HTTPRequest::*)(shill::InputData*)> >, void(shill::InputData*)>::Run 0x00007f16381a77ea (libshill-net-395517.so -callback.h:397 ) shill::IOInputHandler::OnFileCanReadWithoutBlocking 0x00007f1637f6201d (libbase-core-395517.so -message_pump_libevent.cc:346 ) base::MessagePumpLibevent::OnLibeventNotification 0x00007f1636cef0b3 (libevent_core-2.0.so.5.1.9 -event.c:1368 ) event_base_loop 0x00007f1637f61d30 (libbase-core-395517.so -message_pump_libevent.cc:256 ) base::MessagePumpLibevent::Run 0x00007f1637f84fa7 (libbase-core-395517.so -run_loop.cc:35 ) base::RunLoop::Run 0x00007f16380f9145 (libbrillo-core-395517.so -base_message_loop.cc:212 ) brillo::BaseMessageLoop::Run 0x00007f16380c083f (libbrillo-core-395517.so -daemon.cc:29 ) brillo::Daemon::Run 0x00007f16382550f2 (shill -shill_main.cc:253 ) main 0x00007f1637244fb5 (libc-2.19.so -libc-start.c:292 ) __libc_start_main 0x00007f1638255767 (shill + 0x00043767 ) _start 0x00007ffc0f9e8517 Sameer, there are close to 3633 reports for this shill crash on 8530.96.0. Who should take a look at this one? https://crash.corp.google.com/browse?q=product.name%3D%27ChromeOS%27%20AND%20product.version%3D%278530.96.0%27%20AND%20stable_signature%3D%27shill%3A%3AHTTPRequest%3A%3AReadFromServer-87be9517%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports
,
Oct 12 2016
,
Oct 12 2016
Same issue as: https://code.google.com/p/chrome-os-partner/issues/detail?id=52689 Tracking there as that has previous context.
,
Oct 14 2016
I have a fix here: https://chromium-review.googlesource.com/#/c/399398/ Luigi's comment here provides good context: https://code.google.com/p/chrome-os-partner/issues/detail?id=52689#c24 Requesting merge for both 54 and 55 since this bug now has 21k+ crashes in the field.
,
Oct 15 2016
[Automated comment] Less than 2 weeks to go before stable on M54, manual review required.
,
Oct 15 2016
Your change meets the bar and is auto-approved for M55 (branch: 2883)
,
Oct 15 2016
[Automated comment] Less than 2 weeks to go before stable on M54, manual review required.
,
Oct 17 2016
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/connectivity/shill/+/2213b35c05d12a7b90eb6880cc9748d3b410b79a commit 2213b35c05d12a7b90eb6880cc9748d3b410b79a Author: Kirtika Ruchandani <kirtika@google.com> Date: Fri Oct 14 19:49:19 2016 HTTPRequest: Be resilient to bad data in input cb IOInputHandler's implementation calls the input callback if InputData has length < 0 (read failed). So, make the HTTPRequest terminate early if data->len < 0. This is a stop-gap fix for M54, until we fix IOInputHandler's behavior. Add a unit-test to ensure we don't crash on bad data. BUG=654807 TEST=shill unit-tests Change-Id: I98d38a5a359f3a81534a677762b17acf1d8ad35b Reviewed-on: https://chromium-review.googlesource.com/399398 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Kevin Cernekee <cernekee@chromium.org> [modify] https://crrev.com/2213b35c05d12a7b90eb6880cc9748d3b410b79a/http_request.cc [modify] https://crrev.com/2213b35c05d12a7b90eb6880cc9748d3b410b79a/http_request_unittest.cc
,
Oct 17 2016
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/connectivity/shill/+/65545b09cc6660830019f77fe5709a81f355487f commit 65545b09cc6660830019f77fe5709a81f355487f Author: Kirtika Ruchandani <kirtika@google.com> Date: Fri Oct 14 19:49:19 2016 HTTPRequest: Be resilient to bad data in input cb IOInputHandler's implementation calls the input callback if InputData has length < 0 (read failed). So, make the HTTPRequest terminate early if data->len < 0. This is a stop-gap fix for M54, until we fix IOInputHandler's behavior. Add a unit-test to ensure we don't crash on bad data. BUG=654807 TEST=shill unit-tests Change-Id: I98d38a5a359f3a81534a677762b17acf1d8ad35b [modify] https://crrev.com/65545b09cc6660830019f77fe5709a81f355487f/http_request.cc [modify] https://crrev.com/65545b09cc6660830019f77fe5709a81f355487f/http_request_unittest.cc
,
Oct 17 2016
Approved for M54 Kirtika, please add any validation that may be ran to validate all is good
,
Oct 17 2016
+tienchang@ for networking testing
,
Oct 17 2016
Tien, we will need extensive chrome-side testing for this. Are you aware of any Chrome-side tests that cycle through many http request/download cases?
,
Oct 17 2016
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/connectivity/shill/+/eb30c227ca70c7eb6cf541f9e60716d592d1a2a3 commit eb30c227ca70c7eb6cf541f9e60716d592d1a2a3 Author: Kirtika Ruchandani <kirtika@google.com> Date: Fri Oct 14 19:49:19 2016 HTTPRequest: Be resilient to bad data in input cb IOInputHandler's implementation calls the input callback if InputData has length < 0 (read failed). So, make the HTTPRequest terminate early if data->len < 0. This is a stop-gap fix for M54, until we fix IOInputHandler's behavior. Add a unit-test to ensure we don't crash on bad data. BUG=654807 TEST=shill unit-tests Change-Id: I98d38a5a359f3a81534a677762b17acf1d8ad35b [modify] https://crrev.com/eb30c227ca70c7eb6cf541f9e60716d592d1a2a3/http_request.cc [modify] https://crrev.com/eb30c227ca70c7eb6cf541f9e60716d592d1a2a3/http_request_unittest.cc
,
Oct 17 2016
FWIK, there is no end-to-end test that focuses on Chrome-side cycles through HTTP requests, but we do cycle through the top sites every FullRelease testing (https://testtracker.googleplex.com/testplans/testcase/detail/37144?id=375&revision=188). How else can we help verify this change?
,
Oct 18 2016
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/connectivity/shill/+/4a77ef07d730cfd5f3461d779028ab0a60e295f9 commit 4a77ef07d730cfd5f3461d779028ab0a60e295f9 Author: Kirtika Ruchandani <kirtika@google.com> Date: Sat Oct 15 22:09:51 2016 IOInputHandler: Don't call input cb with bad data Fix bug in IOInputHandler where it proceeds to call input_callback after detecting that the input length is negative. Make users of IOInputHandler resilient to bad data - - ICMPSession: Protect against corrupt data in replies - CryptoUtilProxy: Protect against bad data in HandleShimOutput - VPN: Add input validation in the input handler callbacks BUG=chromium:654807 TEST=shill unit-tests Change-Id: I5c7596df345974153bfe71df441ceb4094fb860e Reviewed-on: https://chromium-review.googlesource.com/399541 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Kevin Cernekee <cernekee@chromium.org> [modify] https://crrev.com/4a77ef07d730cfd5f3461d779028ab0a60e295f9/crypto_util_proxy.cc [modify] https://crrev.com/4a77ef07d730cfd5f3461d779028ab0a60e295f9/net/io_input_handler.cc [modify] https://crrev.com/4a77ef07d730cfd5f3461d779028ab0a60e295f9/icmp_session.cc
,
Oct 18 2016
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/connectivity/shill/+/4a77ef07d730cfd5f3461d779028ab0a60e295f9 commit 4a77ef07d730cfd5f3461d779028ab0a60e295f9 Author: Kirtika Ruchandani <kirtika@google.com> Date: Sat Oct 15 22:09:51 2016 IOInputHandler: Don't call input cb with bad data Fix bug in IOInputHandler where it proceeds to call input_callback after detecting that the input length is negative. Make users of IOInputHandler resilient to bad data - - ICMPSession: Protect against corrupt data in replies - CryptoUtilProxy: Protect against bad data in HandleShimOutput - VPN: Add input validation in the input handler callbacks BUG=chromium:654807 TEST=shill unit-tests Change-Id: I5c7596df345974153bfe71df441ceb4094fb860e Reviewed-on: https://chromium-review.googlesource.com/399541 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Kevin Cernekee <cernekee@chromium.org> [modify] https://crrev.com/4a77ef07d730cfd5f3461d779028ab0a60e295f9/crypto_util_proxy.cc [modify] https://crrev.com/4a77ef07d730cfd5f3461d779028ab0a60e295f9/net/io_input_handler.cc [modify] https://crrev.com/4a77ef07d730cfd5f3461d779028ab0a60e295f9/icmp_session.cc
,
Oct 19 2016
This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible! If all merges have been completed, please remove any remaining Merge-Approved labels from this issue. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Oct 19 2016
I'm pretty sure that M-53 isn't going to get any change, so removing that and marking that this change has been merged.
,
Oct 20 2016
This looks good. Ran curl script for about 100 iterations. Samus R55-8872.15.0 Cyan R54-8743.69.0
,
Dec 19 2016
Re-opening this, looks like my fix did not completely eradicate the bug. :( The new stack-trace is more cryptic than the original one. https://crash.corp.google.com/browse?q=product.name%3D%27ChromeOS%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27shill%3A%3AIOInputHandler%3A%3AOnFileCanReadWithoutBlocking%27%20OMIT%20RECORD%20IF%20SUM(ProductData.Key%3D%27exec_name%27)%20%3D%200%20OR%20SUM(ProductData.Value%3D%27shill%27)%20%3D%200&ignore_case=false&enable_rewrite=false&omit_field_name=&omit_field_value=&omit_field_opt=#samplereports The good news at least, is that the # of reports is lower.
,
Apr 6 2017
Merge here seems obsolete. |
|||||||||||||||
►
Sign in to add a comment |
|||||||||||||||
Comment 1 by tienchang@chromium.org
, Oct 11 2016