New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 626373 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocked on:
issue 629966

Blocking:
issue 616183
issue 634965



Sign in to add a comment

Windows Clang ASAN does not detect null ptr crashes and __built_intrap crashes

Project Member Reported by infe...@chromium.org, Jul 7 2016

Issue description

1. test1.html is null ptr crash . On linux it shows
==1==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f8890c1386e bp 0x7ffca5f40460 sp 0x7ffca5f40430 T0)
==1==The signal is caused by a READ memory access.
==1==Hint: address points to the zero page.
    #0 0x7f8890c1386d in isPseudoElement /media/aarya/ssd/chromium/src/out/Release/../../third_party/WebKit/Source/core/dom/Node.h:249:43
    #1 0x7f8890c1386d in blink::Node::hasEditableStyle(blink::Node::EditableLevel, blink::Node::UserSelectAllTreatment) const /media/aarya/ssd/chromium/src/out/Release/../../third_party/WebKit/Source/core/dom/Node.cpp:551

2. test2.html is a crash on release assert which gets routed to IMMEDIATE_CRASH macro using __built_intrap. On linux it shows,
==1==ERROR: AddressSanitizer: ILL on unknown address 0x7f9d21183254 (pc 0x7f9d21183254 bp 0x7ffe84143090 sp 0x7ffe84142fc0 T0)
    #0 0x7f9d21183253 in blink::CreateMarkupAlgorithm<blink::EditingAlgorithm<blink::NodeTraversal> >::createMarkup(blink::PositionTemplate<blink::EditingAlgorithm<blink::NodeTraversal> > const&, blink::PositionTemplate<blink::EditingAlgorithm<blink::NodeTraversal> > const&, blink::EAnnotateForInterchange, blink::ConvertBlocksToInlines, blink::EAbsoluteURLs, blink::Node*) /media/aarya/ssd/chromium/src/out/Release/../../third_party/WebKit/Source/core/editing/serializers/Serialization.cpp:248:5
    #1 0x7f9d211856b5 in blink::createMarkup(blink::PositionTemplate<blink::EditingAlgorithm<blink::NodeTraversal> > const&, blink::PositionTemplate<blink::EditingAlgorithm<blink::NodeTraversal> > const&, blink::EAnnotateForInterchange, blink::ConvertBlocksToInlines, blink::EAbsoluteURLs, blink::Node*) /media/aarya/ssd/chromium/src/out/Release/../../third_party/WebKit/Source/core/editing/serializers/Serialization.cpp:267:12

On Windows Clang ASAN, both of these are these are not caught and get caught by the regular exception filter here (added by jam@ in https://chromium.googlesource.com/chromium/src/+/79dc59ac7602413181079ecb463873e29a1d7d0a). Regular exception filter just give stacktrace and provides no useful information for stack deduplication/determining security impact.

We want ASAN stacktrace here since it helps with two things::
1- For near null crashes or even ones at higher addresses (where ASAN does not have redzone), we will get crash address. Crash address helps in determining security impact.
2- For built_intrap crashes, we need some way to distinguish these from regular crashes. Like on linux, we see string like "ILL on unknown address". If ASAN catches these, it can show something similar.

SyzyASAN catches 1) since it has its own custom exception filter in agent_logger.exe. Unsure if it catches 2) / release assert crashes.

Just a fyi, 1) is way more important for us. So, if that can be fixed, that would be great.
 
test1.html
760 bytes View Download
test2.html
51.4 KB View Download
Cc: glider@chromium.org kcc@chromium.org euge...@chromium.org dxf@google.com
Reid, friendly ping. Can you please atleast look in 1), not catching UNKNOWN address crashes is a bad ASAN regression on Windows. 2) is a lower priority.

Comment 2 by r...@chromium.org, Aug 2 2016

Cc: etienneb@chromium.org

Comment 3 by r...@chromium.org, Aug 3 2016

ASan already appears to detect null derefs and wild memory accesses, according to this test:
https://cs.chromium.org/chromium/src/base/tools_sanity_unittest.cc?q=ToolsSanityTest&sq=package:chromium&dr=C&l=192

If I manually enable the test, it produces the expected output, which is a chrome stack trace followed by an ASan report:
https://ghostbin.com/paste/3dq2u

I'm building full chrome with ASan to try your test case, and I'll try to figure out what the issue is.

Printing an ASan report on illegal instructions should be pretty easy, I have a simple patch for it that I can test now.

Comment 4 by r...@chromium.org, Aug 3 2016

Adding SIGILL handling:
 https://reviews.llvm.org/D23098

We might need to do something more to intercept calls to abort(), since I'm not sure if that raises an exception.

Comment 5 by aarya@google.com, Aug 3 2016

Regarding c#3, I think this is just broken for Chrome (not unittests). I think https://chromium.googlesource.com/chromium/src/+/79dc59ac7602413181079ecb463873e29a1d7d0a caused it, but someone needs to look closely into how ASAN can override the exception filter added by that CL.

Comment 6 by r...@chromium.org, Aug 3 2016

I built and ran CrWinASan on test1.html and got this output, which contains both a chrome stack trace and an ASan report:
https://ghostbin.com/paste/jj2bq

I had to pipe stderr to a file, though, as you may recall: chrome 2>cr_err.txt
I think CF is already doing that...

Is CF doing something to suppress the unhandled exception filter?

Comment 7 by aarya@google.com, Aug 4 2016

Yes CF use python subprocess and hence gets stderr and uses --enable-logging. CF does not do anything to suppress unhandled exception filter. Let me reupload this testcase to CF to show console output.
Project Member

Comment 8 by ClusterFuzz, Aug 4 2016

ClusterFuzz is analyzing your testcase. Developers can follow the progress at https://cluster-fuzz.appspot.com/testcase?key=5642843932852224
Looking at https://ghostbin.com/paste/jj2bq, i found one major difference between your build and the build at builder. You seem to be using component/shared build and builder uses static library build. So, i checked this locally. On static build, this is broken, only backtrace is thrown and no ASAN stack. On shared library build, this works.

Backtrace:
	blink::Node::hasEditableStyle [0x621E3CBC+28]
	blink::CompositeEditCommand::insertNodeBefore [0x6136C18D+125] (D:\src\chromium\src\third_party\WebKit\Source\core\editing\commands\CompositeEditCommand.cpp:324)
	blink::ReplaceSelectionCommand::doApply [0x613F6B0D+17677] (D:\src\chromium\src\third_party\WebKit\Source\core\editing\commands\ReplaceSelectionCommand.cpp:1338)
	blink::CompositeEditCommand::apply [0x6136A20E+558] (D:\src\chromium\src\third_party\WebKit\Source\core\editing\commands\CompositeEditCommand.cpp:209)
	blink::executeInsertHTML [0x613A7E8C+172] (D:\src\chromium\src\third_party\WebKit\Source\core\editing\commands\EditorCommand.cpp:614)
	blink::Editor::Command::execute [0x613A1CB1+929] (D:\src\chromium\src\third_party\WebKit\Source\core\editing\commands\EditorCommand.cpp:1811)
	blink::Document::execCommand [0x62076E77+871]
	blink::DocumentV8Internal::execCommandMethodCallback [0x5FD2D0D8+2888]
	v8::internal::FunctionCallbackArguments::Call [0x06F664E6+886] (D:\src\chromium\src\v8\src\api-arguments.cc:19)
	v8::internal::`anonymous namespace'::HandleApiCallHelper<0> [0x064C523F+3391] (D:\src\chromium\src\v8\src\builtins.cc:5770)
	v8::internal::Builtin_Impl_HandleApiCall [0x0658100A+1338] (D:\src\chromium\src\v8\src\builtins.cc:5800)
	v8::internal::Builtin_HandleApiCall [0x064D118E+238] (D:\src\chromium\src\v8\src\builtins.cc:5788)
	v8::internal::`anonymous namespace'::Invoke [0x05B1D36D+1133] (D:\src\chromium\src\v8\src\execution.cc:98)
	blink::StringCache::createStringAndInsertIntoCache [0x5F366D1D+2173] (D:\src\chromium\src\third_party\WebKit\Source\bindings\core\v8\V8ValueCache.cpp:193)
	base::internal::Invoker<base::internal::BindState<v8::MaybeLocal<v8::Script> (*)(blink::(anonymous namespace)::V8CompileHistogram::Cacheability, v8::Isolate *, v8::Local<v8::String>, v8::ScriptOrigin),blink::(anonymous namespace)::V8CompileHistogram::Cach [0x5F355B7E+414] (D:\src\chromium\src\base\bind_internal.h:249)
=================================================================
==13028==ERROR: AddressSanitizer: access-violation on unknown address 0x00000000 (pc 0x621e3cbc bp 0x0025ac28 sp 0x0025ac18 T0)
==13028==The signal is caused by a READ memory access.
==13028==Hint: address points to the zero page.
==13028==WARNING: Failed to use and restart external symbolizer!
==13028==*** WARNING: Failed to initialize DbgHelp!              ***
==13028==*** Most likely this means that the app is already      ***
==13028==*** using DbgHelp, possibly with incompatible flags.    ***
==13028==*** Due to technical reasons, symbolization might crash ***
==13028==*** or produce wrong results.                           ***
    #0 0x621e3cbb in blink::Node::hasEditableStyle+0x1b (D:\src\chromium\src\out\Release\webcore_shared.dll+0x3283cbb)
    #1 0x6136c18c in blink::CompositeEditCommand::insertNodeBefore D:\src\chromium\src\third_party\WebKit\Source\core\editing\commands\CompositeEditCommand.cpp:322
    #2 0x613f6b0c in blink::ReplaceSelectionCommand::doApply D:\src\chromium\src\third_party\WebKit\Source\core\editing\commands\ReplaceSelectionCommand.cpp:1338
    #3 0x6136a20d in blink::CompositeEditCommand::apply D:\src\chromium\src\third_party\WebKit\Source\core\editing\commands\CompositeEditCommand.cpp:209
    #4 0x613a7e8b in blink::executeInsertHTML D:\src\chromium\src\third_party\WebKit\Source\core\editing\commands\EditorCommand.cpp:614
    #5 0x613a1cb0 in blink::Editor::Command::execute D:\src\chromium\src\third_party\WebKit\Source\core\editing\commands\EditorCommand.cpp:1811
    #6 0x62076e76 in blink::Document::execCommand+0x366 (D:\src\chromium\src\out\Release\webcore_shared.dll+0x3116e76)
    #7 0x5fd2d0d7 in blink::DocumentV8Internal::execCommandMethodCallback+0xb47 (D:\src\chromium\src\out\Release\webcore_shared.dll+0xdcd0d7)
    #8 0x6f664e5 in v8::internal::FunctionCallbackArguments::Call D:\src\chromium\src\v8\src\api-arguments.cc:19
    #9 0x64c523e in v8::internal::`anonymous namespace'::HandleApiCallHelper<0> D:\src\chromium\src\v8\src\builtins.cc:5770
    #10 0x6581009 in v8::internal::Builtin_Impl_HandleApiCall D:\src\chromium\src\v8\src\builtins.cc:5800
    #11 0x64d118d in v8::internal::Builtin_HandleApiCall D:\src\chromium\src\v8\src\builtins.cc:5788
    #12 0x5b1d36c in v8::internal::`anonymous namespace'::Invoke D:\src\chromium\src\v8\src\execution.cc:98
    #13 0x5f366d1c in blink::StringCache::createStringAndInsertIntoCache D:\src\chromium\src\third_party\WebKit\Source\bindings\core\v8\V8ValueCache.cpp:193
    #14 0x5f355b7d in base::internal::Invoker<base::internal::BindState<v8::MaybeLocal<v8::Script> (*)(blink::(anonymous namespace)::V8CompileHistogram::Cacheability, v8::Isolate *, v8::Local<v8::String>, v8::ScriptOrigin),blink::(anonymous namespace)::V8CompileHistogram::Cacheability>,v8::MaybeLocal<v8::Script> (v8::Isolate *, v8::Local<v8::String>, v8::ScriptOrigin)>::Run D:\src\chromium\src\base\bind_internal.h:249

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: access-violation D:\src\chromium\src\third_party\WebKit\Source\core\dom\Node.cpp:550 in blink::Node::hasEditableStyle
==13028==ABORTING

Any idea why we have static library build there ?

I also looked at other things. Like for d8 shell, it works fine there, even with static build, e.g. see https://cluster-fuzz.appspot.com/testcase?key=6523618580496384.

Comment 10 by r...@chromium.org, Aug 4 2016

I rebuilt without DLLs over lunch, and was able to reproduce the problem. I suspect this has to do with the way that Chrome statically links the CRT, which changed with VS2015.

Currently ASan fights with the CRT to replace the unhandled exception filter.

Comment 11 by r...@chromium.org, Aug 4 2016

I think this is why things don't work:
https://cs.chromium.org/chromium/src/chrome_elf/chrome_elf_main.cc?rcl=0&l=53

Basically, chrome_elf defeats ASan's attempts to install its exception filter. We could work around this in ASan, but I think it would be better to ifdef out that chrome_elf code when ASan is active.

This doesn't affect the DLL build because the ELF code is only patching the IAT of the executable, so it wouldn't affect the dynamic asan runtime.
Cc: robertshield@chromium.org
Blockedon: 634965
Reid's patches for the two problems.

1. https://codereview.chromium.org/2220583002/
2. https://reviews.llvm.org/rL277621
Blocking: 634965
Blockedon: -634965
Project Member

Comment 17 by bugdroid1@chromium.org, Aug 5 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3894e46c8c2702e12d2ddec12f186d7c1cf5c412

commit 3894e46c8c2702e12d2ddec12f186d7c1cf5c412
Author: rnk <rnk@chromium.org>
Date: Fri Aug 05 16:32:37 2016

Don't IAT patch SetUnhandledExceptionFilter when ASan is active

ASan installs its own exception filter that ClusterFuzz needs to see.

R=robertshield@chromium.org,inferno@chromium.org
BUG= 626373 

Review-Url: https://codereview.chromium.org/2220583002
Cr-Commit-Position: refs/heads/master@{#410077}

[modify] https://crrev.com/3894e46c8c2702e12d2ddec12f186d7c1cf5c412/chrome_elf/chrome_elf_main.cc

Comment 18 by r...@chromium.org, Aug 5 2016

Status: Fixed (was: Assigned)
Should be fixed, let me know if there are issues with this in CF.

Comment 19 by r...@chromium.org, Aug 5 2016

Status: Assigned (was: Fixed)
Actually, the __builtin_trap side of this is blocked on rolling clang, I spoke too soon.

Comment 20 by r...@chromium.org, Aug 5 2016

Blockedon: 629966
Project Member

Comment 21 by bugdroid1@chromium.org, Aug 5 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/07928da2532a9a7d1149d0878d0931dd209f0d3c

commit 07928da2532a9a7d1149d0878d0931dd209f0d3c
Author: rnk <rnk@chromium.org>
Date: Fri Aug 05 18:00:53 2016

Revert of Don't IAT patch SetUnhandledExceptionFilter when ASan is active (patchset #1 id:1 of https://codereview.chromium.org/2220583002/ )

Reason for revert:
This causes crashpad to emit "haven't called UseHandler()" errors later.

Original issue's description:
> Don't IAT patch SetUnhandledExceptionFilter when ASan is active
>
> ASan installs its own exception filter that ClusterFuzz needs to see.
>
> R=robertshield@chromium.org,inferno@chromium.org
> BUG= 626373 
>
> Committed: https://crrev.com/3894e46c8c2702e12d2ddec12f186d7c1cf5c412
> Cr-Commit-Position: refs/heads/master@{#410077}

TBR=inferno@chromium.org,robertshield@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 626373 

Review-Url: https://codereview.chromium.org/2225453002
Cr-Commit-Position: refs/heads/master@{#410106}

[modify] https://crrev.com/07928da2532a9a7d1149d0878d0931dd209f0d3c/chrome_elf/chrome_elf_main.cc

Cc: ananta@chromium.org
It's not immediately obvious to me why that fix didn't work. Is it in a particular config ([non-]component, etc.)?
I did another clobber build on my local checkout and synced to head and can't reproduce the UseHandler() errors anymore that I pointed to Reid before. So, reverting the revert.
Project Member

Comment 24 by bugdroid1@chromium.org, Aug 6 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/009fe04704c6cd15b521ce44026331fa1bd37f11

commit 009fe04704c6cd15b521ce44026331fa1bd37f11
Author: inferno <inferno@chromium.org>
Date: Sat Aug 06 00:46:48 2016

Reland of Don't IAT patch SetUnhandledExceptionFilter when ASan is active (patchset #1 id:1 of https://codereview.chromium.org/2225453002/ )

Reason for revert:
This patch didn't impact Crashpad.

Original issue's description:
> Revert of Don't IAT patch SetUnhandledExceptionFilter when ASan is active (patchset #1 id:1 of https://codereview.chromium.org/2220583002/ )
>
> Reason for revert:
> This causes crashpad to emit "haven't called UseHandler()" errors later.
>
> Original issue's description:
> > Don't IAT patch SetUnhandledExceptionFilter when ASan is active
> >
> > ASan installs its own exception filter that ClusterFuzz needs to see.
> >
> > R=robertshield@chromium.org,inferno@chromium.org
> > BUG= 626373 
> >
> > Committed: https://crrev.com/3894e46c8c2702e12d2ddec12f186d7c1cf5c412
> > Cr-Commit-Position: refs/heads/master@{#410077}
>
> TBR=inferno@chromium.org,robertshield@chromium.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG= 626373 
>
> Committed: https://crrev.com/07928da2532a9a7d1149d0878d0931dd209f0d3c
> Cr-Commit-Position: refs/heads/master@{#410106}

TBR=robertshield@chromium.org,scottmg@chromium.org,rsesek@chromium.org,aarya@google.com,rnk@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 626373 

Review-Url: https://codereview.chromium.org/2217833004
Cr-Commit-Position: refs/heads/master@{#410232}

[modify] https://crrev.com/009fe04704c6cd15b521ce44026331fa1bd37f11/chrome_elf/chrome_elf_main.cc

Status: Fixed (was: Assigned)
Clang roll is sticking and other patch worked, so closing. We still need to add handle_sigill in CF, but that is tracked in another bug owned by Max.

Sign in to add a comment