Issue metadata
Sign in to add a comment
|
Crash in QPCValueToTimeDelta |
||||||||||||||||||||||||||||||||
Issue descriptionDetailed report: https://cluster-fuzz.appspot.com/testcase?key=4582141380001792 Fuzzer: ifratric-browserfuzzer-v3 Job Type: windows_syzyasan_chrome Platform Id: windows Crash Type: UNKNOWN Crash Address: 0x24cdf8c7 Crash State: QPCValueToTimeDelta QPCNow base::TimeTicks::Now Recommended Security Severity: Medium Regressed: https://cluster-fuzz.appspot.com/revisions?job=windows_syzyasan_chrome&range=405656:405727 Unminimized Testcase: https://cluster-fuzz.appspot.com/download/AMIfv968JJS4oap8bU07zp8jwBslaBtm2eJz0JjuAsKkj92iwYXdIKeNP9m2DyNOaMEDf9mVZNKIMBXb_NH0uNqCi12k2jrn5MBjOASqOLr1HhKQbTvzpelUruzKPwVJSF7S8Fgj3JyfGw_wIYT4TdUmoQNpAZL0CWfSR_M-Ggm3W_k5Gh0TrAk?testcase_id=4582141380001792 Filer: mmoroz See https://dev.chromium.org/Home/chromium-security/bugs/reproducing-clusterfuzz-bugs for more information.
,
Jul 18 2016
,
Jul 18 2016
This issue is a security regression. If you are not able to fix this quickly, please revert the change that introduced it. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jul 18 2016
,
Jul 18 2016
,
Jul 18 2016
,
Jul 18 2016
Ping miu@
,
Jul 19 2016
M53 beta launch is next week.Your bug is labelled as Beta ReleaseBlock, pls make sure to land the fix before 6:00 PM PST, Friday (07/22/16). Thank you.
,
Jul 20 2016
Will look into this tomorrow. But, I suspect it's a WontFix since the code hasn't been changed in a long time, and the test case and stack trace don't indicate the fuzzing values can "reach" the function call in any way.
,
Jul 21 2016
,
Jul 21 2016
,
Jul 21 2016
I'm afraid sheriffbot's label changes were a hiccup - this is still a blocker for Friday's M53 (unless it is found to be WontFix, of course!)
,
Jul 21 2016
This looks almost exactly like bug 601442.
,
Jul 21 2016
BTW--I'm probably not the person who should look at this. I'm trying to set up a Windows system now to do some bisecting w/in the rev range the fuzzer mentioned, and it's taking forever just to "install corp updates" and "gclient sync" and foo. Due to scheduling conflicts, it's unlikely I'll be able too isolate the problem change until EOD Friday. Also, I'm 100% sure I made no code changes w/in the rev range the fuzzer pointed to. Maybe it'd be best for someone else (such as stability sheriff) to run the bisects and triage this issue?
,
Jul 21 2016
+current and future stability sheriffs
,
Jul 22 2016
,
Jul 22 2016
,
Jul 22 2016
Ah, Restrict-View-SecurityTeam means Stability-Sheriff-Desktop won't be effective, I see.
,
Jul 22 2016
+brucedawson: You may be interested in this crash.
,
Jul 22 2016
,
Jul 22 2016
,
Jul 22 2016
I wasn't able to do stability sheriff duties today. I tried examining the minidump in the report using the documentation at https://code.google.com/p/syzygy/wiki/SyzyASanBug, but "error_info->alloc_stack" doesn't exist. That may be where someone else can start off.
,
Jul 22 2016
,
Jul 22 2016
awhalley@, do we need this for next week M53 Dev on Tuesday?
,
Jul 22 2016
What's strange here is that the fuzzer is providing a rev range for the regression. I am interpreting this [perhaps incorrectly] as some change in that rev range that may be causing non-problematic code to crash. For example, maybe new/changed code is stomping on memory in a bad way. Also, yesterday, I was able to confirm the issue is not reproducible on a Linux build. This is definitely a Windows-only issue.
,
Jul 22 2016
I can't repro this at all, using the fuzzer test HTML, from running bisect_builds.py (32-bit on Windows) running on my local Win7 machine. Looking at this further, the fuzz testing is a renderer (not browser) crash. So, this doesn't seem release-blocking to me. The other labels (w.r.t. this being a security issue) seem wrong too (more sheriffbot@ automation gone wrong?). At this point I'm not sure what else I can do here. As this is Windows-specific with a call into Windows platform libraries being the crash point, I'm going to ask Bruce to take a look.
,
Jul 22 2016
I tried taking a look at the crash dump but I couldn't get the binaries or symbols to load. The instructions at:
https://code.google.com/p/syzygy/wiki/SyzyASanBug
just seem to say "Use the normal Chrome and Microsoft symbol servers", which I already had set up, but when I load the minidump from:
https://cluster-fuzz.appspot.com/testcase?key=4582141380001792
it failed to match the binary or symbols, whereas on normal crash dumps this works perfectly.
The 'crashing' instruction is:
mov dword ptr [ecx],eax.
The values of ECX and ESP are:
ECX = 24CDF8C4
ESP = 24CDF898
Therefore ECX is clearly a valid stack location (as in, within range) and it's not clear to me what asan is saying the problem is. Without a binary from the symbol server I can't even examine the parent functions to understand where ECX came from.
The error_type: UNKNOWN_BAD_ACCESS message is not very helpful.
It's hard to imagine how this could be a security bug - there is no user input involved, right? And the code involved has not changed recently, and is mostly straight through with no logic or branches or edge cases or pointer math, so it's not clear to me what could be wrong.
Anybody else have any luck getting the symbols to work? With those I could presumably understand this better, but more information on what asan is claiming the problem is would be helpful.
,
Jul 22 2016
miu: Crashes like this are assumed to be exploitable, hence the release block treatment. I couldn't repro a crash on win64 non-asan canary using all the flags in the report (except for --disable-gl-drawing-for-tests). The call to TimeTicks::Now in the crash is less than a month old. +stanisc, who added it in https://chromium.googlesource.com/chromium/src//+/56ee6be040caa993583170bd5edc472e7b969c9e, might have some insight?
,
Jul 22 2016
I'm removing from the stability sheriff queue; routing clusterfuzz bugs is (I think) handled by the bugs-- team. The stability sheriff is focused on issues that are crashing in the wild.
,
Jul 22 2016
TimeTicks::Now should be safe to call from anywhere, anytime. The bug, whatever it is, is almost certainly elsewhere. Any explanation of what the bug is actually saying (what is wrong with the write? It's to mapped stack memory, so what is asan saying is wrong with the address?) or help on getting symbol server working for the crash might help diagnose. There have apparently been reports in the past which blamed QPCNow for asan bugs and always the cause is elsewhere. So, presumably a real bug but for some reason asan is not reporting the location of the bug.
,
Jul 22 2016
,
Jul 23 2016
,
Jul 23 2016
,
Jul 24 2016
,
Jul 24 2016
,
Jul 25 2016
,
Jul 25 2016
,
Jul 29 2016
,
Aug 2 2016
Okay, progress update: 1) While the syzyasan documentation says that binaries/symbols should download automatically if you have the Chrome and Microsoft symbol servers enabled this does *not* work. This is a significant blocker because it makes the crash dump useless to anybody who is not persistent or does not have specialized knowledge. I managed to find the binaries and symbols by using "lmv m chrome_child" in windbg to find that the crash dump came from version 405727. I then downloaded https://www.googleapis.com/download/storage/v1/b/chromium-browser-syzyasan/o/win32-release%2Fasan-win32-release-405727.zip?generation=1468587677480000&alt=media and extracted the contents of the syzygy directory, added the extraction path to the symbol path, and reloaded symbols. Messy, but it did eventually work. Either the documentation should be changed to describe this process, or the symbols should actually be uploaded to the symbol server. I would prefer the latter. 2) Now what? The testcase data on cluster-fuzz.appspot.com includes 29 lines of "Bad access information", but most of this is just noise. The free_tid, free_stack_size, free_stack, and milliseconds_since_free lines are all irrelevant because this is not a use-after-free bug. I think the only relevant lines are these four: access_size: 4 error_type: UNKNOWN_BAD_ACCESS location: 0x24cdf8c7 Void access_mode: ASAN_WRITE_ACCESS Of these, the access_size and the fact that it is a write are both visible in the crash dump. The location is 3 greater than the address shown in the crash dump, and I don't understand that. The one crucial piece of information that I desperately want is "what is *wrong* with this write", and the metadata is silent on this - it just says UNKNOWN_BAD_ACCESS. So ultimately the 29 lines of information about the crash seems to add zero information - am I missing something? The write is to a stack location - presumably to a temporary created by the compiler to store the 64-bit result of QPCValueToTimeDelta. This should be completely static and consistent, so the bug should not be there. So the bug must be elsewhere - perhaps a stray write from another thread? - but without any indication of what are I filed crbug.com/524748 for these issues and more last year, but it appears that none of the issues reported there have been addressed. I'm closing this as WontFix because I cannot repro the bug. I'm also tempted to mark it as blocked on crbug.com/524748 , but that bug is closed so that would be a futile gesture.
,
Aug 2 2016
+cc a bunch of syzygy folks re: syzygy symbols issue in comment 39.
,
Aug 2 2016
crbug.com/525098 is the bug for the symbols not going on symbol server. A short-term fix would be to update the documentation so it doesn't say that the symbols are on symbol server, and instead says where to get them more directly.
,
Aug 11 2016
,
Nov 9 2016
This bug has been closed for more than 14 weeks. Removing security view restrictions. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot |
|||||||||||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||||||||||
Comment 1 by mmoroz@chromium.org
, Jul 18 2016Labels: Pri-2
Owner: m...@chromium.org