Update sqlite3 revision |
||||||||||||||||||
Issue descriptionCurrent ToT revision has a fix of a high severity security vulnerability (https://www.sqlite.org/src/info/c5dbc599b910c02a) found by our OSS-Fuzz project (https://clusterfuzz-external.appspot.com/v2/testcase-detail/5770842466156544). Also upstream contains updated fuzz target. We should use it instead of our existing fuzz target (I'll file a separate bug for this).
,
Nov 15 2016
,
Nov 15 2016
I don't think the indicated SQLite CL actually fixes this issue. The variable moved from the constant section to the recursion section is already in the recursion section for our SQLite. Probably later development attempted to move it up and now that CL is moving it back. Digging deeper, though.
,
Nov 15 2016
I am having no luck at all reproducing this on Linux or OSX. Building sql_unittests with gn flags:
is_asan = true
enable_nacl = false # Necessary until NaCl GN build is more complete.
is_debug = false # Release build.
use_goma = true
then running this test:
TEST_F(SQLConnectionTest, ASAN) {
sql::test::ScopedErrorExpecter expecter;
expecter.ExpectError(SQLITE_ERROR);
ASSERT_FALSE(db().Execute("CREATE table i(O\rUNIQUE\rUNIQUE"));
ASSERT_TRUE(expecter.SawExpectedErrors());
}
works just fine.
I've spent a couple hours trying to figure out how to repro on a Linux box, but AFAICT basically all of the clusterfuzz and oss-fuzz "How to repro locally" documentation describes some parallel universe of code and output. So I'm not going to continue trying to track this down.
,
Nov 16 2016
Have you tried to follow this instruction: https://github.com/google/oss-fuzz/blob/master/docs/reproducing.md ? Doesn't it work? Sadly to hear about your experience. OSS-Fuzz is in active development stage, so there can be some mistakes or missing points, though reproducing doc should be ok AFAIK. Would you mind sharing what exactly is incorrect or is "from parallel universe"? We would appreciate your feedback.
,
Nov 21 2016
Yet another one high severity bug has been fixed upstream: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=199 Patch link: https://www.sqlite.org/src/info/0a98c8d76ac86412
,
Nov 21 2016
,
Nov 21 2016
WRT #6, I can't see https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=199 either as shess@chromium.org nor as shess@google.com . I'm going to try to walk through things, again, and see if I can't figure out where they go awry. I think the "reproducing.md" document asked for information that wasn't present in the clusterfuzz output, and which I couldn't access in the oss-fuzz bug-tracker, so I assumed that I was supposed to follow the cluster-fuzz repro steps. But that seems to be for a different edition of clusterfuzz than is used on Chromium project, because it seemed to refer to things which didn't match up with anything in the report.
,
Nov 21 2016
OK, so when I follow the reproducing.md instructions, and run: docker run --rm -v /tmp/fuzz-3-sqlite3_ossfuzz:/testcase -t ossfuzz/sqlite3 reproduce ossfuzz I get: + ossfuzz /testcase INFO: Seed: 191371414 INFO: Loaded 0 modules (0 guards): ossfuzz: Running 1 inputs 1 time(s) each. Running: /testcase Executed /testcase in 1 ms I couldn't tell you if that reproduced the problem, or not. I'd guess not, because it didn't print the ASAN backtrace? Given that it's in some docker container, I'm not sure how to verify that it's even building with ASAN in there. There's a bin/compile file which references $SANITIZER_FLAGS, though I can't figure out who sets that. But the compile output contains things like "-fsanitize=address -fsanitize-coverage=edge,indirect-calls,8bit-counters" in CFLAGS, so somebody must be setting it, so the fact that I'm not getting any output maybe means it's flaky? Or there's something different about my build environment?
,
Nov 21 2016
In the testcase link from OP, I see: ASAN_OPTIONS = redzone=16:external_symbolizer_path=/bin/llvm-symbolizer:handle_sigill=1:strict_string_check=1:strict_memcmp=1:detect_container_overflow=1:coverage=0:allocator_may_return_null=1:use_sigaltstack=1:allocator_release_to_os=1:detect_stack_use_after_return=1:alloc_dealloc_mismatch=0:detect_leaks=0:print_scariness=1:strip_path_prefix=/workspace/:max_uar_stack_size_log=16:handle_abort=1:check_malloc_usable_size=0:quarantine_size_mb=10:detect_odr_violation=0:symbolize=1:handle_segv=1:fast_unwind_on_fatal=0 But in the reproduce output I only see: ASAN_OPTIONS=detect_leaks=0 Looks like that's coming from build.sh, maybe I can edit it (or maybe I'll just try to figure out how to pull the sources out of the docker container thing).
,
Nov 21 2016
shess@ That docker command runs a test with the head version of sqlite3. And the issue was already fixed there. If you want to run test with your local version of sqlite3, you can mount your sources over container's: -v $home/src/sqlite3:/src/sqlite3. Note: if your sqlite3 sources are older than couple weeks that wouldn't work either because they don't have the ossfuzz.c target. If this is the case, then taking ossfuzz.c out of sqlite3 head and compiling with your local version should work.
,
Nov 21 2016
Well, I figured out how to spin up a shell in there and patched the longer ASAN_OPTIONS into build.sh, and the build appeared to take that, and ... still no repro. I dunno, that's all I can think of to poke at from that end.
,
Nov 21 2016
aizatsky@ um ... I guess I would suggest that if you're going to automatically log bugs into the Chromium bug tracker, then they should be easy to reproduce against the Chromium version of SQLite, so that I can verify the bug fixed in Chromium. Having to comprehend both Chromium and some arbitrary group of third-party systems puts a barrier in the way of getting these things addressed. Is this something where I can create a separate ossfuzz.c target that builds against Chromium's third_party/sqlite/amalgamation/sqlite3.c, and then you can target the fuzzing infrastructure at that? The current system builds _an_ amalgamation and fuzzes against that, but AFAICT it doesn't contain the same SQLITE_* flags as Chromium uses when building the sqlite3.c it uses.
,
Nov 21 2016
shess@, you can add ossfuzz.c the same way as have another sqlite3 fuzzer (https://cs.chromium.org/chromium/src/third_party/sqlite/BUILD.gn?type=cs&q=sqlite3_prepare_v2_fuzzer&sq=package:chromium&l=270). It should be easy to build following Chromium instructions: https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reproducing.md#Reproducing-LibFuzzer-ASan-bugs (go/libfuzzer-chrome) The only possible problem I see is path to headers included in ossfuzz.c, since in Chromium we tend to use #include "third_party/......."
,
Nov 21 2016
Given a Chromium src dir and the original ossfuzz sqlite3 dir at /tmp/sqlite3 at $CHROMIUMSRC, then the following seems to work docker run --rm -v /tmp/fuzz-3-sqlite3_ossfuzz:/testcase -v$CHROMIUMSRC/third_party/sqlite/src:/src/sqlite3 -v/tmp/sqlite3/test/ossfuzz.c:/src/sqlite3/test/ossfuzz.c -t ossfuzz/sqlite3 reproduce ossfuzz It fails compile in shell.c, which verifies it's copying the right stuff into the container. So I patched the Chromium change out (it should be irrelevant). After that, it all runs successfully, which is to say - still no repro. Note my comment #3. The SQLite CL which the OP claims fixes the problem actually addresses a change which I believe landed after our version of SQLite. I haven't dug down to figure out the history between our SQLite and SQLite's head, but it seems plausible to me that the vulnerability in question never existed in Chromium's SQLite. [The ref in comment #6 is plausible in our SQLite, but since I can't access the ossfuzz report, I have no basis for comment.]
,
Nov 21 2016
I've CC'ed shess@ on https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=199. By the way, do I understand correctly that updating sqlite3 in Chromium is not an easy thing and, to get it done, someone should prove a crash or another significant problem?
,
Nov 21 2016
OK, on https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=199 , I get no-repro at SQLite head, repro when I comment out the lines changed by drh, then no-repro when I roll back to version-3.10.2 and layer in ossfuzz.c . I got no-repro against Chromium's SQLite, also, so I think it's not an issue for Chromium at this time.
,
Nov 21 2016
WRT updating SQLite, basically I am the only person who spends any time on SQLite, and most Chromium SQLite clients are honestly in maintenance mode with poor test coverage (it might take awhile for anyone to notice errors with any subtlety), so I prefer to update about once a year, rather than attempting some sort of rolling integration system. Actually doing the code update is not hard, but there are a couple local patches we have which are somewhat annoying to validate. Notably the WebSQL integration was done by monkey-patching SQLite, and while I have made the actual change cleaner to apply over time, the assumptions baked into the implementation mean that we frequently get bitten by valid SQLite implementation changes. WebSQL has been deprecated for a long time, so there's not much support for rewriting the implementation to be cleaner - yet every time I bring up deleting the feature, I get pushback. Since WebSQL is the only place where non-Chromium SQLite code can be injected, maybe the time is ripe to take another pass at that. [Most fuzzer reports seem to be in parsing-related stuff for very odd statements, and thus should not be relevant to Chromium browser-side SQLite usage.] Also, to be clear, are you asserting that you have more confidence in the security of rolling SQLite integration than in picking a decent release and sticking with it for awhile? At least from the fuzzer level? I ask because while I do trust the SQLite team (they are on average more solid than Chromium developers, IMHO), bugs do get in there, and per the point in my first paragraph, I'm the only resource working on this. I'd prefer not to get on the treadmill of having to track all SQLite issues in case they impact Chromium.
,
Nov 21 2016
Still regardless of how Chromium approaches SQLite integration, I don't think logging SQLite problems against Chromium is worthwhile unless they repro with the SQLite Chromium is actually using. That just wastes time in wild goose chases.
,
Nov 22 2016
Thanks a lot shess@ for detailed explanation, it makes sense. I haven't thought that it's so complicated (i.e. we have some patches on top of the upstream, significantly different build configurations, etc.). I followed the common security sense like "If library A has security bugs and we are using it, we should update it". I've also added ossfuzz.c target and verified that both issues are not reproducible inside Chromium. Your concern about usage of sqlite in Chromium differs from its usage inside fuzzer is totally valid. This is one of the reasons why we are encouraging developers to write/update fuzz targets - they have better knowledge how target is being used and how it should be tested. Since we are not going to update sqlite3 for now, let me add ossfuzz.c fuzzer to the Chromium repo. Then, after an update, we will change sources of that fuzzer_test GN target, but will be running on some existing corpus, which is a good thing.
,
Nov 22 2016
,
Nov 22 2016
,
Nov 22 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/656b55356751d12fd8c643f927691275ef10dded commit 656b55356751d12fd8c643f927691275ef10dded Author: mmoroz <mmoroz@chromium.org> Date: Tue Nov 22 22:04:43 2016 Add ossfuzz.c fuzzer for sqlite3 (taken from the upstream). R=aizatsky@chromium.org, inferno@chromium.org, ochang@chromium.org, shess@chromium.org BUG= 665382 Review-Url: https://codereview.chromium.org/2524753002 Cr-Commit-Position: refs/heads/master@{#434002} [modify] https://crrev.com/656b55356751d12fd8c643f927691275ef10dded/third_party/sqlite/BUILD.gn [add] https://crrev.com/656b55356751d12fd8c643f927691275ef10dded/third_party/sqlite/fuzz/ossfuzz.c [add] https://crrev.com/656b55356751d12fd8c643f927691275ef10dded/third_party/sqlite/fuzz/sql.dict
,
Nov 23 2016
to reflect reality
,
Dec 2 2016
,
Dec 6 2016
shess: Uh oh! This issue still open and hasn't been updated in the last 14 days. This is a serious vulnerability, and we want to ensure that there's progress. Could you please leave an update with the current status and any potential blockers? If you're not the right owner for this issue, could you please remove yourself as soon as possible or help us find the right one? If the issue is fixed or you can't reproduce it, please close the bug. If you've started working on a fix, please set the status to Started. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Dec 21 2016
shess: Uh oh! This issue still open and hasn't been updated in the last 29 days. This is a serious vulnerability, and we want to ensure that there's progress. Could you please leave an update with the current status and any potential blockers? If you're not the right owner for this issue, could you please remove yourself as soon as possible or help us find the right one? If the issue is fixed or you can't reproduce it, please close the bug. If you've started working on a fix, please set the status to Started. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jan 14 2017
We commit ourselves to a 60 day deadline for fixing for high severity vulnerabilities, and have exceeded it here. If you're unable to look into this soon, could you please find another owner or remove yourself so that this gets back into the security triage queue? For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jan 26 2017
,
Feb 14 2017
I don't think we need to keep this in the security queue. The bugs in question have been public for a long time and there's no clear indication they affect Chrome's use of them.
,
Mar 27 2017
,
Mar 31 2017
Awesome! Thank you Scott!
,
Jun 21 2017
,
Jun 27 2017
It doesn't look like this is a localization issue. Can't confirm that it's still relevant to the current English UI
,
Jun 28 2017
Unable to triage this issue from TE-End, adding "TE-NeedsTriageHelp" label for further triage.
,
Sep 12 2017
,
Sep 12 2017
Since shess@ is not at Google anymore, assigning this to sqlite owner michaeln@ (https://cs.chromium.org/chromium/src/third_party/sqlite/OWNERS?sq=package:chromium)
,
Oct 24 2017
For more information, please see https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reference.md. The link referenced in the description is no longer valid. (bulk edit)
,
Oct 24 2017
I have upgraded SQLite since that bug happened. We're on 3.20.1 now.
,
Oct 24 2017
|
||||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||||
Comment 1 by mmoroz@chromium.org
, Nov 15 2016