Out-of-memory in sqlite3_ossfuzz_fuzzer |
||||||||
Issue descriptionDetailed report: https://cluster-fuzz.appspot.com/testcase?key=6381137577639936 Fuzzer: libfuzzer_sqlite3_ossfuzz_fuzzer Job Type: libfuzzer_chrome_ubsan Platform Id: linux Crash Type: Out-of-memory Crash Address: Crash State: sqlite3_ossfuzz_fuzzer Regressed: https://cluster-fuzz.appspot.com/revisions?job=libfuzzer_chrome_ubsan&range=433990:434098 Minimized Testcase (1.34 Kb): Download: https://cluster-fuzz.appspot.com/download/AMIfv96gov3kLhUVJtG75EXN2kp9eZspWi8LejDBETIRyCwrx5Lml3v1i1FI9G3OnWdpqs0LFZFY1u1soEM-GGxJh3O5WLHVyUW-8NT0Z7n180JycdO9ocXK-xOKS0lBDMHw_Bf12aPhHsDAv7il3VVG0rOZ9QujAg?testcase_id=6381137577639936 CREATE TABLE t1(a INTEGER PRI���� KEY, b); INSERT INTO t1 VALUES(1, randomblob(2000)); INSERT INTO t1 VALUES(2, randomblob(1000)); INSERT INTO t1 SELECT a+2, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+4, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+8, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+16, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+2, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+4, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+8, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+2, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+4, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+8, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+16, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+2, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+4, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+8, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+16, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+32, randomblob(1500) FROM t1; INSERT INTO t1 SELECT a+16, randomblob(1500) FROM t1; --sleep 1 INSERT INTO t1 SELECT a+32, randomblob(1500) FROM t1; --wait 1 --task 2 DROP TABLE IF EXISTS t2; CREATE TABLE t2(a INTEGER�PRIMARY KEY, b); INSERT INTO t2 SELECT a, b FROM t1; UPDATE t1 SET b='x'||a||'y'; SELECT sum(length(b)) FROM t2; SELECT a FROM t2 WHERE b='x17y'; Issue filed automatically. See https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reproducing.md for more information.
,
Dec 19 2016
Hm, not sure I understand why this input eats so much RAM. AFAIK, randomblob has a size-in-bytes argument, so these randomblob() calls shouldn't use that much of memory. CC'ing sqlite3 experts to know their opinion.
,
Dec 19 2016
,
Dec 20 2016
The individual randomblob() calls don't, but the INSERT INTO t1 SELECT FROM t1 calls are exponentially growing the database (because the INTEGER PRI* KEY is ignored), and the fuzzer uses an in-memory database. Running those in sqlite3 against the filesystem, it creates a 771MB database through --wait 1, then the t2 INSERT doubles that, and the t1 UPDATE touches a bunch of that (I mention this because the filesystem version ends up with an additional journal factor - I don't think the in-memory database will have that duplication).
,
Dec 21 2016
Thanks for the explanation shess@. I wonder if we can restrict max size during compilation. According to https://www.sqlite.org/limits.html, it seems to be possible via definition of SQLITE_MAX_PAGE_COUNT. With the maximum page size of 65536 bytes, we can use SQLITE_MAX_PAGE_COUNT = 1024, for example, to have max database size of ~64MB. Would it be fine?
,
Dec 26 2016
,
Dec 30 2016
https://codereview.chromium.org/2609473004
,
Jan 1 2017
I'm not sure SQLITE_MAX_PAGE_COUNT=1024 is appropriate under using_sanitizer, because it could prevent running a reasonable Chromium build or reasonable non-fuzzer tests under a sanitizer. I'll think about whether 16384 (up to 1G with 64k pages, but more like 16M or 64M depending on the SQLite default page size). The fuzzer main() could run PRAGMA to restrict the size, but the fuzzing could undo that, I don't see anything offhand that could otherwise fix it while still using a representative allocation strategy (using a custom pcache or allocation pool might not lead to generally-applicable results).
,
Jan 1 2017
Actually, I think SQLITE_MAX_PAGE_COUNT can be overridden at runtime with PRAGMA max_page_count, so maybe just calling this in the fuzzer main() is reasonable enough.
,
Jan 2 2017
That's a valid concern, thanks for pointing this out. I've tried to use PRAGMA max_page_count, but the speed reduced significantly (from ~4500 to ~2500 execs per second). Since we don't want to lose the speed, let's harden the condition for defining that flag (.. if (use_libfuzzer || use|afl) ...). Also I've increased the value to 16384: https://codereview.chromium.org/2609473004
,
Mar 20 2017
,
Apr 19 2017
ClusterFuzz testcase 6381137577639936 is flaky and no longer reproduces, so closing issue. If this is incorrect, please add ClusterFuzz-Wrong label and re-open the issue.
,
Apr 28 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/927053fa078035ee9bea31ef77184ccbe8001745 commit 927053fa078035ee9bea31ef77184ccbe8001745 Author: mmoroz <mmoroz@chromium.org> Date: Fri Apr 28 14:10:22 2017 [sqlite3] Limit max number of memory pages for fuzzing builds. R=shess@chromium.org BUG= 675446 Review-Url: https://codereview.chromium.org/2609473004 Cr-Commit-Position: refs/heads/master@{#467984} [modify] https://crrev.com/927053fa078035ee9bea31ef77184ccbe8001745/third_party/sqlite/BUILD.gn |
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by msrchandra@chromium.org
, Dec 19 2016Labels: Test-Predator-Wrong
Owner: mmoroz@chromium.org
Status: Assigned (was: Untriaged)