New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 761005 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Sep 5
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Windows , Mac
Pri: 1
Type: Bug



Sign in to add a comment

Out-of-memory in pdfium_fuzzer

Project Member Reported by ClusterFuzz, Aug 31 2017

Issue description

Detailed report: https://clusterfuzz.com/testcase?key=6611073234108416

Fuzzer: libFuzzer_pdfium_fuzzer
Job Type: libfuzzer_chrome_msan
Platform Id: linux

Crash Type: Out-of-memory (exceeds 2048 MB)
Crash Address: 
Crash State:
  pdfium_fuzzer
  
Sanitizer: memory (MSAN)

Regressed: https://clusterfuzz.com/revisions?job=libfuzzer_chrome_msan&range=395689:395794

Reproducer Testcase: https://clusterfuzz.com/download?testcase_id=6611073234108416

Issue filed automatically.

See https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reproducing.md for more information.
 
Cc: msrchandra@chromium.org
Labels: M-62 Test-Predator-Wrong
Owner: dsinclair@chromium.org
Status: Assigned (was: Untriaged)
Predator and CL could not provide any possible suspects.
Using Code Search for the file, "pdfium_fuzzer" assigning to the concern owner who previously worked on similar fuzzers.

@dsinclair -- Could you please look into the issue, kindly re-assign if this is not related to your changes.
Thank You.
Cc: dsinclair@chromium.org
Owner: rharrison@chromium.org
rharrison@ can you take a look?
Status: Started (was: Assigned)
Project Member

Comment 4 by bugdroid1@chromium.org, Sep 8 2017

The following revision refers to this bug:
  https://pdfium.googlesource.com/pdfium/+/808b52ac76bb5d9ee3e6a8371ddab25f62c8ed51

commit 808b52ac76bb5d9ee3e6a8371ddab25f62c8ed51
Author: Ryan Harrison <rharrison@chromium.org>
Date: Fri Sep 08 15:44:34 2017

Move decompressing of JPX out of Init and into Decode

In the existing implementation of the JPX decoder, Init extracts the
header from the image and then immediately decompresses it. This is
problematic if it is a very large image that we won't be able to
allocate a bitmap for. The code has been changed to instead delay
decompression until the Decode method, since things like dest Bitmap
generation can be performed using just the header information.

There is also a bit of renaming/casting cleanup, because I was having
a hard time parsing what was a local vs member variable.

BUG= chromium:761005 

Change-Id: I55a55c0be2f88a5352a6ca056c2a816137d7c749
Reviewed-on: https://pdfium-review.googlesource.com/13550
Reviewed-by: Henrique Nakashima <hnakashima@chromium.org>
Commit-Queue: Ryan Harrison <rharrison@chromium.org>

[modify] https://crrev.com/808b52ac76bb5d9ee3e6a8371ddab25f62c8ed51/core/fxcodec/codec/fx_codec_jpx_opj.cpp
[modify] https://crrev.com/808b52ac76bb5d9ee3e6a8371ddab25f62c8ed51/core/fxcodec/codec/cjpx_decoder.h

Status: Fixed (was: Started)
Project Member

Comment 6 by bugdroid1@chromium.org, Sep 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/01bd094255d30e10ec211f8e9b897b32d9c3bc18

commit 01bd094255d30e10ec211f8e9b897b32d9c3bc18
Author: pdfium-deps-roller@chromium.org <pdfium-deps-roller@chromium.org>
Date: Fri Sep 08 21:14:35 2017

Roll src/third_party/pdfium/ f76741e65..e8b3e0cdc (2 commits)

https://pdfium.googlesource.com/pdfium.git/+log/f76741e65e99..e8b3e0cdcea3

$ git log f76741e65..e8b3e0cdc --date=short --no-merges --format='%ad %ae %s'
2017-09-08 tsepez Fix one-second spin in fx_random.cpp
2017-09-08 rharrison Move decompressing of JPX out of Init and into Decode

Created with:
  roll-dep src/third_party/pdfium
BUG= 761005 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


TBR=dsinclair@chromium.org

Change-Id: Icb29aacd236d0890557fa387cffe53b2d3fce484
Reviewed-on: https://chromium-review.googlesource.com/657264
Reviewed-by: <pdfium-deps-roller@chromium.org>
Commit-Queue: <pdfium-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#500668}
[modify] https://crrev.com/01bd094255d30e10ec211f8e9b897b32d9c3bc18/DEPS

Project Member

Comment 7 by bugdroid1@chromium.org, Sep 11 2017

The following revision refers to this bug:
  https://pdfium.googlesource.com/pdfium/+/4588792ebfcfdd628b76f005fa0e1d7935739139

commit 4588792ebfcfdd628b76f005fa0e1d7935739139
Author: Ryan Harrison <rharrison@chromium.org>
Date: Mon Sep 11 20:30:16 2017

Address nits missed from original review

BUG= chromium:761005 

Change-Id: I538e49f1eeb32891b33677d8587d2bed110b1fc1
Reviewed-on: https://pdfium-review.googlesource.com/13692
Reviewed-by: Tom Sepez <tsepez@chromium.org>
Commit-Queue: Ryan Harrison <rharrison@chromium.org>

[modify] https://crrev.com/4588792ebfcfdd628b76f005fa0e1d7935739139/core/fxcodec/codec/fx_codec_jpx_opj.cpp

Cc: npm@chromium.org
 Issue 763178  has been merged into this issue.
Project Member

Comment 9 by bugdroid1@chromium.org, Sep 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/af57fce1776548b5ac5c1d2bd3c17cd851668502

commit af57fce1776548b5ac5c1d2bd3c17cd851668502
Author: pdfium-deps-roller@chromium.org <pdfium-deps-roller@chromium.org>
Date: Mon Sep 11 22:38:21 2017

Roll src/third_party/pdfium/ 56ec0818c..4588792eb (2 commits)

https://pdfium.googlesource.com/pdfium.git/+log/56ec0818c3ed..4588792ebfcf

$ git log 56ec0818c..4588792eb --date=short --no-merges --format='%ad %ae %s'
2017-09-11 rharrison Address nits missed from original review
2017-09-11 rharrison Add guard against reading more then expected from the page

Created with:
  roll-dep src/third_party/pdfium
BUG= 761005 ,763369


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


TBR=dsinclair@chromium.org

Change-Id: I64e5302f7fcbaba07476aafb429cd193e9af9948
Reviewed-on: https://chromium-review.googlesource.com/660884
Reviewed-by: <pdfium-deps-roller@chromium.org>
Commit-Queue: <pdfium-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#501071}
[modify] https://crrev.com/af57fce1776548b5ac5c1d2bd3c17cd851668502/DEPS

Project Member

Comment 10 by bugdroid1@chromium.org, Sep 14 2017

The following revision refers to this bug:
  https://pdfium.googlesource.com/pdfium/+/3ad06a513bc490065b860a543ffb43eb169769bd

commit 3ad06a513bc490065b860a543ffb43eb169769bd
Author: Ryan Harrison <rharrison@chromium.org>
Date: Thu Sep 14 15:44:21 2017

Revert moving JPX library decode from Init to Decode

Due to some of the size parameters for allocating space in Decode()
depending on the values produced by opj_decode(), this change was
causing misallocation of space, which in turn was causing illegal
reads/writes.

The issue with excessive memory usage that the original CL was trying
to change is less significant than the above mentioned problems, so
reverting this fix and looking for another solution to the
problem. This will re-open bugs  https://crbug.com/754423  and
 https://crbug.com/761005 .

BUG= chromium:764177 , chromium:754423 , chromium:761005 

Change-Id: I1cafac8a8117ec1e3bc32b31196bdec719d46477
Reviewed-on: https://pdfium-review.googlesource.com/13950
Commit-Queue: Ryan Harrison <rharrison@chromium.org>
Reviewed-by: Henrique Nakashima <hnakashima@chromium.org>

[modify] https://crrev.com/3ad06a513bc490065b860a543ffb43eb169769bd/core/fxcodec/codec/fx_codec_jpx_opj.cpp

Status: Assigned (was: Fixed)
Project Member

Comment 12 by bugdroid1@chromium.org, Sep 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d9eaae6913711672807d9ca1b36eaf717dade779

commit d9eaae6913711672807d9ca1b36eaf717dade779
Author: pdfium-deps-roller@chromium.org <pdfium-deps-roller@chromium.org>
Date: Thu Sep 14 17:02:20 2017

Roll src/third_party/pdfium/ 038740c2f..3ad06a513 (1 commit)

https://pdfium.googlesource.com/pdfium.git/+log/038740c2fbd2..3ad06a513bc4

$ git log 038740c2f..3ad06a513 --date=short --no-merges --format='%ad %ae %s'
2017-09-14 rharrison Revert moving JPX library decode from Init to Decode

Created with:
  roll-dep src/third_party/pdfium
BUG= 764177 , 754423 , 761005 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


TBR=dsinclair@chromium.org

Change-Id: If61754f47a2144bf9770b3ced03db0f92c4ef288
Reviewed-on: https://chromium-review.googlesource.com/667497
Reviewed-by: <pdfium-deps-roller@chromium.org>
Commit-Queue: <pdfium-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#501969}
[modify] https://crrev.com/d9eaae6913711672807d9ca1b36eaf717dade779/DEPS

Status: Started (was: Assigned)
Components: Internals>Plugins>PDF
Status: WontFix (was: Started)
This is root caused by the encoded image requiring more then 2GB to inflate, so the fuzzer kills it. If given enough RAM the process will complete, so this is WAI.
Status: Assigned (was: WontFix)
But changing the order of operations causes the test to succeed also, which suggests it is short circuiting out.
Owner: dsinclair@chromium.org
Sending this over to dsinclair to look at, since I have sunk way too much time into this.
For more information, please see https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reference.md.

The link referenced in the description is no longer valid.
Cc: thestig@chromium.org rharrison@chromium.org
 Issue 834561  has been merged into this issue.
Project Member

Comment 20 by ClusterFuzz, Jun 27 2018

Labels: OS-Mac
Owner: ----
Status: Untriaged (was: Assigned)
Setting PDF bugs assigned to me back to untriaged so they can get re-assigned as needed.
Owner: rharrison@chromium.org
Status: Assigned (was: Untriaged)
Ryan, are there still more improvements worth making here?
Labels: ClusterFuzz-Ignore
Status: WontFix (was: Assigned)
This is rooted in there being a very large allocation that is causing an OOM and thus killing the fuzzer. In theory it is possible that the code could short circuit by detecting this condition earlier, but that would likely just lead in the real world to the PDF to failing to load instead of the OOMing. Given my previous efforts into fixing this, I don't think it is worth the effect to detect and handle this condition.
Project Member

Comment 24 by ClusterFuzz, Oct 9

Labels: OS-Windows

Sign in to add a comment