This issue looks similar to  bug 779349 , hence merging in to it. Feel free to undupe if it's a different issue.

Thanks!

Unduping.  bug 779349  is fixed, but this one remains open.

This doesn't repro for me. rharrison@ can you try repro'ing? I hit the retest button and the fuzzer says it isn't fixed.

I was able to repro. Bumping up the limit to 2100MB lets it pass, so it is probably just on the threshold of succeeding, which is why it isn't consistent.

I would recommend running a debug build of the fuzzer under callgrind to see where it is spending its time.

The following revision refers to this bug:
  https://pdfium.googlesource.com/pdfium/+/35557c3ce83cc6e26f07cf5d5c520ee6afd39a67

commit 35557c3ce83cc6e26f07cf5d5c520ee6afd39a67
Author: Dan Sinclair <dsinclair@chromium.org>
Date: Thu May 10 14:01:25 2018

Change formcalc javascript method output

This CL changes the formcalc JS conversion code so that method calls no
longer generate exponential amounts of JavaScript.

Previously we'd duplicate the code to call into a method twice. This
would then generate twice once for arrays and once for other types of
object. This CL changes the code to wrap the actual method call into
a JavaScript function which will be used from both the array and
non-array calling code.

For the referenced bug, the generated JS originally needed a buffer of
365meg to generate. With this CL, it needs a buffer of 7.5k.

Bug:  chromium:814840 
Change-Id: Ibb5993fa52b7c13b20b325cf8848a306f82ae014
Reviewed-on: https://pdfium-review.googlesource.com/32312
Reviewed-by: Henrique Nakashima <hnakashima@chromium.org>
Reviewed-by: Ryan Harrison <rharrison@chromium.org>
Commit-Queue: dsinclair <dsinclair@chromium.org>

[modify] https://crrev.com/35557c3ce83cc6e26f07cf5d5c520ee6afd39a67/xfa/fxfa/fm2js/cxfa_fmparser_unittest.cpp
[modify] https://crrev.com/35557c3ce83cc6e26f07cf5d5c520ee6afd39a67/xfa/fxfa/fm2js/cxfa_fmsimpleexpression.cpp

I believe this should be fixed. I was unable to repro the original issue but have greatly reduced the size of the generated javascript.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c89e59df3ffa3736f1a9332493326f755bac1f24

commit c89e59df3ffa3736f1a9332493326f755bac1f24
Author: pdfium-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <pdfium-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Thu May 10 15:44:45 2018

Roll src/third_party/pdfium/ ad18d2fba..35557c3ce (1 commit)

https://pdfium.googlesource.com/pdfium.git/+log/ad18d2fba9dd..35557c3ce83c

$ git log ad18d2fba..35557c3ce --date=short --no-merges --format='%ad %ae %s'
2018-05-10 dsinclair Change formcalc javascript method output

Created with:
  roll-dep src/third_party/pdfium
BUG= chromium:814840 


The AutoRoll server is located here: https://pdfium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=dsinclair@chromium.org

Change-Id: I24e471a028807c98392302493cbbf922a9471f8b
Reviewed-on: https://chromium-review.googlesource.com/1053753
Commit-Queue: pdfium-chromium-autoroll <pdfium-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: pdfium-chromium-autoroll <pdfium-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#557529}
[modify] https://crrev.com/c89e59df3ffa3736f1a9332493326f755bac1f24/DEPS

ClusterFuzz has detected this issue as fixed in range 557528:557544.

Detailed report: https://clusterfuzz.com/testcase?key=4712766297079808

Fuzzer: libFuzzer_pdf_fm2js_fuzzer
Job Type: libfuzzer_chrome_msan
Platform Id: linux

Crash Type: Out-of-memory (exceeds 2048 MB)
Crash Address: 
Crash State:
  pdf_fm2js_fuzzer
  
Sanitizer: memory (MSAN)

Regressed: https://clusterfuzz.com/revisions?job=libfuzzer_chrome_msan&range=395689:395794
Fixed: https://clusterfuzz.com/revisions?job=libfuzzer_chrome_msan&range=557528:557544

Reproducer Testcase: https://clusterfuzz.com/download?testcase_id=4712766297079808

See https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reference.md for more information.

If you suspect that the result above is incorrect, try re-doing that job on the test case report page.