New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 913861 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Today
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug



Sign in to add a comment

Chromium crash when writing a large emscripten memfs file

Reported by b...@debian.org, Dec 11

Issue description

UserAgent: Mozilla/5.0 (X11; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0

Steps to reproduce the problem:
Chromium 70 displays a "Aw, Snap!" when writing a file >= 108MB.

#include <stdio.h>
#include <stdlib.h>

#define BUFSIZE 1024*1024

int main(void) {
  FILE* f = fopen("/archive.rpa", "w");
  unsigned char buf[BUFSIZE] = "";
  for (int i = 0; i < BUFSIZE; i++) {
    buf[i] = rand() % 256;
  }
  for (int i = 0; i < 150; i++) {  // fails at 108 for me
    fwrite(buf, BUFSIZE, 1, f);
  }
  printf("Wrote file.\n");
}

This is consistent whatever the memory options I provide:

$ emcc ../testlarge.c -o testlarge.html -s TOTAL_MEMORY=1GB -s ALLOW_MEMORY_GROWTH=1

Nothing in the (disconnected) console, but a bit of information on the command line:

<--- Last few GCs --->
arking 47 ms) (average mu = 0.915, current mu = 0.978) finalize incremental[70:0x560853e63640]     1612 ms: Mark-sweep 1721.3 (1725.4) -> 1574.0 (1578.1) MB, 5.0 / 0.0 ms  (+ 0.8 ms in 4 steps since start of marking, biggest step 0.6 ms, walltime since start of marking 1216 ms) (average mu = 0.993, current mu = 0.996) allocation[70:0x560853e63640]     1706 ms: Mark-sweep 1574.0 (1578.1) -> 577.5 (581.6) MB, 94.7 / 0.0 ms  (average mu = 0.876, current mu = 0.000) allocation failure GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x56084ddc86ee]
Security context: 0x06953a021079 <String[21]: http://localhost:8001>
    1: write [0x3225aa7a43a1] [http://localhost:8001/testlarge.js:~2400] [pc=0x30bbd6ae1b21](this=0x3225aa7a1d99 <Object map = 0x273c6e0b3851>,0x07eb43631a91 <JSObject>,0x3225aa7a1d51 <Int8Array map = 0x273c6e0a4f81>,6640,1048576,112197632,0x2ffb7b2025b1 <undefined>)
    2: write [0x3225aa7a1de1] [http://localhost:8001/testlarge.js:4...

What is the expected behavior?
No crash.
File written properly to memfs.

What went wrong?
Crash.

Did this work before? N/A 

Chrome version: 70.0.3538.67 (Developer Build) built on Debian buster/sid, running on Debian buster/sid (64-bit)  Channel: n/a
OS Version: Debian buster
Flash Version: 

Not specific to my system as it was initially reported by an end user of my web app.
 
Labels: Needs-Triage-M70
Components: -Blink Blink>JavaScript>WebAssembly
Cc: swarnasree.mukkala@chromium.org
Labels: Triaged-ET Needs-Feedback
Thanks for filing the issue...

@reporter: Could you please provide a sample file or URL that reproduces the issue so that it would be really helpful in triaging the issue and also provide the crash ID by navigating "chrome://crashes".
Note that this happens in both WASM and ASMJS modes.

chrome://crashes/ says "Crash reporting is not available in Chromium."

Attached are the generated files for the above code.

testlarge.zip
258 KB Download
Project Member

Comment 5 by sheriffbot@chromium.org, Dec 13

Labels: -Needs-Feedback
Thank you for providing more feedback. Adding the requester to the cc list.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: Available (was: Unconfirmed)

Comment 7 by jakub.tu...@gmail.com, Jan 18 (4 days ago)

Is there any progress on this?

Comment 8 by adamk@chromium.org, Jan 18 (4 days ago)

Cc: yangguo@chromium.org
Components: -Blink>JavaScript>WebAssembly Blink>JavaScript>Runtime
Owner: verwa...@chromium.org
Reproduced locally (http://crash/b8ab008879ce2607), and this is an attempt to grow a v8-internal structure beyond FixedArray::kMaxLength (similar to issue 919549).

So this doesn't really have anything to do with WebAssembly, re-tagging appropriately. Given that this means we're trying to allocate a FixedArray greater than 1G in size, I think it's probably not something we're likely to be able to support, but it would be nice if we could throw an exception instead of crashing.

verwaest, is there anything about these "invalid array length" crashes on our roadmap?

Comment 9 by adamk@chromium.org, Jan 18 (4 days ago)

Status: Assigned (was: Available)

Comment 10 by adamk@chromium.org, Jan 18 (4 days ago)

Cc: adamk@chromium.org

Comment 11 by b...@debian.org, Jan 18 (4 days ago)

Hi,

Yeah this isn't specific to WASM, as I mentioned this also occurs with ASMJS.

1GB sounds very large for handling a 100-110MB file in-memory. Where do you think things are bloating?

I'm worried about reading "probably not something we're likely to be able to support".  Please note that the provided code is a reasonable use case for emscripten and works fine with Firefox.

Comment 12 by adamk@chromium.org, Jan 18 (4 days ago)

Cc: azakai@chromium.org
Thanks for the feedback. CCing azakai who may have thoughts, from the Emscripten side, about how a 100M file could result in attempting to allocate a 1G array.

Comment 13 by azakai@google.com, Yesterday (34 hours ago)

Looks like the code here appends multiple times to a file, so we end up using a JS array (that we can append to, to avoid creating a new typed array each time). The crash happens in a loop like this:

  while (node.contents.length < newCapacity) node.contents.push(0);

where newCapacity is around 108MB.

Reading some v8 code, I think I see that a FixedArray uses 8 bytes per element (on a 64-bit system). So a 1GB limit on the stored size means a little above 100MB for the length of the JS array, which is around what we're seeing here. And for a typed array, we'd use a Uint8Array instead, which takes 1/8th the storage.

Emscripten has a MEMFS_APPEND_TO_TYPED_ARRAYS option which will avoid JS arrays and "append" to typed arrays, that is, create a copy on each append operation - which can be slower. Perhaps we should turn that option on by default, as it's better to be slow than to possibly crash?

Comment 14 by b...@debian.org, Yesterday (32 hours ago)

FYI the initial use case is downloading and decompressing a .zip file in memfs.
The attached test case basically reproduces a 'unzip'.

Comment 15 by azakai@google.com, Today (10 hours ago)

Status: WontFix (was: Assigned)
@beuc yeah, I agree this is something fairly basic and we should get working. We should figure this out on the emscripten side ( https://github.com/emscripten-core/emscripten/issues/7630 ), given the discussion above, closing here.

Comment 16 by b...@debian.org, Today (10 hours ago)

How about we keep this open and still convert the crash to an exception, as adamk suggested in comment 8?
(This would avoid spending a month identifying the issue :))

Comment 17 by adamk@chromium.org, Today (8 hours ago)

Apologies for the delay in triage, I think it just got lost in the shuffle over the holidays. Once I took a look, the attached test case was sufficient for me to triage.

Making this an exception would be tricky architecturally, and wouldn't really help the root problem very much (that this is putting bytes into an 8-byte-per-slot array). So I'm inclined to leave this one as WontFix.

Comment 18 by b...@debian.org, Today (8 hours ago)

I expected Chromium to give some explicit info when a size limitation is reached, rather than crash, but as you wish.

Comment 19 by adamk@chromium.org, Today (8 hours ago)

I think that's a fair point; I misread your "spending a month" thing as being about Chromium's bug triage process.

That said, the problem running up against FixedArray::kMaxSize is broader than this particular case. So I'd rather track that elsewhere; I've added a note to our (sadly not currently public-facing) backlog to take up as a possible future project.

Thanks for the initial report and this dialogue.

Sign in to add a comment