Bisect confirms the OP's suspicions.
427014 (good) - 427017 (bad), 56.0.2900.0
https://chromium.googlesource.com/chromium/src/+log/0500571c..8d67cd7a?pretty=fuller
The only relevant change is r427015 "WebMessaging: Send transferable ArrayBuffers by copy-and-neuter semantics"

Repro note: use Chrome Task manager (Shift-Esc) to view the tab's memory for half a minute - in good builds it'll stay low (e.g. 10MB), in affected builds it'll increase in 50MB steps every 5 seconds. You can speed up the process by replacing 5000 with 1000 in test.html

Able to reproduce the issue on Windows 10, Ubuntu 14.04 and Mac 10.12.2 using reported version #56.0.2924.87 and latest canary #58.0.3003.0.

Bisect Information:
=====================
Good build: 56.0.2899.0	Revision(426989)

Bad Build : 56.0.2900.0	Revision(427219)

Change Log URL: 
https://chromium.googlesource.com/chromium/src/+log/0500571c0a41d977f64653eeaed4809351205a35..99a2756462a76ca70613375784a1a4fa8f3576bb

From the above change log suspecting below change

Review URL: https://codereview.chromium.org/2414333003

nhiroki@ - Could you please check whether this is caused with respect to your change, if not please help us in assigning it to the right owner.

Thanks...!!

Thank you for reporting this.

This is on my TODO list but I haven't had time to take a look at this yet. For now, I'll add an appropriate label for triage.

Started looking into this.

I suspect that this could be caused by an overflow of a message queue, and my patch couldn't be a direct cause. Actually, I can reproduce this with a following modified script that sends an array buffer as non-transferable:

  // c.port1.postMessage(a.buffer, [a.buffer]);
  c.port1.postMessage(a.buffer);

Before my patch, transferable objects passed to postMessage() was just ignored and an actual message didn't contain them, so the message size is relatively small. After my change, transferable objects are copied into a message and the message size becomes bigger, so we hit OOM crash easier.

To avoid the OOM crash, we could...

  - teach a sender port that a target port is already closed and stop it from sending messages anymore, or
  - limit the message queue size and throw an exception when it reaches the limit,

... I'm not sure if these are spec-compatible behavior though.

jbroman@, could you triage this issue?

Additional notes: This also happens when a target port is NOT closed.