On Fuchsia, some very long or memory-intensive tests (Such as JSONPerfTest.* or MessageLoopPerfTest.*) can trigger an OOM exception which then causes Fuchsia to kill the test process. This may be an error with base_perftests leaking memory, or some part of Fuchsia may be leaking memory. In the case of MessageLoopPerfTest.*, this failure does not occur in isolation, but only when the test is executed as part of the whole suite.
Fuchsia's output during the failure running on physical hardware:
[ RUN ] MessageLoopPerfTest.PostTaskRate/1_Posting_Thread
[448750.842] OOM: 29.7M free (-222.4M) / 8078.4M total
[448750.842] OOM: oom_lowmem(shortfall_bytes=21295104) called
[448750.842] OOM: Process mapped committed bytes:
[448750.843] OOM: proc 1104 8M 'fshost'
[448750.843] OOM: proc 3707 155M 'blobfs:/blob'
[448750.843] OOM: proc 4313 12M 'pkgfs'
[448750.843] OOM: proc 4483 129M 'minfs:/data'
[448750.844] OOM: proc 5798 13M 'netstack'
[448752.332] OOM: proc 13226290 7515M 'base_perftests'
[448752.332] OOM: Finding a job to kill...
[448752.332] OOM: *KILL* job 13225972 ''
[448752.332] OOM: + proc 13225973 run '/system/bin/run'
[448752.332] OOM: = 1 running procs (1 total), 0 jobs
[448752.332] OOM: (next) job 13221309 ''
[448752.332] OOM: (next) job 13220892 'fe80::1a60:24ff:fe89:3c58:40916'
[448752.332] OOM: (next) job 6070 'tcp:22'
2018-06-11 14:43:16,322:WARNING:root:Process exited with status code 255.
2018-06-11 14:43:16,323:INFO:root:Terminating kernel log reader.
2018-06-11 14:43:16,323:INFO:root:Removing package source from device.
This failure can be replicated by executing bin/run_base_perftests in the fuchsia output directory, using the filter file for known flaky or failing tests on Fuchsia.
Comment 1 by stephanstross@google.com
, Jun 11 2018This issue is not only present on Fuchsia, but it appears on Linux computers as well. There is a bug in the base_perftests that causes it to leak upwards of about 7GB of memory on Linux, and perhaps a bit more on Fuchsia, as reported by the /usr/bin/time -v command: Command being timed: "out/default/base_perftests" User time (seconds): 104.42 System time (seconds): 144.55 Percent of CPU this job got: 195% Elapsed (wall clock) time (h:mm:ss or m:ss): 2:07.59 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 6912992 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 3592381 Voluntary context switches: 9727257 Involuntary context switches: 1060 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 1