blink_perf.bindings failing on chromium.perf/Android Nexus5X WebView Perf (3) |
|||||||||||||
Issue descriptionblink_perf.bindings failing on chromium.perf/Android Nexus5X WebView Perf (3) Builders failed on: - Android Nexus5X WebView Perf (3): https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus5X%20WebView%20Perf%20%283%29 first failure: https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus5X%20WebView%20Perf%20%283%29/builds/4854
,
Jun 15 2017
=== Auto-CCing suspected CL author hpayer@chromium.org === Hi hpayer@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : hpayer Commit : 0cef7f7cf70453599887bbffd25b2e216cd63db3 Date : Mon Jun 12 15:58:22 2017 Subject: [heap] Register an idle embedder garbage collection when starting incremental marking through reached marking limit. Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : blink_perf.bindings Metric : structured-clone-long-string-serialize/structured-clone-long-string-serialize.html Change : 12.51% | 7.96036666667 -> 8.95588 Revision Result N chromium@478738 7.96037 +- 0.696973 9 good chromium@478811 7.78479 +- 0.609738 6 good chromium@478829 7.83853 +- 0.49238 6 good chromium@478838 7.80825 +- 0.494526 9 good chromium@478840 7.80717 +- 0.537445 9 good chromium@478840,v8@1624da378e 7.79528 +- 0.686233 9 good chromium@478840,v8@0cef7f7cf7 8.91564 +- 0.290283 4 bad <-- chromium@478840,v8@711073a340 9.21092 +- 0.772552 5 bad chromium@478841 8.8456 +- 0.147893 4 bad chromium@478843 8.99358 +- 1.70408 5 bad chromium@478847 9.05759 +- 0.314576 5 bad chromium@478883 8.95588 +- 0.463845 5 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=structured.clone.long.string.serialize.html blink_perf.bindings Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8976709549399344768 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5650189311803392 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Jun 20 2017
Issue 735113 has been merged into this issue.
,
Jun 22 2017
ulan@, hpayer@ is OOO for several weeks. Can you take a look? Would it be safe to revert this CL?
,
Aug 3 2017
Hi, blink_perf.bindings is failing on most android bots now, mostly with these (and related) stories: structured-clone-long-string-serialize.html structured-clone-long-string-deserialize.html They were re-enabled in crrev.com/b12201a1e1689fff89421ec0152459b9dcfb3622, but immediately started failing; see https://luci-milo.appspot.com/buildbot/chromium.perf/Android%20Nexus5%20Perf/267 for an example of this. Can you look into this again?
,
Aug 11 2017
hablich: any help getting the CL reverted or fixed? The failing stories are consistently * blink_perf.bindings/structured-clone-long-string-deserialize.html * blink_perf.bindings/structured-clone-long-string-serialize.html I'm going to try to do as narrow a disable as possible.
,
Aug 11 2017
Disable CL up for review: https://chromium-review.googlesource.com/c/612503
,
Aug 11 2017
Sorry, this slipped through as I am swamped with higher priority issues recently. Bumping up priority of this issue. Looks like the benchmark is crashing due to OOM: F/libc (27194): Fatal signal 5 (???) at 0x77acda34 (code=4), thread 27207 (CrRendererMain) I/DEBUG ( 178): process stopped due to unexpected signal 5 D/Zygote ( 182): Process 27194 terminated by signal (5) W/cr_ChildProcessConn(27167): onServiceDisconnected (crash or killed by oom): pid=27194 I/ActivityManager( 760): Process org.chromium.chrome:sandboxed_process0 (pid 27194) has died. The benchmark creates 8MB strings in a loop. Hannes' CL affects V8/Blink GC and should improve memory usage. Looks like for some reason the strings are not GC'ed. Investigating.
,
Aug 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a6b22032cf07921ace819db149c3f9b6e81a9dd4 commit a6b22032cf07921ace819db149c3f9b6e81a9dd4 Author: Annie Sullivan <sullivan@chromium.org> Date: Fri Aug 11 21:19:51 2017 Disable failing blink_perf.bindings stories on android. Bug: 733655 Change-Id: Ib8b112d707273459efabc04c94358fbf42df831e Reviewed-on: https://chromium-review.googlesource.com/612503 Commit-Queue: Annie Sullivan <sullivan@chromium.org> Reviewed-by: rnephew <rnephew@chromium.org> Cr-Commit-Position: refs/heads/master@{#493859} [modify] https://crrev.com/a6b22032cf07921ace819db149c3f9b6e81a9dd4/tools/perf/benchmarks/blink_perf.py
,
Aug 14 2017
The crash happens due to Blink memory growing to more than 1GB. Looks like passing kGCCallbackScheduleIdleGarbageCollection flag to Oilpan effectively disable Oilpan GC if there is no idle time (as https://codereview.chromium.org/2937433002 does). I added logging to ThreadState::RunScheduledGC and it shows that GC is skipped because GcState() == kIdleGCScheduled. haraken@, keishi@, shouldn't there be a guard for number of skipped GCs? The following log shows three V8 GCs without an Oilpan GC. 08-14 14:03:25.378 15933 15946 I v8 : [15933:0xef197000] 8674 ms: Mark-sweep 131.9 (136.3) -> 59.9 (64.1) MB, 3.5 / 0.1 ms (+ 9.4 ms in 12 steps since start of marking, biggest step 7.1 ms, walltime since start of marking 130 ms) finalize incremental marking via stack guard GC in old space requested 08-14 14:03:25.409 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.429 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.448 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.451 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.471 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.490 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.490 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.608 15933 15946 I v8 : [15933:0xef197000] 8904 ms: Mark-sweep 147.9 (152.3) -> 75.9 (80.1) MB, 3.5 / 0.1 ms (+ 9.6 ms in 16 steps since start of marking, biggest step 4.5 ms, walltime since start of marking 172 ms) finalize incremental marking via stack guard GC in old space requested 08-14 14:03:25.630 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.631 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.650 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.670 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.672 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.691 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.692 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.738 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.739 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.759 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.759 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.785 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.786 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.810 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.810 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.834 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.834 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.865 15933 15946 I v8 : [15933:0xef197000] 9162 ms: Mark-sweep 155.9 (160.3) -> 75.9 (80.1) MB, 19.2 / 0.1 ms (+ 9.3 ms in 16 steps since start of marking, biggest step 6.7 ms, walltime since start of marking 186 ms) finalize incremental marking via stack guard GC in old space requested 08-14 14:03:25.877 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.897 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.917 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.937 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.940 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.961 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.961 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.982 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:25.982 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:26.002 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:26.002 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:26.027 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:26.027 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:26.047 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:26.047 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:26.069 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:26.069 15933 15946 E chromium: [ERROR:ThreadState.cpp(799)] skipping GC!!! 08-14 14:03:26.087 15933 15946 I v8 : [15933:0xef197000] 9384 ms: Mark-sweep 155.9 (160.3) -> 67.9 (72.1) MB, 4.8 / 1.2 ms (+ 9.6 ms in 15 steps since start of marking, biggest step 4.7 ms, walltime since start of marking 162 ms) finalize incremental marking via stack guard GC in old space requested
,
Aug 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9e850c1815a77040039130efced673f7475ec00b commit 9e850c1815a77040039130efced673f7475ec00b Author: rnephew <rnephew@chromium.org> Date: Mon Aug 14 23:30:00 2017 [Telemetry] Disable failing blink_perf.bindings tests on Android One. Bug: 755137 ,733655 Change-Id: I348d96f240c640b7d5f74ba1dc19b2cdbe9e721b Reviewed-on: https://chromium-review.googlesource.com/614611 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: rnephew <rnephew@chromium.org> Cr-Commit-Position: refs/heads/master@{#494243} [modify] https://crrev.com/9e850c1815a77040039130efced673f7475ec00b/tools/perf/benchmarks/blink_perf.py
,
Aug 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/v8/v8.git/+/cc29092d52acdcb0584cec2ea0930c8dd7e55a4b commit cc29092d52acdcb0584cec2ea0930c8dd7e55a4b Author: Ulan Degenbaev <ulan@chromium.org> Date: Wed Aug 16 15:04:12 2017 [heap] Pass external memory callback flags on external memory pressure. This patch ensures that all code path of the external memory pressure handler pass the kGCCallbackFlagCollectAllExternalMemory flag to the embedder. This is necessary to force Oilpan GC. Bug: chromium:733655 Change-Id: I2e680d1e7f9087d1930d56ead39ef7cf1bc93e56 Reviewed-on: https://chromium-review.googlesource.com/616149 Reviewed-by: Michael Lippautz <mlippautz@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#47378} [modify] https://crrev.com/cc29092d52acdcb0584cec2ea0930c8dd7e55a4b/src/heap/heap.cc
,
Aug 16 2017
I will re-enable benchmarks after #12 rolls in Chromium.
,
Aug 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1ac9ff818032c4f06a3fcba7e0c826293abc9eca commit 1ac9ff818032c4f06a3fcba7e0c826293abc9eca Author: Ulan Degenbaev <ulan@chromium.org> Date: Thu Aug 17 20:53:00 2017 [Telemetry] Enable failing blink_perf.bindings tests on Android One. This reverts commit 9e850c1815a77040039130efced673f7475ec00b. Reason for revert: a fix landed in chromium. Original change's description: > [Telemetry] Disable failing blink_perf.bindings tests on Android One. > > Bug: 755137 ,733655 > Change-Id: I348d96f240c640b7d5f74ba1dc19b2cdbe9e721b > Reviewed-on: https://chromium-review.googlesource.com/614611 > Reviewed-by: Ned Nguyen <nednguyen@google.com> > Commit-Queue: rnephew <rnephew@chromium.org> > Cr-Commit-Position: refs/heads/master@{#494243} Bug: 755137 , 733655 Change-Id: I583325b8350e5ccdd7cced5edfd12635b7d38dca Reviewed-on: https://chromium-review.googlesource.com/619190 Reviewed-by: rnephew <rnephew@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#495314} [modify] https://crrev.com/1ac9ff818032c4f06a3fcba7e0c826293abc9eca/tools/perf/benchmarks/blink_perf.py
,
Aug 19 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/275a99d1fabd0ec329f7723490ca18a8e46dca05 commit 275a99d1fabd0ec329f7723490ca18a8e46dca05 Author: Ulan Degenbaev <ulan@chromium.org> Date: Sat Aug 19 15:44:52 2017 "Enable blink_perf.bindings stories on android." This reverts commit a6b22032cf07921ace819db149c3f9b6e81a9dd4. Reason for revert: a fix landed in chromium. Original change's description: > Disable failing blink_perf.bindings stories on android. > > Bug: 733655 > Change-Id: Ib8b112d707273459efabc04c94358fbf42df831e > Reviewed-on: https://chromium-review.googlesource.com/612503 > Commit-Queue: Annie Sullivan <sullivan@chromium.org> > Reviewed-by: rnephew <rnephew@chromium.org> > Cr-Commit-Position: refs/heads/master@{#493859} Bug: 733655 Change-Id: I1bd83d01d44ec6a31bacbe70c0b40ce0013862ef Reviewed-on: https://chromium-review.googlesource.com/619427 Reviewed-by: rnephew <rnephew@chromium.org> Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Cr-Commit-Position: refs/heads/master@{#495817} [modify] https://crrev.com/275a99d1fabd0ec329f7723490ca18a8e46dca05/tools/perf/benchmarks/blink_perf.py
,
Aug 21 2017
Nexus 5X crash is fixed: https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus5X%20Perf Android One is still failing: https://build.chromium.org/p/chromium.perf/builders/Android%20One%20Perf I'll get Android One device and try to reproduce.
,
Aug 28 2017
Requesting merge to M61 for the CL in comment #12 as it fixes OOM crash on Nexus5 and helps to reduce memory usage.
,
Aug 28 2017
This bug requires manual review: We are only 7 days from stable. Please contact the milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), ketakid@(ChromeOS), govind@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Aug 28 2017
,
Aug 29 2017
hablich@, wdyt about merging this to M61?
,
Aug 31 2017
hablich@, ping? Need to make the call here by tomorrow plz.
,
Aug 31 2017
,
Aug 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/v8/v8.git/+/b5263465d590396c9e1ae00e0fd77dfb085f6776 commit b5263465d590396c9e1ae00e0fd77dfb085f6776 Author: Michael Hablich <hablich@chromium.org> Date: Thu Aug 31 07:05:12 2017 Merged: [heap] Pass external memory callback flags on external memory pressure. Revision: cc29092d52acdcb0584cec2ea0930c8dd7e55a4b R=mlippautz@chromium.org BUG=chromium:733655 LOG=N NOTRY=true NOPRESUBMIT=true NOTREECHECKS=true Change-Id: Ib9f9ee95a73ae4e197fbb85436edabc370c6bb1b Reviewed-on: https://chromium-review.googlesource.com/645087 Reviewed-by: Michael Lippautz <mlippautz@chromium.org> Cr-Commit-Position: refs/branch-heads/6.1@{#66} Cr-Branched-From: 1bf2e10ddb194d4c2871a87a4732613419de892d-refs/heads/6.1.534@{#1} Cr-Branched-From: e825c4318eb2065ffdf9044aa6a5278635c36427-refs/heads/master@{#46746} [modify] https://crrev.com/b5263465d590396c9e1ae00e0fd77dfb085f6776/src/heap/heap.cc
,
Aug 31 2017
|
|||||||||||||
►
Sign in to add a comment |
|||||||||||||
Comment 1 by 42576172...@developer.gserviceaccount.com
, Jun 15 2017