New issue
Advanced search Search tips

Issue 836550 link

Starred by 2 users
Status: Assigned
Owner:
g...@chromium.org
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Task



Sign in to add a comment

Grouping bug: Performance reports for ThinLTO optimizations for Chrome on Android

Project Member Reported by g...@chromium.org, Apr 25 2018 
Parent bug:  issue 807147 .

I just don't want to spam it with pinpoint.

The default import threshold appears to be too much of a binary size regression. -O2 with a threshold of 10 is binary size neutral, -O2 with a threshold of 5 is binary size minimal, and a threshold of 25 seems to be a decent increase. Will try a few benchmarks on each to get a feel for what the perf implications are.

(I'll also try -O3 with 5, but...)

For exact binary size measurements, please see https://docs.google.com/spreadsheets/d/1CF3b1q7jwbj6rYdVRQq2rDRYzdYh-Mv6jW1rO6JDnoI/edit?usp=sharing
Showing comments 49 - 148 of 148 Older ›
Project Member

Comment 50 by 42576172...@developer.gserviceaccount.com, May 31 2018 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/14de8446240000

Comment 51 by g...@chromium.org, May 31 2018 

^^ That's a lie, by the way.

I assumed that rerunning the -O2+threshold=5 run would restore my -O2+threshold=5 patchset (or, well, use it). It appears to be using my -O3+threshold=5 patchset.

This is a sensible default, but still something I tripped over. :)

Actual -O2+threshold=5 run coming soon...
Project Member

Comment 54 by 42576172...@developer.gserviceaccount.com, Jun 1 2018 

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/14819506240000

All of the attempts failed. See the individual attempts for details on each error.
Project Member

Comment 57 by 42576172...@developer.gserviceaccount.com, Jun 3 2018 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/14b019b6240000

Comment 58 by g...@chromium.org, Jun 4 2018 

...And I can't repro these build failures on my machine. Trying once more before I ask for help, since this config (with 2 cflag changes) worked perfectly a few days ago.
Project Member

Comment 60 by 42576172...@developer.gserviceaccount.com, Jun 4 2018 

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/11dc394e240000

All of the attempts failed. See the individual attempts for details on each error.
Project Member

Comment 62 by 42576172...@developer.gserviceaccount.com, Jun 6 2018 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/12c4f3fe240000

Comment 63 by g...@chromium.org, Jun 7 2018 

Ohh joy. Looks like there's been a fair number of updates to thread_times.key_silk_cases since I did the baseline runs. While the notebook can recover, we're losing some results on our speedup charts.

Guess I get to rerun all the other configurations for that again. Joy.

Starting with O2Threshold25.
Project Member

Comment 65 by 42576172...@developer.gserviceaccount.com, Jun 7 2018 

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/148e0d41240000

All of the runs failed. The most common error (20/20 runs) was:
BuildError: Build failed: BUILD_FAILURE
Project Member

Comment 67 by 42576172...@developer.gserviceaccount.com, Jun 7 2018 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/11f36b01240000

Comment 68 by g...@chromium.org, Jun 7 2018 

vv O2Threshold10 vv
Project Member

Comment 70 by 42576172...@developer.gserviceaccount.com, Jun 8 2018 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/17978b71240000

Comment 71 Deleted

Comment 72 by g...@chromium.org, Jun 8 2018 

vv O3Threshold5 vv
Project Member

Comment 74 by 42576172...@developer.gserviceaccount.com, Jun 10 2018 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/118851d1240000

Comment 75 by g...@chromium.org, Jun 10 2018 

vv O0 vv
Project Member

Comment 77 by 42576172...@developer.gserviceaccount.com, Jun 10 2018 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/129c1769240000

Comment 78 by g...@chromium.org, Jul 25 

Was requested to run a few additional benchmarks:

- system_health.common_mobile
- system_health.memory_mobile
- experimental.startup.android.coldish
- speedometer2

I'll only test threshold=5+-O2 and -O0.

-O2 first :)
Project Member

Comment 80 by 42576172...@developer.gserviceaccount.com, Jul 26 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/14843feda40000

Comment 81 by g...@chromium.org, Jul 27 

Starting over, since ThinLTO -O0 was landed on master (yay!) since my last run.
Project Member

Comment 86 by 42576172...@developer.gserviceaccount.com, Jul 27 

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/127f478ba40000

Buildbucket says the build completed successfully, but Pinpoint can't find the isolate hash.
Project Member

Comment 88 by 42576172...@developer.gserviceaccount.com, Jul 28 

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/13e8586ba40000

The swarming task expired. The bots are likely overloaded, dead, or misconfigured.
Project Member

Comment 89 by 42576172...@developer.gserviceaccount.com, Jul 28 

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/17eb0b3ba40000

The swarming task expired. The bots are likely overloaded, dead, or misconfigured.
Project Member

Comment 90 by 42576172...@developer.gserviceaccount.com, Jul 28 

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/125e7617a40000

The swarming task expired. The bots are likely overloaded, dead, or misconfigured.
Project Member

Comment 91 by 42576172...@developer.gserviceaccount.com, Jul 28 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/118131e3a40000

Comment 92 by g...@chromium.org, Jul 28 

Looks like all failures. Kicking them off again...

(Also, the ThinLTO -O0 CL was reverted. So, if these go off of ToT, comparisons will again be versus non-ThinLTO)
Project Member

Comment 97 by 42576172...@developer.gserviceaccount.com, Jul 30 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/16afb587a40000
Project Member

Comment 98 by 42576172...@developer.gserviceaccount.com, Jul 30 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/125430c3a40000
Project Member

Comment 99 by 42576172...@developer.gserviceaccount.com, Jul 30 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/1442c273a40000
Project Member

Comment 100 by 42576172...@developer.gserviceaccount.com, Jul 30 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/15f743dda40000

Comment 101 by g...@chromium.org, Jul 30 

Failures look somewhat consistent, and I see some "chrome crashed" failures, so this may (?) be due to the first patch in the series. Dropping ThinLTO for a moment to see if we still see similar failures.

Comment 103 by g...@chromium.org, Jul 30 

(Also trying ThinLTO on a 5X, since 5X is shinier and crashes on the 5 could mean "we're now generating code that's too new for the 5," which would be a pretty nasty bug in my patch :) )
Project Member

Comment 105 by 42576172...@developer.gserviceaccount.com, Jul 30 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/12860433a40000

Comment 106 by g...@chromium.org, Jul 31 

^^ Infra failures on building ToT for the ThinLTO+-O2'ed 5X. Trying again
Project Member

Comment 108 by 42576172...@developer.gserviceaccount.com, Jul 31 

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/14c0edb7a40000

Buildbucket says the build completed successfully, but Pinpoint can't find the isolate hash.
Project Member

Comment 110 by 42576172...@developer.gserviceaccount.com, Jul 31 

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/14a9064da40000

Buildbucket says the build completed successfully, but Pinpoint can't find the isolate hash.
Project Member

Comment 111 by 42576172...@developer.gserviceaccount.com, Jul 31 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/11190e4ba40000

Comment 112 by g...@chromium.org, Jul 31 

The retries will continue until greenness improves.
Project Member

Comment 114 by 42576172...@developer.gserviceaccount.com, Aug 2 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/14f12c1fa40000

Comment 116 by g...@chromium.org, Aug 2 

...OK, so the nexus5 appears to be happy with the zlib change.

So either ThinLTO itself is causing unhappiness, or the crashes above are lab flakiness.

I'm going to hope for the latter, since the former sounds harder to debug.
Project Member

Comment 117 by 42576172...@developer.gserviceaccount.com, Aug 2 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/1592bd8fa40000

Comment 118 by g...@chromium.org, Aug 2 

Ooh, a SIGILL.

It appears to be in `Cr_z_crc32`, and the instruction we're executing is a `udf #254`. Exciting times :)

Comment 119 by g...@chromium.org, Aug 3 

Looks like (though I haven't verified) LLVM was optimizing the call to `Cr_z_arm_check_features` to the udf, since we define `Cr_z_arm_check_features` to have the `arm_aapcs_vfpcc` calling convention. Callsites of it use the vanilla C calling convention, though. This mismatch = undefined behavior, so it makes sense that this was optimized to undef.

I suspect that this is the fault of my zlib patch. Non-ThinLTO builds (and, I imagine, ThinLTO -O0 builds) don't have the visibility to do this optimization, so it makes sense that the patch appeared to work when it was tested in isolation. Tweaking to find a fix now...

Comment 120 by g...@chromium.org, Aug 3 

Yup, it was my zlib patch. Passing -mfloat-abi=hard gave us a custom calling convention.

I can't figure out how to silence some backend complaints (two of them. Neither of which are actual warnings/errors), so I added a FIXME for those, which should be just as good as a fix.

Trying once more.
Project Member

Comment 122 by 42576172...@developer.gserviceaccount.com, Aug 3 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/16c7ef57a40000

Comment 123 by g...@chromium.org, Aug 3 

ᕕ( ᐛ )ᕗ As a reminder, ^ was speedometer2 on -O2

-O2 runs for the following coming up:
- system_health.common_mobile
- system_health.memory_mobile
- experimental.startup.android.coldish
Project Member

Comment 127 by 42576172...@developer.gserviceaccount.com, Aug 3 

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/16fa72afa40000

Buildbucket says the build completed successfully, but Pinpoint can't find the isolate hash.
Project Member

Comment 129 by 42576172...@developer.gserviceaccount.com, Aug 3 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/12b49bdfa40000
Project Member

Comment 130 by 42576172...@developer.gserviceaccount.com, Aug 3 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/14d0c0f7a40000
Project Member

Comment 131 by 42576172...@developer.gserviceaccount.com, Aug 3 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/14ced86fa40000

Comment 132 by g...@chromium.org, Aug 4 

Excellent. Going to swap to -O0 now
Project Member

Comment 137 by 42576172...@developer.gserviceaccount.com, Aug 4 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/14848a48640000
Project Member

Comment 138 by 42576172...@developer.gserviceaccount.com, Aug 4 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/14ae7eb8640000
Project Member

Comment 139 by 42576172...@developer.gserviceaccount.com, Aug 4 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/14de923fa40000
Project Member

Comment 140 by 42576172...@developer.gserviceaccount.com, Aug 4 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/11c62fafa40000

Comment 143 by g...@chromium.org, Aug 6 

Hrm. The memory benchmark results are interesting. Rerunning with -O2 and -O0 (in that order) to get a feel for how much noise there is in them.

If this behavior persists, my first suspect is going to be "ThinLTO is making the orderfile less effective," which is fixed by landing ThinLTO and waiting for the orderfile to update. :)
Project Member

Comment 144 by 42576172...@developer.gserviceaccount.com, Aug 7 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/1297e0bfa40000
Project Member

Comment 145 by 42576172...@developer.gserviceaccount.com, Aug 7 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/11f6d820640000

Comment 147 by g...@chromium.org, Aug 7 

And one more run of the -O2 common_mobile to see how much noise is in some of the metrics...
Project Member

Comment 148 by 42576172...@developer.gserviceaccount.com, Aug 8 

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/17a85420640000
Showing comments 49 - 148 of 148 Older ›
►

Sign in to add a comment