All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=769548

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=30d4617442d7c1569523010ba829226af6cbad8e4580bd1d827472d28d3e071f


Bot(s) for this bug's original alert(s):

android-nexus5X

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8967254596743270896

=== Auto-CCing suspected CL author agrieve@chromium.org ===

Hi agrieve@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Andrew Grieve
  Commit : 8a218af5c68598645675d76980984777c8429bb3
  Date   : Tue Sep 26 04:31:21 2017
  Subject: Android: Roll ndk to add CHROMIUM_CXX_TWEAK_INLINES

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : rasterize_and_record_micro.top_25
  Metric       : record_time/file___static_top_25_amazon.html
  Change       : 4.94% | 0.111333333333 -> 0.116833333333

Revision             Result                       N
chromium@504259      0.111333 +- 0.0011547        6      good
chromium@504274      0.111667 +- 0.0011547        6      good
chromium@504282      0.111833 +- 0.000912871      6      good
chromium@504283      0.115833 +- 0.000912871      6      bad       <--
chromium@504284      0.116167 +- 0.000912871      6      bad
chromium@504286      0.116333 +- 0.00182574       6      bad
chromium@504289      0.117 +- 0.00282843          6      bad
chromium@504319      0.116833 +- 0.00168325       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=file...static.top.25.amazon.html rasterize_and_record_micro.top_25

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8967254596743270896


For feedback, file a bug with component Speed>Bisection

 Issue 771203  has been merged into this issue.

 Issue 771253  has been merged into this issue.

Andrew: Is there anything that can be done about the regressions caused by the NDK roll?

This was caused not by a an NDK roll, but by adding some __noinline__ annotations to std::string and std::vector.

The patch saved almost 1MB of native code size, so overall I think there's a very strong argument to say that it's worth a few small regressions. 

It's certainly possible that these microbenchmarks could be improved by looking at their use of vector and string, but I don't think the optimizations would be different from what you'd find from profiling anyways. 

Do you know who the owners for these benchmarks are? I have no idea if this amount of slowdown merits a deep-dive into them.

Since Blink doesn't use std::vector/string much, I'd guess these uses are within skia or cc. +fmalita who knows the Skia side better.

Historically, Skia has steered clear of std::{vector,string} also.  New code (last year or so?) has started using them though.

From a quick grep, the most significant user is the SKSL compiler - but that wouldn't explain a *record time* regression.  I suspect this is unrelated to Skia.

Agreed it's unlikely to be Skia.  We don't use std::vector much, and not in any performance critical place I can see but sksl.

I'm also agreed on the sentiment in Comment 7... 1MB of code on mobile is a big deal.  In fact, the only reason we're kind of considering ramping up our use of std::vector in Skia might be so that the linker can dedup our std::vector instances with everyone else's, rather than bloating code size with workalike SkT[D]Arrays and such.

Thanks for the analysis! vmpstr, assigning to you as owner of rasterize_and_record: are you okay with accepting this regression since we save 1MB of native code size?

Please WontFix this bug if you think this is an acceptable tradeoff.

I think the tradeoff is absolutely worth it. It's really nice savings on the code size and a fairly small time regression.