Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8968750539121468048

Confirmed first failures actually are:
https://luci-milo.appspot.com/buildbot/chromium.perf/Android%20Nexus6%20WebView%20Perf/231
https://luci-milo.appspot.com/buildbot/chromium.perf/Android%20Nexus5X%20WebView%20Perf/246

With overlapping CL range:
http://test-results.appspot.com/revision_range?start=500324&end=500358

=== BISECT JOB RESULTS ===
Test failure found with culprit

Suspected Commit
  Author : xunjieli
  Commit : 724bb776bcdea642a5bc9cc20fe14bb0282b4900
  Date   : Wed Sep 06 14:29:20 2017
  Subject: [Telemetry] Use --ignore-certificate-errors-spki-list to bypass cert errors

Bisect Details
  Configuration: android_webview_nexus6_aosp_perf_bisect
  Benchmark    : power.typical_10_mobile
  Metric       : application_energy_consumption_mwh/http___m.facebook.com_barackobama

Revision                                 Exit Code      N
chromium@500323                          0 +- N/A       5      good
chromium@500341                          0 +- N/A       5      good
chromium@500344                          0 +- N/A       5      good
chromium@500344,catapult@c5a474f50f      0 +- N/A       5      good
chromium@500344,catapult@724bb776bc      1 +- N/A       5      bad       <--
chromium@500344,catapult@80c47581b1      1 +- N/A       5      bad
chromium@500344,catapult@4f611510b9      1 +- N/A       5      bad
chromium@500344,catapult@442f46edf7      1 +- N/A       5      bad
chromium@500345                          1 +- N/A       5      bad
chromium@500346                          1 +- N/A       5      bad
chromium@500350                          1 +- N/A       5      bad
chromium@500358                          1 +- N/A       5      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...m.facebook.com.barackobama power.typical_10_mobile

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8968750539121468048


For feedback, file a bug with component Speed>Bisection

If WebView doesn't load, then we have a bug. Deduplicate this one.

perezju@: Do you know if WebView has a "--user-data-dir" flag ? 

I think we should revert the CL now, since this is creating a big hole in our perf measurements for webview and can make some regressions go undetected.

With respect to your specific question, I see we are passing:

  --user-data-dir=/data/data/org.chromium.webview_shell/

which is actually probably wrong. It's using the package name of the webview shell, rather than the webview itself.

Full command line flags are visible on the logs, e.g.:
https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus6_WebView_Perf%2F231%2F%2B%2Frecipes%2Fsteps%2Fpower.typical_10_mobile_on_Android%2F0%2Fstdout

Instead of reverting the CL, I think we should let add some logic that does:
"if browser is not webview: 
   Use  --ignore-certificate-errors-spki-list
else:
   ..
"

Then reassign the bug about making sure --ignore-certificate-errors-spki-list integration work on Webview to webview folks.

I agree. Let's not revert that CL.
Please use --ignore-certificate-errors for webview 

Using the webview shell's data directory would be correct since that's the app which is actually running. Nothing ever gets to write to WebView's own data directory because nothing runs as WebView's UID.

..however I don't think WebView actually accepts --user-data-dir in the first place; it's probably ignored.

--user-data-dir is a misdirect for the case of WebView.

https://cs.chromium.org/chromium/src/content/public/browser/ignore_errors_cert_verifier.h?rcl=769d0e3570a4f836cff9821c87afbfec7c87c4e5&l=27 is the code to set it up. //chrome does it, as does content_shell, but webview does not appear to call the //content API.

Because --user-data-dir isn't necessarily a //content flag, the IgnoreErrorsCertVerifier lets you pass the flag as an incoming parameter as the (other) command-line flag to expect in order to safely ignore errors.

That is, I believe the root cause is WebView does not call into //content to setup that flag

The code in WebView is at android_webview/browser/net/aw_url_request_context_getter.cc. We need to change it to set up IgnoreErrorsCertVerifier like how io_thread.cc does it.

The breakage appears to be fixed now thanks to the workaround in https://codereview.chromium.org/3012143003/

Let's follow up on  issue 753948  on how to implement the --ignore-certificate-errors-spki-list for WebView.