Can you make sure "--ignore-certificate-errors-spki-list" is set in Chrome's startup command line flags?

I checked in chrome://version that "--ignore-certificate-errors-spki-list" was set in command line (shown as attachment). 

I just updated Telemetry to the latest and executed 'run_benchmark', did I miss anything in setting up the environment?

Deleted: Screenshot 2017-09-07 at 5.51.54 PM.png 393 KB

	Screenshot 2017-09-07 at 5.51.54 PM.png 393 KB View Download

--ignore-certificate-errors-spki-list should work for ChromeOS.
What version of Chrome are you using? 
This is a new flag. It might be because you are using an older version of Chrome.

The Chrome version I'm using is 62.0.3198.0, ChromeOS version is 9887.0.0 in dev-channel, does the command line work for this version?

If more data and support are needed, please let me know. I'd like to help analyze and fix this issue.  

@Comment 3: I don't see --user-data-dir (which has to be set in order for --ignore-certificate-errors-spki-list to be set)

@Comment 4: Can you check the Chrome log for the run? You can reasonably expect any useful debugging will be there :)

Thanks Ryan. 

lingyun.cai@:  Can you add a --user-data-dir flag when starting Chrome? 
I did it for Android in Telemetry in https://codereview.chromium.org/3002243002/.
I am not sure how ChromeOS sets the flags.

+ achuith@

 Issue 763489  has been merged into this issue.

testWebPageReplay is failing as a result of this bug.

I tried testWebPageReplay which illustrates the bug on TOT chrome with TOT catapult, and it fails, so this is not a staleness issue.

--user-data-dir is always specified for chromeos (you can see it in the screenshot of comment #2).

I see the unsupported command line flag banner, and this in the chrome log file:
[31123:31152:0908/165203.764887:ERROR:cert_verify_proc_nss.cc(922)] CERT_PKIXVerifyCert for www.example.com failed err=-8172

The banner appears to be a red herring, looks like we always print this message for this flag:
https://cs.chromium.org/chromium/src/chrome/browser/ui/startup/bad_flags_prompt.cc?l=107

It also looks like we actually do create the IgnoreErrorsCertVerifier in this case, we are getting to this line:
https://cs.chromium.org/chromium/src/content/browser/ssl/ignore_errors_cert_verifier.cc?l=47

I can't immediately tell why the page is blocked on chromeos.

I'm happy to assist with debugging this, but this should probably be owned by someone more familiar with ssl code. Jochen? Ryan?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c06c7130fbfc6ed8d46b9fff995367232f5a86e3

commit c06c7130fbfc6ed8d46b9fff995367232f5a86e3
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Sat Sep 09 02:01:42 2017

Roll src/third_party/catapult/ 7a80ad340..e3fe21f50 (6 commits)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/7a80ad3406d1..e3fe21f5029c

$ git log 7a80ad340..e3fe21f50 --date=short --no-merges --format='%ad %ae %s'
2017-09-08 dtu [pinpoint] Remove JobResults object.
2017-09-08 dtu [pinpoint] Fix mistake in posting bug comment.
2017-09-08 achuith Disable testWebPageReplay on chromeos.
2017-09-08 dtu [pinpoint] Collapse 'state' into top-level in AsDict().
2017-09-08 chiniforooshan Use trace streams if input is too long
2017-09-08 dtu [pinpoint] Add "pending" status to execution status boxes.

Created with:
  roll-dep src/third_party/catapult
BUG= 762819 , 731047 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I39bfa117e930f9ea13737f80053f5697b190a673
Reviewed-on: https://chromium-review.googlesource.com/658490
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#500779}
[modify] https://crrev.com/c06c7130fbfc6ed8d46b9fff995367232f5a86e3/DEPS

Thanks for all the comments. I did some debugging further.
On ChromeOS, the following two Verify() were called in order:
MultiThreadedCertVerifier::Verify()
IgnoreErrorsCertVerifier::Verify()

The error achuith@ mentioned in C#10 
[31123:31152:0908/165203.764887:ERROR:cert_verify_proc_nss.cc(922)] CERT_PKIXVerifyCert for www.example.com failed err=-8172
was from MultiThreadedCertVerifier::Verify()

But on Linux, which airbnb.com would be loaded successfully, only IgnoreErrorsCertVerifier::Verify() was called.

I'm not sure if the difference in ChromeOS is reasonable? I'll break-down deeper and come up w/ more details later.

Some more update.
The cert_verifier_that calls MultiThreadedCerfVerifier::Verify() on ChromeOS is initiated in ProfileIOData:Init() here:
https://cs.chromium.org/chromium/src/chrome/browser/profiles/profile_io_data.cc?type=cs&q=MakeUnique%3Cnet::CachingCertVerifier%3E&l=1115

To make --ignore-certificate-errors-spki-list work on ChromeOS, could we possibly also add 
cert_verifier_ = cotnent::IgnoreErrorsCertVerifier::MaybeWrapCertVerifier(command_line, switches::kUserDataDir, std::move(cert_verifier_))
before 
builder->set_shared_cert_verifier(cert_verifier_.get()) ?

Re #14. That sounds good to me.

+ martinkr@
Martin, could you advise?

That's the right place in my book :)

This reproduces on chromeos-chrome on linux. You can build chrome with gn arg chromeos=1, and testWebPageReplay will fail with the same error.

I tried the suggestion in comment #14, and it doesn't work:
https://chromium-review.googlesource.com/c/chromium/src/+/664358

I can see this new code being invoked, and the relevant IgnoreErrorsCertVerifier object being created, but same error.

Thanks for the comments, I uploaded a patch in:
https://chromium-review.googlesource.com/c/chromium/src/+/664380
PTAL.

achuith@ thanks for verification, but I think your modification might not be the right place. I've tried my patch on ChromeOS, it worked for page_cycler_v2. Could you also review and verify my patch?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3124f840cc31c041bd3e320023b791a6be86e43b

commit 3124f840cc31c041bd3e320023b791a6be86e43b
Author: lycai <lingyun.cai@intel.com>
Date: Wed Sep 13 21:32:10 2017

Make --ignore-certificate-errors-spki-list work on ChromeOS

Bug:  762819 
Change-Id: I75aa6ff470b4b9d8054bfa2070a0b9142511c8e1
Reviewed-on: https://chromium-review.googlesource.com/664380
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Reviewed-by: Ryan Sleevi <rsleevi@chromium.org>
Reviewed-by: Matt Menke <mmenke@chromium.org>
Commit-Queue: Achuith Bhandarkar <achuith@chromium.org>
Cr-Commit-Position: refs/heads/master@{#501755}
[modify] https://crrev.com/3124f840cc31c041bd3e320023b791a6be86e43b/AUTHORS
[modify] https://crrev.com/3124f840cc31c041bd3e320023b791a6be86e43b/chrome/browser/profiles/profile_io_data.cc

Believe this is fixed.

Did you re-enable testWebPageReplay on chromeos?

https://codereview.chromium.org/3011253002/

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b9df3441dc4c014462b6d0a29d2ade02abdc9a83

commit b9df3441dc4c014462b6d0a29d2ade02abdc9a83
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Thu Sep 14 02:20:45 2017

Roll src/third_party/catapult/ c4a8d9578..444bcc528 (5 commits)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/c4a8d9578ef1..444bcc528f7b

$ git log c4a8d9578..444bcc528 --date=short --no-merges --format='%ad %ae %s'
2017-09-13 achuith Revert of Disable testWebPageReplay on chromeos. (patchset #1 id:1 of https://codereview.chromium.org/3012103003/ )
2017-09-13 simonhatch Dashboard - Fix kicking off Pinpoint functional bisects.
2017-09-13 benjhayden Reland Plumb trace canonicalUrl through TelemetryInfo.
2017-09-13 benjhayden Plumb device id through Histogram Pipeline.
2017-09-13 rnephew [BattOr] Improve logging when start or stop tracing time is not present.

Created with:
  roll-dep src/third_party/catapult
BUG= 762819 , 763375 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I92e768e4c369c1cf8719686403008ab4357fac28
Reviewed-on: https://chromium-review.googlesource.com/666009
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#501845}
[modify] https://crrev.com/b9df3441dc4c014462b6d0a29d2ade02abdc9a83/DEPS

Issue 778990 has been merged into this issue.