Use --ignore-certificate-errors-spki-list in Telemetry |
||||||||||||
Issue descriptionFor Android perf bots, we'd like to install a test root certificate authority(CA) during device setup. Currently, certs are signed by WebPageReplay and not chained to a trusted local anchor. We use --ignore-certificate-errors to bypass certificate errors. However that flag isn't good for perf testing because we will end up making two SSL socket connections for every HTTPs request and skip disk cache. With the new WebPageReplay (https://codereview.chromium.org/2997573003/), a test root certificate authority can be installed via "/path/to/wprgo_binary installroot --android_device_id=foo --adb_binary_path=/path/to/adb" , and removed via the "removeroot" option. We need to find a way to invoke that go binary during android device setup.
,
Aug 9 2017
How long do installroot and removeroot take to execute? My suspicion is that this should be done in telemetry on a per-task basis.
,
Aug 9 2017
installroot and removeroot take 2-3 seconds on my Nexus5x. I don't know about Telemetry, so I will let Ned take that.
,
Aug 9 2017
Yes, I think it should be done when webpagereplay server is up & running (probably before https://github.com/catapult-project/catapult/blob/master/telemetry/telemetry/internal/platform/network_controller_backend.py#L235). For the case of Android, you don't need to worry about unittests trying to install test root CA on a same device in parallel.
,
Aug 9 2017
#4: I sure hope not :)
,
Aug 9 2017
,
Aug 9 2017
Thanks, Ned. I only tested the installroot script on a nexus5x and a nexus7, so fingers crossed that this actually works :)
,
Aug 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/25acaba9b5d17a807322f25b5a4af95f9c799a8d commit 25acaba9b5d17a807322f25b5a4af95f9c799a8d Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Thu Aug 17 23:56:19 2017 Roll src/third_party/catapult/ 8995f3ead..d9436e5a0 (2 commits) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/8995f3ead39e..d9436e5a07a3 $ git log 8995f3ead..d9436e5a0 --date=short --no-merges --format='%ad %ae %s' 2017-08-17 nednguyen Implement wprgo certificate installation & removal for Android platform 2017-08-17 sullivan Add additional logging for bug comments. Created with: roll-dep src/third_party/catapult BUG= 753948 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: Ic2624b70ac5c6011cb0a97c6d774fba020ff7df5 Reviewed-on: https://chromium-review.googlesource.com/619602 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#495388} [modify] https://crrev.com/25acaba9b5d17a807322f25b5a4af95f9c799a8d/DEPS
,
Aug 19 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6a0929203603f1ebd5f85622f75aa21cdd767768 commit 6a0929203603f1ebd5f85622f75aa21cdd767768 Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Sat Aug 19 03:08:19 2017 Roll src/third_party/catapult/ 986edc627..d2ffc23f1 (1 commit) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/986edc627e1e..d2ffc23f1e05 $ git log 986edc627..d2ffc23f1 --date=short --no-merges --format='%ad %ae %s' 2017-08-18 xunjieli [wpr-go] Add detailed error msg to adb_cert_installer.go Created with: roll-dep src/third_party/catapult BUG= 753948 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: I911d1c600d077851e01bd6fce1e019054b9a4d92 Reviewed-on: https://chromium-review.googlesource.com/621275 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#495794} [modify] https://crrev.com/6a0929203603f1ebd5f85622f75aa21cdd767768/DEPS
,
Aug 21 2017
,
Aug 22 2017
Just as a drive-by - Is there a reason --ignore-certificate-errors-spki-list wouldn't work? This avoids the downsides you mentioned, while also avoiding the need for device maintenance for cert installation/removal :)
,
Aug 22 2017
Thanks, Ryan. I didn't know about --ignore-certificate-errors-spki-list. All certs served by the WebPageReplay servers are self signed. I thought the only way to bypass the cert verification errors is by installing a trusted root. Does --ignore-certificate-errors-spki-list work for all self signed certs? If so, that will definitely make our life easier.
,
Aug 22 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e753ea5052f62c1cee2d4cfa0d0378b5d239f3c2 commit e753ea5052f62c1cee2d4cfa0d0378b5d239f3c2 Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Tue Aug 22 14:07:30 2017 Roll src/third_party/catapult/ 3f95ee609..a28c28f82 (1 commit) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/3f95ee609aa3..a28c28f82bf0 $ git log 3f95ee609..a28c28f82 --date=short --no-merges --format='%ad %ae %s' 2017-08-22 xunjieli [wpr-go] Include stderr in error msg Created with: roll-dep src/third_party/catapult BUG= 753948 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: I1f2fbb7946f877b70c90f90c69083aeddfe07fd8 Reviewed-on: https://chromium-review.googlesource.com/625685 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#496296} [modify] https://crrev.com/e753ea5052f62c1cee2d4cfa0d0378b5d239f3c2/DEPS
,
Aug 22 2017
I'm not sure I understand your deployment scenario - that is, you can either have a self-signed root (issuing fake certs) or generate ad-hoc fake self-signed certs (with no root) It sounded like you have an existing fake root, which then signs various server certs. If you have that scenario, then ignore-certificate-errors-spki-list specifying the root's fake key will work. If you have fake server certs, you'd need to specify all the hashes - or, if they share the same key, just that key's hash. Does that work? It's part of //content now, but it's up to the embedder to require any additional constraints (like user-data-dir, as Chrome does). See https://cs.chromium.org/chromium/src/content/public/browser/ignore_errors_cert_verifier.h?rcl=3f15cf722e5173251ca88d36a5ba73a16e362bbd&l=40
,
Aug 22 2017
Wpr has a fake root which it uses to sign all server certs. If I understand correctly, we can just specify the root's key in ignore-certificate-errors-spki-list. I will try that today. One more question, if we use "ignore-certificate-errors-spki-list" instead of "ignore-certificate-errors", will we have the same problem with two socket connections and skipping disk cache? Thanks!
,
Aug 22 2017
Yup. spki-list requires user-data-directory because we allow it to use the disk cache, and it's integrated into the CertVerifier (rather than the URLRequest), so it avoids the disconnect&reconnect. Thus, it almost indistinguishable from a real request. I say almost, because we actually skip OS verification if it's on the whitelist, so it's slightly faster. But using a fake root cert is no more representative of a 'real' connection, because of how the OS handles it, so if that (slight performance improvement) is a concern, happy to explain why it shouldn't be :)
,
Aug 24 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/56255a6fa1af0fc35457737cfbe8f5ab57a3ebd7 commit 56255a6fa1af0fc35457737cfbe8f5ab57a3ebd7 Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Thu Aug 24 01:54:47 2017 Roll src/third_party/catapult/ b9777c776..92387bc7d (1 commit) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/b9777c776dee..92387bc7d0ac $ git log b9777c776..92387bc7d --date=short --no-merges --format='%ad %ae %s' 2017-08-23 xunjieli [catapult] Roll forward Chrome reference builds Created with: roll-dep src/third_party/catapult BUG= 753948 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: Ifeb0c32e0dbe9b39d60bb9eef3ec148ad9d00811 Reviewed-on: https://chromium-review.googlesource.com/630700 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#496916} [modify] https://crrev.com/56255a6fa1af0fc35457737cfbe8f5ab57a3ebd7/DEPS
,
Aug 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/15976caadc6dfee1df124d31383f83b2f51ec578 commit 15976caadc6dfee1df124d31383f83b2f51ec578 Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Thu Aug 31 17:28:51 2017 Roll src/third_party/catapult/ 8a17d6431..b0b51eab2 (1 commit) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/8a17d6431927..b0b51eab27da $ git log 8a17d6431..b0b51eab2 --date=short --no-merges --format='%ad %ae %s' 2017-08-31 xunjieli [Telemetry] Add --user-data-dir to android_browser_backend.py Created with: roll-dep src/third_party/catapult BUG= 753948 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: I03a7ccc0164abf001f1877ae90f3f97e3c44a2e9 Reviewed-on: https://chromium-review.googlesource.com/646230 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#498906} [modify] https://crrev.com/15976caadc6dfee1df124d31383f83b2f51ec578/DEPS
,
Sep 5 2017
xunjieli@ is the one who is doing work here.
,
Sep 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/54468efa2d28668d7547404902aa4915a79b09a4 commit 54468efa2d28668d7547404902aa4915a79b09a4 Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Thu Sep 07 18:48:20 2017 Roll src/third_party/catapult/ c9667ecd2..29f450ae4 (16 commits) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/c9667ecd29cb..29f450ae4dcb $ git log c9667ecd2..29f450ae4 --date=short --no-merges --format='%ad %ae %s' 2017-09-07 littlecvr [Telemetry] Add ChromeOS to desktop platform list 2017-09-07 dtu [pinpoint] Separate Execution exceptions from result_values. 2017-09-07 lalitm Update memtrack binary version for ARMv7 devices 2017-09-07 nednguyen Revert of Smoke test for heap profiler. (patchset #2 id:20001 of https://codereview.chromium.org/3010173002/ ) 2017-09-06 rnephew [Telemetry] Fully get rid of PermanentlyDisableBenchmark. 2017-09-06 benjhayden Add traceUrls to CSVs. 2017-09-06 benjhayden Format traceUrls in generic-set-span. 2017-09-06 benjhayden Remove incorrect test testGet_WithFinish_LabelsBugWithLowestMilestonePossible 2017-09-06 benjhayden Produce all statistics in CSVBuilder. 2017-09-06 sullivan Add CORS headers for whitelisted origins. 2017-09-06 xunjieli [wpr-go] Update comment in telemtry/bin/update_wpr_go_binary 2017-09-06 simonhatch Dashboard - Add calls to graph_revisions and find_anomalies in add_histograms_queue 2017-09-06 loloangela Fix errors related to invalid-name pt. 10 2017-09-06 dtu [pinpoint] Limit executions to one test run each + device sharding. 2017-09-06 xunjieli [Telemetry] Use --ignore-certificate-errors-spki-list to bypass cert errors 2017-09-06 kraynov Smoke test for heap profiler. Created with: roll-dep src/third_party/catapult BUG=753279, 670828 ,713222, 753948 , 670828 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: I81717857fe493d9edb734becf08f69534f3adbb5 Reviewed-on: https://chromium-review.googlesource.com/655362 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#500345} [modify] https://crrev.com/54468efa2d28668d7547404902aa4915a79b09a4/DEPS
,
Sep 7 2017
,
Sep 11 2017
,
Sep 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b808d5e225a0ba21d3eb720173a27f8feac2f563 commit b808d5e225a0ba21d3eb720173a27f8feac2f563 Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Mon Sep 11 20:09:55 2017 Roll src/third_party/catapult/ f465506fe..99ec81878 (6 commits) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/f465506fe2a7..99ec8187805e $ git log f465506fe..99ec81878 --date=short --no-merges --format='%ad %ae %s' 2017-09-11 dtu [pinpoint] Rename some variables on the front-end. 2017-09-11 loloangela Fix errors related to invalid-name pt. 8 2017-09-11 nednguyen Use --ignore-certificate-errors for webview 2017-09-11 benjhayden Upgrade <dom-module name> to <dom-module id> for Polymer 2.0. 2017-09-11 charliea Release a new version of the BattOr agent binary 2017-09-11 yuzus Add PrepareForLeakDetection method in telemetry Created with: roll-dep src/third_party/catapult BUG= 753948 , 763110 , 763280 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: I3bd453c50dd53950415724cc3b68c180fa50e56f Reviewed-on: https://chromium-review.googlesource.com/661179 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#501012} [modify] https://crrev.com/b808d5e225a0ba21d3eb720173a27f8feac2f563/DEPS
,
Sep 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/158a7a01ceb96fd019e996e3501d2fd2995bebb2 commit 158a7a01ceb96fd019e996e3501d2fd2995bebb2 Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org> Date: Mon Sep 11 23:09:59 2017 Roll src/third_party/catapult/ 99ec81878..c59db25d8 (5 commits) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/99ec8187805e..c59db25d8116 $ git log 99ec81878..c59db25d8 --date=short --no-merges --format='%ad %ae %s' 2017-09-11 eakuefner [Tracing] Add add_shared_diagnostics.AddValueDiagnostics 2017-09-11 benjhayden Export {raw,merged} {csv,json} from results.html. 2017-09-11 dtu [pinpoint] Re-layout job page header. 2017-09-11 xunjieli [wpr-go] Use wprgo for test_page_sets data. 2017-09-11 rnephew [Telemetry] Expose disable_stories in StoryExpectations api. Created with: roll-dep src/third_party/catapult BUG= 753948 ,723636 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=sullivan@chromium.org Change-Id: I759eca1f159ddd866b45992789ab2f39dcf8fbca Reviewed-on: https://chromium-review.googlesource.com/660896 Reviewed-by: <catapult-deps-roller@chromium.org> Commit-Queue: <catapult-deps-roller@chromium.org> Cr-Commit-Position: refs/heads/master@{#501080} [modify] https://crrev.com/158a7a01ceb96fd019e996e3501d2fd2995bebb2/DEPS
,
Sep 12 2017
Two platforms that are missing --ignore-certificate-errors-spki-list are WebView (more at 763880) and ChromeOs ( Issue 762819 ). I will do it for WebView once CQ support ( Issue 763974 ) is in.
,
Sep 15 2017
ChromeOS ( Issue 762819 ) is fixed. I've sent out a CL for WebView (https://chromium-review.googlesource.com/c/chromium/src/+/667696).
,
Sep 15 2017
,
Sep 15 2017
I uploaded a CL to address WebView (https://chromium-review.googlesource.com/c/chromium/src/+/667696). That approach doesn't work, and I've abandoned it. To summarize, (1) switches::kUserDataDir is a //chrome concept. (2) WebView doesn't allow setting command line flags on end user builds of Android. Having WebView check kUserDataDir doesn't make sense. I will leave WebView as it is. Someone can pick it up if we need that flag there.
,
Sep 15 2017
,
Jan 10 2018
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/fef1e79fe037bbe15630b692832efa7ea0ab4ade commit fef1e79fe037bbe15630b692832efa7ea0ab4ade Author: Juan Antonio Navarro Perez <perezju@google.com> Date: Wed Jan 10 09:53:05 2018 [Telemetry] Move GetReplayArgs to chrome_startup_args module Factor out GetReplayArgs from chrome_browser_backend into the new chrome_startup_args module. Also move their corresponding tests. This allows to simplify further more the computation of startup args as we decouple them from the existence of the browser_backend. Bug: chromium:787834 Bug: chromium:753948 Change-Id: I16fd0ed996d9fed67e67575d427bd407a98be968 Reviewed-on: https://chromium-review.googlesource.com/857461 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org> [modify] https://crrev.com/fef1e79fe037bbe15630b692832efa7ea0ab4ade/telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py [modify] https://crrev.com/fef1e79fe037bbe15630b692832efa7ea0ab4ade/telemetry/telemetry/internal/backends/chrome/chrome_startup_args_unittest.py [modify] https://crrev.com/fef1e79fe037bbe15630b692832efa7ea0ab4ade/telemetry/telemetry/internal/backends/chrome/android_browser_finder.py [modify] https://crrev.com/fef1e79fe037bbe15630b692832efa7ea0ab4ade/telemetry/telemetry/internal/backends/chrome/cros_browser_finder.py [delete] https://crrev.com/d4706cb285d8caaa8e7c0334e0528efe7b9a26ff/telemetry/telemetry/internal/backends/chrome/chrome_browser_backend_unittest.py [modify] https://crrev.com/fef1e79fe037bbe15630b692832efa7ea0ab4ade/telemetry/telemetry/internal/backends/chrome/chrome_startup_args.py [modify] https://crrev.com/fef1e79fe037bbe15630b692832efa7ea0ab4ade/telemetry/telemetry/internal/backends/chrome/desktop_browser_finder.py [modify] https://crrev.com/fef1e79fe037bbe15630b692832efa7ea0ab4ade/telemetry/telemetry/internal/backends/chrome/android_browser_backend.py
,
Jan 16
,
Jan 16
|
||||||||||||
►
Sign in to add a comment |
||||||||||||
Comment 1 by xunji...@chromium.org
, Aug 9 2017