The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8c2a381d0adbb6d791172cee1b311c1ad4f31e8e

commit 8c2a381d0adbb6d791172cee1b311c1ad4f31e8e
Author: csharrison <csharrison@chromium.org>
Date: Thu Nov 10 20:24:41 2016

Cache net::HostPortPair in ResourceScheduler::ScheduledResourceRequest

When loading http://cr.kungfoo.net/loading/1000spacers/ with resources
coming from the disk cache, responding to priority changes takes up
~26% of the CPU time spent on the IO thread. This change moves that
to ~7.7% (Linux).

The time is saved in EffectiveIntPort, which must do a lookup for the
scheme in a set (DoIsStandard).

There is a memory overhead, but since we are only storing host and port
this should not be too bad.

BUG=664174

Review-Url: https://codereview.chromium.org/2488323002
Cr-Commit-Position: refs/heads/master@{#431339}

[modify] https://crrev.com/8c2a381d0adbb6d791172cee1b311c1ad4f31e8e/content/browser/loader/resource_scheduler.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b8ec6582c1c86524b3cb47dbf65605213d35930e

commit b8ec6582c1c86524b3cb47dbf65605213d35930e
Author: csharrison <csharrison@chromium.org>
Date: Fri Nov 11 01:05:06 2016

[extensions] Remove unnecessary checks in IsSensitiveURL

The current code re-parses the url by removing the query and ref components
before sending the modified URL to IsWebstoreUpdateUrl. This is not
necessary, as IsWebstoreUpdateUrl will return true if the host and path
are identical.

Creating the modified URL takes ~3-4% of the total task time on the IOthread
when loading http://cr.kungfoo.net/loading/1000spacers/

This CL also cleans up checks in the IsSensitiveURL function.

BUG=664174,664285

Review-Url: https://codereview.chromium.org/2489233003
Cr-Commit-Position: refs/heads/master@{#431432}

[modify] https://crrev.com/b8ec6582c1c86524b3cb47dbf65605213d35930e/extensions/browser/BUILD.gn
[modify] https://crrev.com/b8ec6582c1c86524b3cb47dbf65605213d35930e/extensions/browser/api/web_request/web_request_permissions.cc
[modify] https://crrev.com/b8ec6582c1c86524b3cb47dbf65605213d35930e/extensions/browser/api/web_request/web_request_permissions.h
[add] https://crrev.com/b8ec6582c1c86524b3cb47dbf65605213d35930e/extensions/browser/api/web_request/web_request_permissions_unittest.cc
[modify] https://crrev.com/b8ec6582c1c86524b3cb47dbf65605213d35930e/extensions/common/extension_urls.cc
[modify] https://crrev.com/b8ec6582c1c86524b3cb47dbf65605213d35930e/extensions/test/test_extensions_client.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f95b1d92dfe18b6350cdc0f23e34ddabb07d041d

commit f95b1d92dfe18b6350cdc0f23e34ddabb07d041d
Author: jkarlin <jkarlin@chromium.org>
Date: Fri Nov 11 20:44:33 2016

[base::FilePath] Optimize FilePath::ReferencesParent

ReferencesParent is expensive but in the majority of cases can early return if
no '..' is present in the string, significantly reducing the average cost.

When loading 1,000 tiny images (35 bytes each), the SimpleCache worker threads
spend 10.7% of their time in ReferencesParent according to linux perf (during
File::Initialize). After this change, they spend .4% of their time in ReferencesParent.

BUG=664174

Review-Url: https://codereview.chromium.org/2500473002
Cr-Commit-Position: refs/heads/master@{#431632}

[modify] https://crrev.com/f95b1d92dfe18b6350cdc0f23e34ddabb07d041d/base/files/file_path.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4ff7b40b01046d703b33c4c96a491fdb923d20cd

commit 4ff7b40b01046d703b33c4c96a491fdb923d20cd
Author: csharrison <csharrison@chromium.org>
Date: Sat Nov 12 04:41:22 2016

[extensions] Stop parsing webstore urls so much

The following hooks parse the webstore url per request, sometimes twice.
 - OnHeadersReceived
 - OnResponseStarted
 - OnBeforeRequest
 - OnBeforeSendHeaders
 - OnSendHeaders
 - OnCompleted

This accounts for ~3% of task CPU time on the IO thread from loading
http://cr.kungfoo.net/loading/1000spacers/ on Linux.

This patch just caches the url at the ExtensionsClient layer, so we never
have to do this more than once.

BUG=664174

Review-Url: https://codereview.chromium.org/2493053002
Cr-Commit-Position: refs/heads/master@{#431771}

[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/chrome/browser/extensions/chrome_content_verifier_delegate.cc
[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/chrome/common/extensions/chrome_extensions_client.cc
[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/chrome/common/extensions/chrome_extensions_client.h
[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/chrome/common/extensions/extension_constants.cc
[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/chrome/common/extensions/extension_constants.h
[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/extensions/common/extension_urls.cc
[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/extensions/common/extensions_client.h
[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/extensions/shell/common/shell_extensions_client.cc
[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/extensions/shell/common/shell_extensions_client.h
[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/extensions/test/test_extensions_client.cc
[modify] https://crrev.com/4ff7b40b01046d703b33c4c96a491fdb923d20cd/extensions/test/test_extensions_client.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a628b8fc2a3e19ce72cef9cd0802ad79a892f7f6

commit a628b8fc2a3e19ce72cef9cd0802ad79a892f7f6
Author: csharrison <csharrison@chromium.org>
Date: Mon Nov 14 23:58:27 2016

Fast path for google_util::IsValidHostName

IsValidHostName takes a host and tries to match it with
[www.]<domain_in_lower_case>.<TLD>

However, finding valid TLDs entails searching a big DAFSA. This patch
uses a fast path to fail fast if <domain_in_lower_case> is not found in
the input host.

This function was seen in profiles on the IO thread in
variations::AppendVariationHeaders.

BUG=664174

Review-Url: https://codereview.chromium.org/2495773003
Cr-Commit-Position: refs/heads/master@{#431995}

[modify] https://crrev.com/a628b8fc2a3e19ce72cef9cd0802ad79a892f7f6/components/google/core/browser/google_util.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/11d71cb74a864e5459c97a701a1e2f8eadb694c8

commit 11d71cb74a864e5459c97a701a1e2f8eadb694c8
Author: csharrison <csharrison@chromium.org>
Date: Tue Nov 15 01:45:04 2016

Invert host/domain checks in SameDomainOrHost

Checking for domains in the DAFSA is slow. Let's check if hosts are
identical first as a quick fast path for URLs with the same host.

The lookup in the fixed string set takes ~10% of
RDHI::ContinuePendingBeginRequest and ~7.5% of URLRequest::Start.

BUG=664174

Review-Url: https://codereview.chromium.org/2497753002
Cr-Commit-Position: refs/heads/master@{#432047}

[modify] https://crrev.com/11d71cb74a864e5459c97a701a1e2f8eadb694c8/net/base/registry_controlled_domains/registry_controlled_domain.cc
[modify] https://crrev.com/11d71cb74a864e5459c97a701a1e2f8eadb694c8/net/base/registry_controlled_domains/registry_controlled_domain.h

Charlie, Erik: would be great if these improvements are reflected on our power benchmarks.

We've been using cr.kungfoo.net/loading/1000spacers with cached content to benchmark the IO thread.

Note that the main purpose of this work was to unblock requests from making progress. Since the work is limited to resource requests, I would be surprised if it corresponded to major power wins.

As an FYI I also worked on some optimizations in the render process in issue 	348655

Is it worth setting up a telemetry test or suite of tests that is able to discern these improvements? Otherwise, this may slowly regress over time.

Probably. There is the deprecated test in issue 348655 (send.html) that I used for render process things. A trace based test would be better.

+mmenke and let's reference  issue 662165  (net perf tests).

I've been thinking more about more unit-y perf tests of top level net/ objects / RDH requests.  I certainly don't oppose integration tests as well.

SGTM. Note that many of these fixes will apply to unit test style perf tests as well as trace based integration tests.

Erik, Charlie: IIRC, the current cpu time metrics may not capture the business of IO thread?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/26a0f5bf7030a09f679d41532318282b3e8ee47d

commit 26a0f5bf7030a09f679d41532318282b3e8ee47d
Author: csharrison <csharrison@chromium.org>
Date: Wed Nov 16 15:40:02 2016

Avoid string copies when determining approximate memory usage in RDHI

This patch just pulls the lengths out of the request headers and GURL
rather than serializing/copying them and then querying the length.

BUG=664174

Review-Url: https://codereview.chromium.org/2503713004
Cr-Commit-Position: refs/heads/master@{#432495}

[modify] https://crrev.com/26a0f5bf7030a09f679d41532318282b3e8ee47d/content/browser/loader/resource_dispatcher_host_impl.cc
[modify] https://crrev.com/26a0f5bf7030a09f679d41532318282b3e8ee47d/content/browser/loader/resource_dispatcher_host_impl.h
[modify] https://crrev.com/26a0f5bf7030a09f679d41532318282b3e8ee47d/content/browser/loader/resource_dispatcher_host_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4eb0252ac0c718fe297a363b57850f7da11c0a40

commit 4eb0252ac0c718fe297a363b57850f7da11c0a40
Author: csharrison <csharrison@chromium.org>
Date: Wed Nov 16 16:09:45 2016

Fast path for ContentSettingsPattern::Wildcard

This method used to built up a ContentSettingsPattern and validate it
using the builder pattern. All this validation work is not needed, and
this patch removes it in favor of building the pattern directly.

BUG=664174

Review-Url: https://codereview.chromium.org/2507813002
Cr-Commit-Position: refs/heads/master@{#432506}

[modify] https://crrev.com/4eb0252ac0c718fe297a363b57850f7da11c0a40/components/content_settings/core/common/content_settings_pattern.cc
[modify] https://crrev.com/4eb0252ac0c718fe297a363b57850f7da11c0a40/components/content_settings/core/common/content_settings_pattern_unittest.cc

[Bulk edit]

Branch issues caused these items to be erroneously tagged as merged to branch 2922, please ignore.

nednguyen@: the current power metrics definitely capture what's happening on the IO thread. The question that tdresser@ raised was whether the current system health user stories have any main thread scrolling. I think that rnephew@ had a better story for why the scroll bar rendering power regression didn't regress the system health benchmarks, which was that the scroll on those stories was very short and we immediately navigated away from the page after that scroll. He changed this in a CL submitted yesterday, and I have it on my backlog to check whether this change would have made the scrollbar regression show up in SH stories.

FWIW, I see no measurable impact on our lab power metrics, at least on the MBP retina laptops that we currently have the system health benchmarks running on: https://chromeperf.appspot.com/report?sid=568a6a10393d1bbc88e3e69d8231dc71a68efeb529432fd30f58632ca1e7575f

csharrison@, is this likely to have more of an impact using live sites where the site is likely to be blocked on network resource requests?

#23 Nope, this is primarily designed to make IO tasks shorter so we don't have task starvation for resources where everything is cached. Note that the IO thread is generally much less of a power/CPU hog than Blink, so I would be surprised if this actually moved power metrics.

Okay, GTK. Thanks! (I'll go ahead and remove myself from the bug, but keep an eye out on the waterfall for any performance improvements/regressions that might be attributable to your work.)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1

commit 5d1f214db0f7996f3c17cd87093d439ce4c7f8f1
Author: csharrison <csharrison@chromium.org>
Date: Tue Nov 22 20:25:47 2016

Avoid parsing the webstore base url so much.

This is a followup to [1] which modified the extension url API to vend
GURLs rather than std::string for webstore update urls.

This patch does the same for the webstore base url
(the webstore "launch" url). We only parse this url in
ChromeResourceDispatcherHostDelegate::OnResponseStarted so there is less
of a performance win than [1], but still we should avoid doing needless
work.

Parsing this URL takes ~12% of the CPU time of
ResourceLoader::CompleteResponseStarted.

[1] https://codereview.chromium.org/2493053002/

BUG=664174

Review-Url: https://codereview.chromium.org/2506713002
Cr-Commit-Position: refs/heads/master@{#433963}

[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/chrome/browser/chromeos/app_mode/kiosk_app_data.cc
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/chrome/browser/extensions/api/inline_install_private/inline_install_private_apitest.cc
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/chrome/browser/extensions/webstore_standalone_installer.cc
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/chrome/browser/prerender/prerender_browsertest.cc
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/chrome/browser/ui/app_list/search/webstore/webstore_provider_browsertest.cc
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/chrome/browser/ui/webui/ntp/ntp_resource_cache.cc
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/chrome/common/extensions/chrome_extensions_client.cc
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/chrome/common/extensions/chrome_extensions_client.h
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/extensions/common/extension_urls.cc
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/extensions/common/extension_urls.h
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/extensions/common/extensions_client.h
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/extensions/shell/common/shell_extensions_client.cc
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/extensions/shell/common/shell_extensions_client.h
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/extensions/test/test_extensions_client.cc
[modify] https://crrev.com/5d1f214db0f7996f3c17cd87093d439ce4c7f8f1/extensions/test/test_extensions_client.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/db2deb26ff656262712010cfd9a8580a005dc16f

commit db2deb26ff656262712010cfd9a8580a005dc16f
Author: csharrison <csharrison@chromium.org>
Date: Mon Nov 28 18:10:18 2016

Allow getting both reading/setting cookie settings at one time

Getting the underlying WebsiteSetting to find whether reading/setting
cookies is allowed is fairly expensive. The current cookie API requires
retrieving these settings twice if we want to know both values.

This patch updates the API to allow for querying reading and setting
policies with one call to GetWebsiteSettings. This is used in
net::NetworkDelegate::CanEnablePrivacyMode, which went from taking 18%
of the CPU time in net::URLRequestHttpJob::Start, to 12% after this
patch (Linux perf results).

BUG=664174

Review-Url: https://codereview.chromium.org/2502743003
Cr-Commit-Position: refs/heads/master@{#434691}

[modify] https://crrev.com/db2deb26ff656262712010cfd9a8580a005dc16f/chrome/browser/extensions/api/content_settings/content_settings_api.cc
[modify] https://crrev.com/db2deb26ff656262712010cfd9a8580a005dc16f/chrome/browser/net/chrome_network_delegate.cc
[modify] https://crrev.com/db2deb26ff656262712010cfd9a8580a005dc16f/chrome/browser/ui/content_settings/content_setting_bubble_model.cc
[modify] https://crrev.com/db2deb26ff656262712010cfd9a8580a005dc16f/components/content_settings/core/browser/cookie_settings.cc
[modify] https://crrev.com/db2deb26ff656262712010cfd9a8580a005dc16f/components/content_settings/core/browser/cookie_settings.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/16060210ca945bcd601e380617d75f916a084932

commit 16060210ca945bcd601e380617d75f916a084932
Author: csharrison <csharrison@chromium.org>
Date: Mon Nov 28 22:52:53 2016

net::SimplifyUrlForRequest fast path for URLs without {user,pass,ref} parts.

This function uses GURL::ReplaceComponents which triggers
canonicalization of the new URL. For the normal case where urls do not
have usernames, passwords, or ref parts, we can just return the original
URL.

ReplaceComponents takes around 13% of the time spent in
HttpCache::Transaction::Start with a SimpleCache backend on Linux, but
this method is used in a few different places.

BUG=664174

Review-Url: https://codereview.chromium.org/2498423004
Cr-Commit-Position: refs/heads/master@{#434756}

[modify] https://crrev.com/16060210ca945bcd601e380617d75f916a084932/net/base/url_util.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d5c68c496fa9111fb45e482d4feea0f1a2cdcb9e

commit d5c68c496fa9111fb45e482d4feea0f1a2cdcb9e
Author: csharrison <csharrison@chromium.org>
Date: Sat Dec 03 00:15:27 2016

Avoid gurl -> origin -> gurl conversion in AddCookieHeaderAndStart

SameDomainOrHost does not require Origins. In fact, we just convert the
origins back to GURLs to do the check, which just looks at the host
piece of the url.

This conversion is not free in either direction, as we need to do
allocations, copies, and in some cases, url-related policies
(e.g. in constructing the underlying SchemeHostPort of Origin).

BUG=664174

Review-Url: https://codereview.chromium.org/2542843009
Cr-Commit-Position: refs/heads/master@{#436092}

[modify] https://crrev.com/d5c68c496fa9111fb45e482d4feea0f1a2cdcb9e/net/url_request/url_request_http_job.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/37cdff911bf891aacbd747c8d9a8be24678e8448

commit 37cdff911bf891aacbd747c8d9a8be24678e8448
Author: csharrison <csharrison@chromium.org>
Date: Tue Dec 06 19:20:47 2016

Stop initializing url::Origin in extensions OnBeforeURLRequest when not needed

This patch also removes a GURL copy.

BUG=664174

Review-Url: https://codereview.chromium.org/2552453002
Cr-Commit-Position: refs/heads/master@{#436681}

[modify] https://crrev.com/37cdff911bf891aacbd747c8d9a8be24678e8448/chrome/browser/net/chrome_extensions_network_delegate.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/475851dad085b226d5459363e4daf152eb4cfc77

commit 475851dad085b226d5459363e4daf152eb4cfc77
Author: csharrison <csharrison@chromium.org>
Date: Sat Dec 17 02:19:42 2016

Merge logic for SameDomainOrHost for GURLs and Origins

Previously, we used to just call:
SameDomainOrHost(origin1.GetURL(), origin2.GetURL())

This isn't necessary, as we have all the information in the Origin itself.
Additionally, GetURL() isn't free. We do a bunch of string copies,
allocations, and some non-cheap policy decisions.

This patch moves GURL::HostIsIPAddress logic to url_util, and reuses
that code for both GURLs and Origins.

BUG=664174,348655

Review-Url: https://codereview.chromium.org/2562813003
Cr-Commit-Position: refs/heads/master@{#439287}

[modify] https://crrev.com/475851dad085b226d5459363e4daf152eb4cfc77/net/base/registry_controlled_domains/registry_controlled_domain.cc
[modify] https://crrev.com/475851dad085b226d5459363e4daf152eb4cfc77/url/gurl.cc
[modify] https://crrev.com/475851dad085b226d5459363e4daf152eb4cfc77/url/url_util.cc
[modify] https://crrev.com/475851dad085b226d5459363e4daf152eb4cfc77/url/url_util.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/59501351d872c1348665ad6dfd23b09396821056

commit 59501351d872c1348665ad6dfd23b09396821056
Author: csharrison <csharrison@chromium.org>
Date: Tue Feb 07 16:02:43 2017

Get rid of quadratic behavior in ResourceScheduler

The ResourceScheduler often receives bursts of IPCs from the renderer,
which call LoadAnyStartablePendingRequests. This method is O(n) for n
pending requests for that client/tab.

Because the IPCs are busty, we often have a message queue of m messages,
each one calling into LoadAnyStartablePendingRequests, yielding O(m*n)
behavior.

This patch removes the O(m*n) behavior. As soon as one of these "bursty"
messages is handled, we schedule a new task to call
LoadAnyStartablePendingRequests. If the message queue has m messages
queued up, this will put the call to LoadAnyStartablePendingRequests at
the end. By ensuring we only have one scheduled task to load the startable
requests, we effectively coalesce all of the other calls into this method.

Another technique to remove this inefficiency would be to batch the IPC
messages into one. This is being explored by kinuko@ in  issue 672370 .
This approach however is very simple, and could easily be removed if we
move to a batching system.

BUG=664174

Review-Url: https://codereview.chromium.org/2670843007
Cr-Commit-Position: refs/heads/master@{#448634}

[modify] https://crrev.com/59501351d872c1348665ad6dfd23b09396821056/content/browser/loader/resource_scheduler.cc