All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=811457

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=f8d774a8e6c3ef03d86eab1937e94357d52285ca5038fa657b1db1a92ecb6d84


Bot(s) for this bug's original alert(s):

chromium-rel-win7-dual
chromium-rel-win7-gpu-ati
chromium-rel-win7-gpu-nvidia
chromium-rel-win7-x64-dual
chromium-rel-win8-dual

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/15d10d6d840000

📍 Found significant differences after each of 3 commits.
https://pinpoint-dot-chromeperf.appspot.com/job/15d10d6d840000

Reland "ozone/drm: Always export format modifiers" by hoegsberg@chromium.org
https://chromium.googlesource.com/chromium/src/+/d1809bb40a31f4488cdcc4a8b19e40af2860d7db

Consider SETTINGS identifier 0x07 as invalid. by bnc@chromium.org
https://chromium.googlesource.com/chromium/src/+/e04a14f4769d6b33b32686ccc3414e917a625ae7

Enable root layer scrolling. by skobes@chromium.org
https://chromium.googlesource.com/chromium/src/+/96f85b68747a679ea1ac4cd05d6743ae5f7142b7

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

+dtu can you take a look at this bisect result? 

I think Pinpoint got confused by the pattern of failures, since all the runs passed on hoegsberg's change but not the changes around it, so it thought that one of those changes caused the flakiness.

When comparing the cpu_time_percentage result values, skobe's change gave a p-value below the threshold (p=0.000530038145972 < 0.001), so I think that's the one.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6c2704c29d7bc4659f8004c39db39803d8948148

commit 6c2704c29d7bc4659f8004c39db39803d8948148
Author: Stefan Zager <szager@chromium.org>
Date: Fri Feb 16 03:42:22 2018

Reduce calls to MapLocalToAncestor in UpdateGeometry

BUG= 811457 
R=chrishtr@chromium.org

Change-Id: Ifc97bd14dc13ed546fa89711de531e517fa046f7
Reviewed-on: https://chromium-review.googlesource.com/923005
Reviewed-by: Chris Harrelson <chrishtr@chromium.org>
Commit-Queue: Stefan Zager <szager@chromium.org>
Cr-Commit-Position: refs/heads/master@{#537180}
[modify] https://crrev.com/6c2704c29d7bc4659f8004c39db39803d8948148/third_party/WebKit/Source/core/layout/LayoutEmbeddedContent.cpp

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6c2704c29d7bc4659f8004c39db39803d8948148

commit 6c2704c29d7bc4659f8004c39db39803d8948148
Author: Stefan Zager <szager@chromium.org>
Date: Fri Feb 16 03:42:22 2018

Reduce calls to MapLocalToAncestor in UpdateGeometry

BUG= 811457 
R=chrishtr@chromium.org

Change-Id: Ifc97bd14dc13ed546fa89711de531e517fa046f7
Reviewed-on: https://chromium-review.googlesource.com/923005
Reviewed-by: Chris Harrelson <chrishtr@chromium.org>
Commit-Queue: Stefan Zager <szager@chromium.org>
Cr-Commit-Position: refs/heads/master@{#537180}
[modify] https://crrev.com/6c2704c29d7bc4659f8004c39db39803d8948148/third_party/WebKit/Source/core/layout/LayoutEmbeddedContent.cpp

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/52614404e7a423928cde0e96c53b5c5673f6df5e

commit 52614404e7a423928cde0e96c53b5c5673f6df5e
Author: Stefan Zager <szager@chromium.org>
Date: Fri Feb 16 05:54:34 2018

Eliminate potential redundant calls to UpdateGeometry.

LocalFrameView::UpdateGeometries is called from
PerformPostLayoutTasks, and then UpdateGeometriesIfNeeded is called
after layout in UpdateStyleAndLayoutIfNeededRecursiveInternal.  If
the first call doesn't clear the needs_update_geometries_ flag, then
the second call will do unnecessary work.

BUG= 811457 
R=chrishtr@chromium.org

Change-Id: Idc8e11c444d8b02cd9d54c2fd158bbdd59bd5de4
Reviewed-on: https://chromium-review.googlesource.com/923119
Reviewed-by: Chris Harrelson <chrishtr@chromium.org>
Commit-Queue: Stefan Zager <szager@chromium.org>
Cr-Commit-Position: refs/heads/master@{#537226}
[modify] https://crrev.com/52614404e7a423928cde0e96c53b5c5673f6df5e/third_party/WebKit/Source/core/frame/LocalFrameView.cpp
[modify] https://crrev.com/52614404e7a423928cde0e96c53b5c5673f6df5e/third_party/WebKit/Source/core/frame/LocalFrameView.h

This regression is still happening, and none of the recent changes made
any difference.

-> szager

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/cf91f7965c29e2bc93e81f79cef5d77481b00e6e

commit cf91f7965c29e2bc93e81f79cef5d77481b00e6e
Author: Stefan Zager <szager@chromium.org>
Date: Sat Feb 17 00:55:45 2018

Omit the parent LayoutView's scroll offset from FrameRect

This is a performance optimization in the same vein as PaintLayer
(https://chromium-review.googlesource.com/854628).

Since EmbeddedContentView::FrameRect() now has to compute its location, this patch tries to
minimize the number of calls, in particular when only the size of the frame rect is needed.

BUG= 811457 
R=chrishtr@chromium.org

Cq-Include-Trybots: master.tryserver.blink:linux_trusty_blink_rel;master.tryserver.chromium.linux:linux_layout_tests_slimming_paint_v2
Change-Id: I8285b91519d461ae5230796f4355e8b814f756cb
Reviewed-on: https://chromium-review.googlesource.com/923427
Reviewed-by: Chris Harrelson <chrishtr@chromium.org>
Commit-Queue: Stefan Zager <szager@chromium.org>
Cr-Commit-Position: refs/heads/master@{#537486}
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/exported/WebPluginContainerImpl.cpp
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/exported/WebPluginContainerImpl.h
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/exported/WebViewImpl.cpp
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/frame/EmbeddedContentView.h
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/frame/FrameViewAutoSizeInfo.cpp
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/frame/LocalFrameView.cpp
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/frame/LocalFrameView.h
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/frame/RemoteFrameView.cpp
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/frame/RemoteFrameView.h
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/layout/LayoutBox.cpp
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/layout/LayoutEmbeddedContent.cpp
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/paint/FramePainter.cpp
[modify] https://crrev.com/cf91f7965c29e2bc93e81f79cef5d77481b00e6e/third_party/WebKit/Source/core/paint/PaintLayerScrollableArea.cpp

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14dd476b840000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/1285e713840000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14e9becb840000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/1488f91b840000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14f37cab840000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/12fbe25b840000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/148dc2ab840000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/12bb48ab840000

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/148dc2ab840000

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/12bb48ab840000

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/12bb48ab840000

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/14e9becb840000

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/12fbe25b840000

Ensure that an OOPIF renderer knows its size before layout. by lfg@chromium.org
https://chromium.googlesource.com/chromium/src/+/e5d27a36230aea4b3664f8b5345b09bb0690a3ff

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/1285e713840000

Ensure that an OOPIF renderer knows its size before layout. by lfg@chromium.org
https://chromium.googlesource.com/chromium/src/+/e5d27a36230aea4b3664f8b5345b09bb0690a3ff

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/14f37cab840000

Ensure that an OOPIF renderer knows its size before layout. by lfg@chromium.org
https://chromium.googlesource.com/chromium/src/+/e5d27a36230aea4b3664f8b5345b09bb0690a3ff

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/14f37cab840000

Pasting from an email:

Last week, we turned on Root Layer Scrolling (RLS) in tip-of-tree.  After we got some bugs, including perf regressions, we turned it off for a couple of days; and then we turned it on again.

Between the time we turned it off and then turned it on for the second time, this OOPIF-related patch landed:

https://chromium-review.googlesource.com/c/chromium/src/+/882021

It now appears that OOPIF patch likely caused some perf regressions that may not have been attributed correctly, because they were sandwiched between our RLS changes.  Here are the relevant commits, in the order they landed:

96f85b68747a679ea1ac4cd05d6743ae5f7142b7 Enable root layer scrolling.
066296a5708f57ba89b68605ae8e65292eb77d4d Revert "Enable root layer scrolling."
e5d27a36230aea4b3664f8b5345b09bb0690a3ff Ensure that an OOPIF renderer knows its size before layout.
39cb84e649e045d2233a395009d5ccf1d08854a0 Enable root layer scrolling

Here's one of the perf-related bugs we got after first turning on RLS:

https://bugs.chromium.org/p/chromium/issues/detail?id=811457

Here's the associated perf builder report:

https://chromeperf.appspot.com/group_report?bug_id=811457

... and here, I've annotated some of the perf reports that seem to show regressions associated with the OOPIF patch:

https://docs.google.com/drawings/d/1xB9T0ThGNgOzK_ux45fot_RSjjjxn_3JCIS9jNw6QvI/edit?usp=sharing

https://docs.google.com/drawings/d/1hNbo7FvAV3InWSakMba8rtpdN_U2-Ghzt2zJzE4XAQI/edit?usp=sharing

We have been doing a bunch of work to address perf regressions associated with RLS, and as you can see, most of the graphs fall off to the right.  At this point, it's unclear how much of the remaining regression is due to RLS, and how much due to the OOPIF patch.

It should be possible to isolate the performance effects of the OOPIF patch by running perf tests at tip of tree; and then running them again with this flag:

 --disable-blink-features=RootLayerScrolling

Alternately, you can apply the patch from the 'Revert "Enabled root layer scrolling."' patch.

📍 Found significant differences after each of 5 commits.
https://pinpoint-dot-chromeperf.appspot.com/job/14dd476b840000

Roll src/third_party/webrtc/ ab86e7ffe..27a0741fd (1 commit) by webrtc-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com
https://chromium.googlesource.com/chromium/src/+/6710a7d0c3c582c0618f8a7a6f26cdf4fec4acd6

Ensure that an OOPIF renderer knows its size before layout. by lfg@chromium.org
https://chromium.googlesource.com/chromium/src/+/e5d27a36230aea4b3664f8b5345b09bb0690a3ff

Roll src/third_party/skia/ a36db6984..1fb2da5e4 (1 commit) by skia-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com
https://chromium.googlesource.com/chromium/src/+/22c5ccd98a8991f0c047df1bc36a217f350316e9

PermissionDialog: Separate the dialog from control by asimjour@chromium.org
https://chromium.googlesource.com/chromium/src/+/28042407186a5000da43bc3eb4cd23ed669f5432

Convert RemoveDataMask to a "class enum" by sdefresne@chromium.org
https://chromium.googlesource.com/chromium/src/+/0bfef394edda0144e539e941b286ec1fa262a0c0

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

📍 Found significant differences after each of 6 commits.
https://pinpoint-dot-chromeperf.appspot.com/job/1488f91b840000

Enable root layer scrolling by bokan@chromium.org
https://chromium.googlesource.com/chromium/src/+/39cb84e649e045d2233a395009d5ccf1d08854a0

[Chromecast] Enable extensions for device builds by achaulk@google.com
https://chromium.googlesource.com/chromium/src/+/e9b35dc1bb4f0cd4aedb9ab98e9c4db0828f7a69

[iOS] Remove unecessary include. by kkhorimoto@chromium.org
https://chromium.googlesource.com/chromium/src/+/337d9df8d547b7cebf84caf4394130237d20fe70

[iOS] Allow ViewControllerSwapping implementors to not be UIViewControllers. by marq@google.com
https://chromium.googlesource.com/chromium/src/+/b66e111357bece445f2917a82d0c29a85c288d06

Print Windows crash codes in hex by brucedawson@chromium.org
https://chromium.googlesource.com/chromium/src/+/8c50b3ed049c736406943ac24dc40fc8ac1e782c

Reland "Adjust location and size of TapDisambiguator" by jaebaek@chromium.org
https://chromium.googlesource.com/chromium/src/+/6de124adeb753f1bb2ce05a8495f817986300fa3

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

Reland "Adjust location and size of TapDisambiguator" by jaebaek@chromium.org
https://chromium.googlesource.com/chromium/src/+/6de124adeb753f1bb2ce05a8495f817986300fa3

changes only the part surrounded by OS_ANDROID
(RenderViewImpl::DidTapMultipleTargets).

If this bug is related to Windows GPU performance, my CL must not be executed by the test related to this bug.

lfg: can you take a look per #28?

lfg@ is OOO this week.

The perf regressions are all in tests with infinite scrollers?

Lucas' CL makes LocalFrameView::FrameRectsChanged() send a message to the browser process notifying it of the new size. Possibly the regression reflects an increased volume of IPCs.

If so, then that behavior is important to a functional bug fix with out-of-process iframes. I can't think of a good way to address that.

+tdresser: do you know who the right person to make a call on the cpu time regressions in this bug per #34?

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/12e87c17840000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/15c89057840000

The magnitude of the regression is large enough that I doubt it's just due to IPC overhead.

It's also interesting that some other stories improved at the same time. Twitter for example improved. I've kicked off a bisect on the improvement in twitter.

I expect that the change causing this regression is also responsible for the improvement in twitter, which seems unlikely for the OOPIF change, right?

I kicked off a perf tryjob reverting RLS (https://pinpoint-dot-chromeperf.appspot.com/job/15c89057840000). Maybe we'll get cleaner results from a tryjob?

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/15c89057840000

📍 Found significant differences after each of 2 commits.
https://chromeperf.appspot.com/job/12e87c17840000

Enable root layer scrolling. by skobes@chromium.org
https://chromium.googlesource.com/chromium/src/+/96f85b68747a679ea1ac4cd05d6743ae5f7142b7

Implement has_more for ReadDirectory by allenvic@chromium.org
https://chromium.googlesource.com/chromium/src/+/88d5add9736bd3fc27c7ef1cb705b13489289a22

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

Assigning to tdresser to help route: The link in the bisect that had results was https://pinpoint-dot-chromeperf.appspot.com/job/12e87c17840000, it shows a big regression at "Enable root layer scrolling" and a much smaller one that may be in the noise for "Implement has_more for ReadDirectory".

That pinpoint job looks like its showing cpu time _decreasing_, doesn't it?

Whoops, yeah, didn't look closely enough at the metric. Still not sure what to do next with this bug.

📍 Pinpoint job started.
https://chromeperf.appspot.com/job/12b154f7840000

https://pinpoint-dot-chromeperf.appspot.com/job/15c89057840000 indicates that reverting root layer scrolling doesn't help.

There are some page load regressions that look unrelated. Triggered a separate bisect there.

Because the other regressions have corresponding improvements elsewhere, I think it wouldn't be crazy to ignore them. Does that sound reasonable?

Yep, sounds reasonable.

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/12b154f7840000

Enable root layer scrolling. by skobes@chromium.org
https://chromium.googlesource.com/chromium/src/+/96f85b68747a679ea1ac4cd05d6743ae5f7142b7

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

Re#28: I agree with your annotations, and I've landed a CL a couple of days ago to address the perf regressions caused by my CL. The scrolling benchmarks have improved noticeably, though they are still not quite to the same levels as before, so it looks like there's still some perf regressions caused by enabling RLS.

lfg@, could you post a link to your recent change?

Re#49: https://chromium-review.googlesource.com/c/chromium/src/+/938651

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/16945ddc440000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14858ef4440000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/15112938440000

📍 Found significant differences after each of 2 commits.
https://pinpoint-dot-chromeperf.appspot.com/job/16945ddc440000

Enable root layer scrolling. by skobes@chromium.org
https://chromium.googlesource.com/chromium/src/+/96f85b68747a679ea1ac4cd05d6743ae5f7142b7

Use LocalFrames in ScrollingCoordinator compositing update. by wjmaclean@chromium.org
https://chromium.googlesource.com/chromium/src/+/0e7b308819ce2a660b1b86393f5fa56ff817c369

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/15112938440000

Enable root layer scrolling. by skobes@chromium.org
https://chromium.googlesource.com/chromium/src/+/96f85b68747a679ea1ac4cd05d6743ae5f7142b7

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/14858ef4440000

Enable root layer scrolling. by skobes@chromium.org
https://chromium.googlesource.com/chromium/src/+/96f85b68747a679ea1ac4cd05d6743ae5f7142b7

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/148bd83c440000

📍 Found significant differences after each of 3 commits.
https://pinpoint-dot-chromeperf.appspot.com/job/148bd83c440000

Revert "Enable root layer scrolling." by skobes@chromium.org
https://chromium.googlesource.com/chromium/src/+/066296a5708f57ba89b68605ae8e65292eb77d4d

Ensure that an OOPIF renderer knows its size before layout. by lfg@chromium.org
https://chromium.googlesource.com/chromium/src/+/e5d27a36230aea4b3664f8b5345b09bb0690a3ff

[test_env.py] Warm up vpython virtualenv cache on swarming task shards. by iannucci@chromium.org
https://chromium.googlesource.com/chromium/src/+/8348d507e280f4bd02a7737bf5577f838d64437e

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

8348d507e280f4bd02a7737bf5577f838d64437e was reverted a while ago

Almost all the alerts here recovered. Assigning to lfg since the ones remaining show an improvement at r540018. Split off the ones that really didn't recover into bug 822877.