All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=903754

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=ed04409b10998f7b601a7cee7454a4958673f6ac2f83c730ba403888a82991e9


Bot(s) for this bug's original alert(s):

linux-perf

system_health.memory_desktop - Benchmark documentation link:
  https://bit.ly/system-health-benchmarks

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/1260b853e40000

📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/1260b853e40000

Promote PaintTouchActionRects to stable by pdr@chromium.org
https://chromium.googlesource.com/chromium/src/+/564c80e84a5c6ec370e85ce1890446da108687c4
memory:chrome:all_processes:reported_by_chrome:skia:effective_size: 6.448e+05 → 6.887e+05 (+4.389e+04)

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions

Benchmark documentation link:
  https://bit.ly/system-health-benchmarks

Xianda's going to take a look. I think we just need to run the test locally and dump the hit test rects and see if something pathalogical is happening. I wouldn't expect much memory to be used for PTAR. The hit test display items are drawing display items and it's possible that this regression is due to more display items being sent to cc.

Issue 903313 has been merged into this issue.

 Issue 903208  has been merged into this issue.

I ran the test locally and put a logging in ScrollingCoordinator, and dumped these results, looks like we have a layer with a ton of touch action rects.

[1:1:1110/152922.622367:ERROR:scrolling_coordinator.cc(820)] Update all rects =======================
[1:1:1110/152922.622539:ERROR:scrolling_coordinator.cc(274)] Update layer "Scrolling Contents Layer"
[1:1:1110/152922.622614:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0 2514x62616)]}"
[1:1:1110/152922.622700:ERROR:scrolling_coordinator.cc(274)] Update layer "LayoutBlockFlow CODE class='hljs bash'"
[1:1:1110/152922.622778:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0.25 1380x332)]}"
[1:1:1110/152922.622859:ERROR:scrolling_coordinator.cc(274)] Update layer "Scrolling Contents Layer"
[1:1:1110/152922.622923:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0.25 4665.47x302), (14,22.25 100.813x22), (14,61.25 336.047x22), (14,100.25 3108.44x22), (14,140.25 1108.95x22), (954.953,138.25 33.5938x24), (14,179.25 4637.47x22), (14,218.25 1075.38x22), (14,257.25 1781.08x22)]}"
[1:1:1110/152922.623053:ERROR:scrolling_coordinator.cc(274)] Update layer "LayoutBlockFlow CODE class='lang-nohighlight'"
[1:1:1110/152922.623124:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0.25 1380x1020)]}"
[1:1:1110/152922.623200:ERROR:scrolling_coordinator.cc(274)] Update layer "Scrolling Contents Layer"
[1:1:1110/152922.623264:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0.25 6677.08x9341), (20,18.25 655.297x22), (20,57.25 252.031x22), (20,96.25 504.078x22), (20,135.25 369.656x22), (20,174.25 537.688x22), (20,213.25 50.4063x22), (20,252.25 772.922x22), (20,291.25 319.25x22), (20,330.25 1041.77x22), (20,369.25 1310.61x22), (20,408.25 789.719x22), (20,447.25 1562.64x22), (20,486.25 789.719x22), (20,525.25 1327.41x22), (20,564.25 789.719x22), (20,603.25 1260.2x22), (20,642.25 789.719x22), (20,681.25 1596.25x22), (20,720.25 789.719x22), (20,759.25 3007.69x22), (20,798.25 1949.11x22), (20,837.25 789.719x22), (20,876.25 1344.22x22), (20,915.25 789.719x22), (20,954.25 1377.81x22), (20,993.25 789.719x22), (20,1032.25 1663.47x22), (20,1071.25 789.719x22), (20,1110.25 1915.5x22), (20,1149.25 789.719x22), (20,1188.25 1512.23x22), (20,1227.25 1529.05x22), (20,1266.25 1461.83x22), (20,1305.25 1529.05x22), (20,1344.25 789.719x22), (20,1383.25 4032.66x22), (20,1422.25 789.719x22), (20,1461.25 1629.86x22), (20,1500.25 789.719x22), (20,1539.25 1596.25x22), (20,1578.25 789.719x22), (20,1617.25 1192.98x22), (20,1656.25 789.719x22), (20,1695.25 1243.39x22), (20,1734.25 789.719x22), (20,1773.25 3259.72x22), (20,1812.25 2688.44x22), (20,1851.25 3074.89x22), (20,1890.25 2873.27x22), (20,1929.25 2570.81x22), (20,1968.25 3511.77x22), (20,2007.25 1932.31x22), (20,2046.25 789.719x22), (20,2085.25 2923.67x22), (20,2124.25 2621.22x22), (20,2163.25 3125.3x22), (20,2202.25 3024.48x22), (20,2241.25 3478.16x22), (20,2280.25 3024.48x22), (20,2319.25 2688.44x22), (20,2358.25 3780.61x22), (20,2397.25 940.953x22), (20,2436.25 1562.64x22), (20,2475.25 1545.84x22), (20,2514.25 1394.63x22), (20,2553.25 1075.38x22), (20,2592.25 789.719x22), (20,2631.25 1545.84x22), (20,2670.25 1545.84x22), (20,2709.25 1461.83x22), (20,2748.25 67.2031x22), (20,2787.25 1327.41x22), (20,2826.25 789.719x22), (20,2865.25 1949.11x22), (20,2904.25 1495.44x22), (20,2943.25 1411.42x22), (20,2982.25 655.297x22), (20,3021.25 789.719x22), (20,3060.25 2520.41x22), (20,3099.25 873.734x22), (20,3138.25 0x22), (20,3177.25 2100.33x22), (20,3216.25 991.359x22), (20,3255.25 0x22), (20,3294.25 2083.53x22), (20,3333.25 873.734x22), (20,3372.25 1058.56x22), (20,3411.25 0x22), (20,3450.25 2100.33x22), (20,3489.25 890.531x22), (20,3528.25 1058.56x22), (20,3567.25 0x22), (20,3606.25 2083.53x22), (20,3645.25 873.734x22), (20,3684.25 1058.56x22), (20,3723.25 0x22), (20,3762.25 2100.33x22), (20,3801.25 890.531x22), (20,3840.25 1058.56x22), (20,3879.25 0x22), (20,3918.25 1344.22x22), (20,3957.25 3108.5x22), (20,3996.25 0x22), (20,4035.25 1764.28x22), (20,4074.25 1613.06x22), (20,4113.25 1226.59x22), (20,4152.25 1108.97x22), (20,4191.25 0x22), (20,4230.25 1461.83x22), (20,4269.25 1478.64x22), (20,4308.25 1697.06x22), (20,4347.25 1797.89x22), (20,4386.25 1747.48x22), (20,4425.25 789.719x22), (20,4464.25 1965.91x22), (20,4503.25 2117.14x22), (20,4542.25 1159.38x22), (20,4581.25 1041.77x22), (20,4620.25 0x22), (20,4659.25 3074.89x22), (20,4698.25 789.719x22), (20,4737.25 3125.3x22), (20,4776.25 789.719x22), (20,4815.25 1478.64x22), (20,4854.25 1697.06x22), (20,4893.25 1495.44x22), (20,4932.25 1461.83x22), (20,4971.25 1478.64x22), (20,5010.25 1512.23x22), (20,5049.25 2150.75x22), (20,5088.25 2033.13x22), (20,5127.25 1512.23x22), (20,5166.25 1881.91x22), (20,5205.25 2638.02x22), (20,5244.25 1058.56x22), (20,5283.25 4704.77x22), (20,5322.25 6637.08x22), (20,5361.25 1041.77x22), (20,5400.25 1445.03x22), (20,5439.25 1898.7x22), (20,5478.25 0x22), (20,5517.25 1445.03x22), (20,5556.25 789.719x22), (20,5595.25 1965.91x22), (20,5634.25 789.719x22), (20,5673.25 1377.81x22), (20,5712.25 722.516x22), (20,5751.25 806.531x22), (20,5790.25 1243.39x22), (20,5829.25 772.922x22), (20,5868.25 873.734x22), (20,5907.25 907.344x22), (20,5946.25 806.531x22), (20,5985.25 907.344x22), (20,6024.25 940.953x22), (20,6063.25 772.922x22), (20,6102.25 688.906x22), (20,6141.25 688.906x22), (20,6180.25 688.906x22), (20,6219.25 856.938x22), (20,6258.25 856.938x22), (20,6297.25 856.938x22), (20,6336.25 890.531x22), (20,6375.25 856.938x22), (20,6414.25 823.328x22), (20,6453.25 1209.8x22), (20,6492.25 201.625x22), (20,6531.25 1008.16x22), (20,6570.25 0x22), (20,6609.25 1730.67x22), (20,6648.25 722.516x22), (20,6687.25 789.719x22), (20,6726.25 789.719x22), (20,6765.25 1663.47x22), (20,6804.25 655.297x22), (20,6843.25 2049.92x22), (20,6882.25 0x22), (20,6921.25 1529.05x22), (20,6960.25 789.719x22), (20,6999.25 1579.45x22), (20,7038.25 789.719x22), (20,7077.25 1545.84x22), (20,7116.25 789.719x22), (20,7155.25 1999.52x22), (20,7194.25 789.719x22), (20,7233.25 3108.5x22), (20,7272.25 789.719x22), (20,7311.25 3494.97x22), (20,7350.25 789.719x22), (20,7389.25 2150.75x22), (20,7428.25 789.719x22), (20,7467.25 2587.61x22), (20,7506.25 789.719x22), (20,7545.25 2537.2x22), (20,7584.25 789.719x22), (20,7623.25 2049.92x22), (20,7662.25 2335.58x22), (20,7701.25 537.688x22), (20,7740.25 789.719x22), (20,7779.25 2150.75x22), (20,7818.25 789.719x22), (20,7857.25 1411.42x22), (20,7896.25 789.719x22), (20,7935.25 1243.39x22), (20,7974.25 789.719x22), (20,8013.25 1982.72x22), (20,8052.25 2822.86x22), (20,8091.25 2285.17x22), (20,8130.25 2772.44x22), (20,8169.25 1277x22), (20,8208.25 1125.78x22), (20,8247.25 0x22), (20,8286.25 1545.84x22), (20,8325.25 1176.19x22), (20,8364.25 940.953x22), (20,8403.25 1176.19x22), (20,8442.25 0x22), (20,8481.25 1243.39x22), (20,8520.25 4133.47x22), (20,8559.25 1125.78x22), (20,8598.25 1159.38x22), (20,8637.25 1176.19x22), (20,8676.25 840.125x22), (20,8715.25 2419.59x22), (20,8754.25 1192.98x22), (20,8793.25 1092.17x22), (20,8832.25 907.344x22), (20,8871.25 1226.59x22), (20,8910.25 739.313x22), (20,8949.25 1108.97x22), (20,8988.25 0x22), (20,9027.25 0x22), (20,9066.25 100.813x22), (20,9105.25 336.047x22), (20,9144.25 1865.09x22), (20,9183.25 1108.97x22), (20,9222.25 1965.91x22), (20,9261.25 1075.38x22), (20,9300.25 1781.08x22)]}"
[1:1:1110/152922.625344:ERROR:scrolling_coordinator.cc(274)] Update layer "Scroll Corner Layer"
[1:1:1110/152922.625424:ERROR:scrolling_coordinator.cc(274)] Update layer "LayoutBlockFlow CODE class='hljs php'"
[1:1:1110/152922.625492:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0.25 1322x176)]}"
[1:1:1110/152922.625570:ERROR:scrolling_coordinator.cc(274)] Update layer "Scrolling Contents Layer"
[1:1:1110/152922.625633:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0.25 1624.2x146), (14,22.25 1596.2x22), (14,62.25 1176.17x22), (148.422,60.25 84x24), (14,101.25 940.953x22)]}"
[1:1:1110/152922.625724:ERROR:scrolling_coordinator.cc(274)] Update layer "LayoutBlockFlow (positioned) DIV class='timeline-container'"
[1:1:1110/152922.625796:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0 384x776), (84,28 300x706), (84,28 300x706), (84,28 300x39), (84,681 300x53), (84,32 118.281x31), (84,699 73.1719x31), (84,695 73.1719x39), (84,699 73.1719x31)]}"
[1:1:1110/152922.625900:ERROR:scrolling_coordinator.cc(274)] Update layer "LayoutBlockFlow (relative positioned) DIV class='timeline-scrollarea'"
[1:1:1110/152922.625974:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0 286x600), (2,0 284x42.7031), (-3.3125,42.7031 289.313x100), (2,142.703 284x457.281), (6.48438,53.7031 143.219x78), (34.4844,53.7031 115.219x39), (34.4844,92.7031 115.219x39), (34.4844,57.7031 101.188x31), (34.4844,96.7031 115.203x31)]}"
[1:1:1110/152922.626108:ERROR:scrolling_coordinator.cc(274)] Update layer "LayoutFlexibleBox (positioned) DIV id='reply-control' class='closed show-preview ember-view processed'"
[1:1:1110/152922.626191:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0 2514x0)]}"
[1:1:1110/152922.626262:ERROR:scrolling_coordinator.cc(274)] Update layer "LayoutBlockFlow (positioned) HEADER class='d-header clearfix'"
[1:1:1110/152922.626332:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(0,0 2514x125.984), (131,6 2252x116), (147,22 2220x84), (243.203,22 1808.69x84)]}"
[1:1:1110/152922.626415:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(243.203,22 1808.69x84), (243.203,22 1627.81x84)]}"
[1:1:1110/152922.626499:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(147,22 96.2031x79.9844), (147,44.2344 96.2031x31)]}"
[1:1:1110/152922.626569:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(243.203,22 1627.81x50), (243.203,22 1627.81x50)]}"
[1:1:1110/152922.626639:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(243.203,23 392.906x47)]}"
[1:1:1110/152922.626706:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(2051.89,22 315.109x77.9844), (2051.89,60.9844 0x0), (2051.89,60.9844 0x0), (2051.89,60.9844 0x0), (2051.89,37.2813 145.141x53), (2051.89,46.2813 145.141x31), (2065.89,37.2813 131.141x53), (2085.89,49.2813 91.1406x29), (2085.89,50.2813 91.125x27), (2085.89,49.2813 17.4063x29), (2085.89,51.2813 17.3906x24), (2207.03,22 159.969x77.9844), (2207.03,99.9844 0x0), (2207.03,99.9844 0x0), (2207.03,99.9844 0x0), (2207.03,22 79.9844x77.9844), (2207.03,22 79.9844x77.9844)]}"
[1:1:1110/152922.626871:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(2287.02,22 79.9844x77.9844), (2287.02,22 79.9844x77.9844)]}"
[1:1:1110/152922.626946:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(243.203,72 101.063x34), (243.203,78 18x18), (271.203,72 73.0625x34)]}"
[1:1:1110/152922.627025:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(271.203,72 73.0625x34)]}"
[1:1:1110/152922.627093:ERROR:scrolling_coordinator.cc(279)] "{touch_action_rects: [(271.203,75 73.0469x27)]}"

I visualize the touch action rects in this jsfiddle: http://jsfiddle.net/avm3rxbc/, looks like each text div contributes to a touch action rect.

Actually I updated the fiddle to http://jsfiddle.net/avm3rxbc/1/.
Now it's easy to see that there is one big touch action rect (the blue one) and a ton of red touch action rect, all of the rects have kTouchActionNone. It looks like we don't need to submit the red ones.

There 240 touch action rects submitted in this single paint chunk.

If we really want to fix the "too many rects" perf bug, we need to alter the way cc hit tests touch event. Now cc only finds the topmost and closest matching layer that is hit by the touch event (which is one of the red rects in this case), and we only check this |matching_layer|'s whitelisted_touch_action.

It is not trivial to fix it on cc side: if we don't submit red hit test rect for descendant layers, in cc we don't know whether these layers have touch-action: auto that cancels the ancestor's touch-action-none property or they should inherit it.

I'm cc-ing xida to verify the cc's touch action logic.

Is my understanding correct that all these red rects are actually inside that blue rect? If that's the case, CC would get a region with one rect in it. You can see this logic in:
https://cs.chromium.org/chromium/src/third_party/blink/renderer/core/page/scrolling/scrolling_coordinator.cc?sq=package:chromium&dr=CSs&g=0&l=293

In there, we call HitTestRect::BuildRegion which does union of all rects to build a region. In this case, the region would have the big blue rect only. sunxd@: please double check and make sure that this is correct.

pdr@: I want to say this is working as intended. Whenever a RecordHitTestData is called, a new HitTestDisplayItem is getting created and I think that's the cause of this regression. I can't think of a way to not create this new HitTestDisplayItem without sacrificing performance.

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/168edb1be40000

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14a079cfe40000

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/14a079cfe40000

All of the runs failed. The most common error (1/20 runs) was:
IOError: [Errno 2] No such file or directory: '/b/s/w/itQz1XpM/tmpRo_qkAtelemetry/histograms.json'

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/168edb1be40000

All of the runs failed. The most common error (1/20 runs) was:
IOError: [Errno 2] No such file or directory: '/b/s/w/itGZI7lI/tmpBI3r9ctelemetry/histograms.json'

Nice investigation! I agree that the expected behavior is a lot of rects.

The perf benchmark shows a regression of around 45,000bytes which seems like a lot for just ~300 rects.

Given what has been said above, I wonder if we're incorrectly sending the rects to cc both in the display item list and in the region on the cc::Layer? Xianzhu has a patch out for review that will fix this by making hit test display items non-drawing. I recommend we wait for Xianzhu's patch to land and then revisit this. If Xianzhu's patch does not fix the regression, a next step would be to find out exactly where memory is going (i.e., print out the display item list, paint chunk and cc Layer data structure memory on this testcase with and without PTAR).

📍 Pinpoint job started.
https://pinpoint-dot-chromeperf.appspot.com/job/14b712dfe40000

😿 Pinpoint job stopped with an error.
https://pinpoint-dot-chromeperf.appspot.com/job/14b712dfe40000

All of the runs failed. The most common error (1/20 runs) was:
IOError: [Errno 2] No such file or directory: '/b/s/w/itTPR181/tmpSsPyjetelemetry/histograms.json'

This pinpoint job succeeded and shows no perf regression with enabling PTAR: https://pinpoint-dot-chromeperf.appspot.com/job/17c2e780140000
(system_health.memory_desktop browse.tech.discourse.infinite.scroll.2018)

I think this perf issue may have been resolved. I confirmed the benchmark was previously regressing and no longer is with PTAR.

I think we can call this fixed and re-land PTAR.