Similar failures:

compositing/squashing/dont-squash-into-blend-mode.html
compositing/squashing/dont-squash-with-scale-transform.html

2 additional layer "Scrolling Layer" and "Scrolling Contents Layer" are formed for RLS

so if we increase the index of the layer that are present in the assert, the test ran as expected 

dont-squash-into-animated-layers.html

-        assert_true(layers[2].compositingReasons[1] == "Layer was separately composited because it could not be squashed.");
-        assert_true(layers[2].squashingDisallowedReasons[0] == "Cannot squash into a layer that is animating.");
+        assert_true(layers[4].compositingReasons[1] == "Layer was separately composited because it could not be squashed.");
+        assert_true(layers[4].squashingDisallowedReasons[0] == "Cannot squash into a layer that is animating.");

we need to change the index for the layers by 2

we have to change the test case w.r.t RLS, Have to change layer index by 2. for all the 3 - test cases mentioned. 

Nice find. You can change the test to use layer 2 for !RLS and layer 4 for RLS. In javascript you can use the following:
var layer = internals.runtimeFlags.rootLayerScrollingEnabled ? layers[4] : layers[2];

submitted the changes in 
https://chromium-review.googlesource.com/c/chromium/src/+/753041

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e1b77f69f0191c5171b2e7caa2e23ec54fc1a03e

commit e1b77f69f0191c5171b2e7caa2e23ec54fc1a03e
Author: Suneel Kota <suneel.kota@samsung.com>
Date: Fri Nov 03 16:03:23 2017

[RLS] Make compositing/sqashing tests RLS aware

When RLS is enabled additional layer "Scrolling Layer" and
"Scrolling Contents Layer" are formed

hence update the index of the layers accordingly

Bug:  776974 
Change-Id: Iae0bfc76d4da847b2d753fd2cbe95eeb77846a93
Reviewed-on: https://chromium-review.googlesource.com/753041
Reviewed-by: Mu Ven <sataya.m@samsung.com>
Reviewed-by: Philip Rogers <pdr@chromium.org>
Commit-Queue: Philip Rogers <pdr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#513794}
[modify] https://crrev.com/e1b77f69f0191c5171b2e7caa2e23ec54fc1a03e/third_party/WebKit/LayoutTests/FlagExpectations/root-layer-scrolls
[modify] https://crrev.com/e1b77f69f0191c5171b2e7caa2e23ec54fc1a03e/third_party/WebKit/LayoutTests/compositing/squashing/dont-squash-into-animated-layers.html
[modify] https://crrev.com/e1b77f69f0191c5171b2e7caa2e23ec54fc1a03e/third_party/WebKit/LayoutTests/compositing/squashing/dont-squash-into-blend-mode.html
[modify] https://crrev.com/e1b77f69f0191c5171b2e7caa2e23ec54fc1a03e/third_party/WebKit/LayoutTests/compositing/squashing/dont-squash-with-scale-transform.html

Thanks for the fix!