platform_FilePerms has started failing flakily on Pre-CQ as well. e.g. https://luci-milo.appspot.com/buildbot/chromiumos.tryserver/pre_cq/43289

Recently, a Chrome change started ARC++ on the login screen: http://google.com/url?sa=D&q=https%3A//chromium-review.googlesource.com/c/563868/ - this was picked up by ChromeOS in 61.0.3154.0 as per https://chromium.googlesource.com/chromium/src/+log/61.0.3153.0..61.0.3154.0?pretty=fuller&n=10000

Bumping the priority here - eventually we should fix the test to work properly when ARC++ is running (e.g. proper expectations for /run/arc/debugfs/sync and /run/arc/debugfs/tracing as per the report here) so the test doesn't flake based on timing of the test and ARC++ startup.

(It also wouldn't surprise me if we decide to first revert the Chrome change until we sort this out, and issues with other tests like in crbug.com/741110)

 Issue 741080  has been merged into this issue.

I've hit this consistently on Pre-CQ, blocking all but one of my CLs from going in today. I think this should be a P0.

Is there a known CL to revert? Adding sheriffs, deputies,

That would be https://chromium-review.googlesource.com/c/563868/, I think.

Sheriff here.  I'm reverting a CL (the one mentioned in #6) that starts the Android container on login.  It's not the root cause per se but it's throwing off the timing in the tests can causing them to flake.  It's working it's way through the waterfalls now.

Sorry, that isn't the right duplicate.  But the root cause should be the same.

This test is "somewhat" broken on kevin as well

/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms                    [  FAILED  ]
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms                      FAIL: Found 18 permission errors
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms [  FAILED  ]
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   FAIL: Found 18 permission errors
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.139 ERROR|platform_FilePerms:0426| No expectations entry for "/home/chronos/u-61667275ba80301feb06c69fb27f7d8e3a78b1a7" with info "{'device': '/dev/mmcblk0p1', 'type': 'ext4', 'options': ['rw', 'seclabel', 'nosuid', 'nodev', 'noexec', 'noatime', 'commit=600', 'data=ordered']}"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.148 ERROR|platform_FilePerms:0426| No expectations entry for "/run/arc/sdcard/read/emulated" with info "{'device': '/dev/fuse', 'type': 'fuse', 'options': ['rw', 'nosuid', 'nodev', 'noexec', 'noatime', 'user_id=656383', 'group_id=656383', 'default_permissions', 'allow_other']}"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.159 ERROR|platform_FilePerms:0426| No expectations entry for "/media/removable/OEM" with info "{'device': '/dev/mmcblk1p8', 'type': 'ext4', 'options': ['rw', 'dirsync', 'seclabel', 'nosuid', 'nodev', 'noexec', 'relatime', 'data=ordered']}"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.168 ERROR|platform_FilePerms:0433| [/etc/mtab] "/run/containers/android/root/data" has type "ext4", expected type "ext2"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.178 ERROR|platform_FilePerms:0441| [/etc/mtab] "/run/containers/android/root/data" is device "/dev/mmcblk0p1", expected device "/dev/root$"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.186 ERROR|platform_FilePerms:0453| [/etc/mtab] "/run/containers/android/root/data" is missing options "set(['ro'])"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.197 ERROR|platform_FilePerms:0426| No expectations entry for "/home/root/61667275ba80301feb06c69fb27f7d8e3a78b1a7" with info "{'device': '/dev/mmcblk0p1', 'type': 'ext4', 'options': ['rw', 'seclabel', 'nosuid', 'nodev', 'noexec', 'noatime', 'commit=600', 'data=ordered']}"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.207 ERROR|platform_FilePerms:0453| [/etc/mtab] "/run/arc/debugfs/tracing" is missing options "set(['gid=605', 'mode=750'])"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.217 ERROR|platform_FilePerms:0426| No expectations entry for "/home/user/61667275ba80301feb06c69fb27f7d8e3a78b1a7" with info "{'device': '/dev/mmcblk0p1', 'type': 'ext4', 'options': ['rw', 'seclabel', 'nosuid', 'nodev', 'noexec', 'noatime', 'commit=600', 'data=ordered']}"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.229 ERROR|platform_FilePerms:0426| No expectations entry for "/run/arc/sdcard/write/emulated" with info "{'device': '/dev/fuse', 'type': 'fuse', 'options': ['rw', 'nosuid', 'nodev', 'noexec', 'noatime', 'user_id=656383', 'group_id=656383', 'default_permissions', 'allow_other']}"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.241 ERROR|platform_FilePerms:0426| No expectations entry for "/home/chronos/user" with info "{'device': '/dev/mmcblk0p1', 'type': 'ext4', 'options': ['rw', 'seclabel', 'nosuid', 'nodev', 'noexec', 'noatime', 'commit=600', 'data=ordered']}"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.249 ERROR|platform_FilePerms:0426| No expectations entry for "/media/removable/STATE" with info "{'device': '/dev/mmcblk1p1', 'type': 'ext4', 'options': ['rw', 'dirsync', 'seclabel', 'nosuid', 'nodev', 'noexec', 'relatime', 'data=ordered']}"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.259 ERROR|platform_FilePerms:0426| No expectations entry for "/run/arc/media/removable" with info "{'device': 'passthrough', 'type': 'fuse', 'options': ['rw', 'nosuid', 'nodev', 'relatime', 'user_id=1000', 'group_id=1000', 'default_permissions', 'allow_other']}"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.267 ERROR|platform_FilePerms:0433| [/etc/mtab] "/run/containers/android/root/cache" has type "ext4", expected type "ext2"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.277 ERROR|platform_FilePerms:0441| [/etc/mtab] "/run/containers/android/root/cache" is device "/dev/mmcblk0p1", expected device "/dev/root$"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.286 ERROR|platform_FilePerms:0453| [/etc/mtab] "/run/containers/android/root/cache" is missing options "set(['ro'])"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.296 ERROR|platform_FilePerms:0426| No expectations entry for "/run/arc/sdcard/default/emulated" with info "{'device': '/dev/fuse', 'type': 'fuse', 'options': ['rw', 'nosuid', 'nodev', 'noexec', 'noatime', 'user_id=656383', 'group_id=656383', 'default_permissions', 'allow_other']}"
/tmp/test_that_results_7sz0wa/results-1-platform_FilePerms/platform_FilePerms   07/11 16:43:44.304 ERROR|platform_FilePerms:0426| No expectations entry for "/opt/google/containers/android/rootfs/android-data" with info "{'device': '/dev/mmcblk0p1', 'type': 'ext4', 'options': ['rw', 'seclabel', 'nosuid', 'nodev', 'noexec', 'noatime', 'commit=600', 'data=ordered']}"

Oh it also detects mounts from an SD card I had inserted in my kevin and treats them as failures. 

> (It also wouldn't surprise me if we decide to first revert the Chrome change
> until we sort this out, and issues with other tests like in crbug.com/741110)

Agreed. Let me revert the Chromium change first to unblock Pre-CQ.

Revert is in CQ https://chromium-review.googlesource.com/c/566585/

Oh I see, thanks. I have abandoned mine at https://chromium-review.googlesource.com/c/567552 .

comment #10-#11
This is probably because you ran the test while your kevin was in user session. Please run the test after signing out from the user session, then the test would likely pass (and that's what auto tests on the builders do, IIUC.)

It seems that fixing this test to run while user is logged in with ARC running is not going to be very simple.

Tentative fix here: https://chromium-review.googlesource.com/#/c/567242/

Do we need to do anything for this Chrome change to make it over to the PreCQ bots?

Can the tests be temporarily disabled? This is blocking my ability to land unrelated changes.

This blocked my ability to land changes for all of yesterday as well. It seems like we should be at least pinning Chrome to a previous version and not hoping the PFQ will pass.

Seeing as we have a fix for this one (comment #16), should we just consider chumping it in rather than disabling? (not sure why CQ / pre-CQ hasn't picked it up...but I +2'd patchset 3 just in case that was what caused it be stuck)

If the test CL is passing locally I'd be OK with chumping. We could also chump https://chromium-review.googlesource.com/c/568358 for security_ASLR.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/4ef5136d51734a2d5aab5624737c03ec69297b07

commit 4ef5136d51734a2d5aab5624737c03ec69297b07
Author: Dominik Behr <dbehr@chromium.org>
Date: Wed Jul 12 15:50:02 2017

platform_FilePerms: fix sync and tracing debugfs expectations

And little bit of ARC mounts.

BUG= chromium:741027 
TEST=run platform_FilePerms on kevin login screen.

Change-Id: I0ecb16e61a57e58e81e82ae1b9927382f03d9afc
Signed-off-by: Dominik Behr <dbehr@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/567242
Reviewed-by: Josh Horwich <jhorwich@chromium.org>

[modify] https://crrev.com/4ef5136d51734a2d5aab5624737c03ec69297b07/client/site_tests/platform_FilePerms/platform_FilePerms.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/446a29df8cc31ed3356ae2434a5b480132a9f302

commit 446a29df8cc31ed3356ae2434a5b480132a9f302
Author: Sean Kau <skau@chromium.org>
Date: Wed Jul 12 17:43:39 2017

Chrome: Pin to version 61.0.3153.0_rc-r1

DO NOT REVERT THIS CL.
In general, reverting chrome (un)pin CLs does not do what you expect.
Instead, use `cros pinchrome` to generate new CLs.

BUG= 741027 
TEST=None
CQ-DEPEND=*I47a3ffcfd9109dcb6041a03d5e248bde11964e1f

Change-Id: I81a13cc9ef51a058f8763ded7fe8c7681b051a93
Reviewed-on: https://chromium-review.googlesource.com/567723
Reviewed-by: Sean Kau <skau@chromium.org>
Commit-Queue: Sean Kau <skau@chromium.org>
Tested-by: Sean Kau <skau@chromium.org>
Trybot-Ready: Sean Kau <skau@chromium.org>

[add] https://crrev.com/446a29df8cc31ed3356ae2434a5b480132a9f302/profiles/default/linux/package.mask/chromepin
[modify] https://crrev.com/446a29df8cc31ed3356ae2434a5b480132a9f302/chromeos-base/chromeos-chrome/Manifest
[modify] https://crrev.com/446a29df8cc31ed3356ae2434a5b480132a9f302/chromeos/binhost/target/amd64-generic-LATEST_RELEASE_CHROME_BINHOST.conf
[modify] https://crrev.com/446a29df8cc31ed3356ae2434a5b480132a9f302/chromeos/binhost/target/daisy-LATEST_RELEASE_CHROME_BINHOST.conf
[modify] https://crrev.com/446a29df8cc31ed3356ae2434a5b480132a9f302/chromeos/binhost/target/arm-generic-LATEST_RELEASE_CHROME_BINHOST.conf
[rename] https://crrev.com/446a29df8cc31ed3356ae2434a5b480132a9f302/chromeos-base/chromium-source/chromium-source-61.0.3153.0_rc-r1.ebuild
[modify] https://crrev.com/446a29df8cc31ed3356ae2434a5b480132a9f302/chromeos/binhost/target/veyron_jerry-LATEST_RELEASE_CHROME_BINHOST.conf
[rename] https://crrev.com/446a29df8cc31ed3356ae2434a5b480132a9f302/chromeos-base/chromeos-chrome/chromeos-chrome-61.0.3153.0_rc-r1.ebuild
[modify] https://crrev.com/446a29df8cc31ed3356ae2434a5b480132a9f302/chromeos/binhost/host/amd64-LATEST_RELEASE_CHROME_BINHOST.conf

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/overlays/chromeos-partner-overlay/+/8daedd65043d96ff93001930679aac7121417ffc

commit 8daedd65043d96ff93001930679aac7121417ffc
Author: Sean Kau <skau@chromium.org>
Date: Wed Jul 12 18:12:18 2017

Re #24: For future reference, please inform the gardeners and cc' them when you pin chrome :)

+derat@, +glevin@

It looks like there may be a fix in comment #23, do we have reason to suspect that will work?

security_ASLR was also failing because of starting ARC++ on the login screen, but we think that has been fixed as well.

Since we landed a fix for the test (comment #23), should we consider this issue fixed now?