eve-paladin isn't running any HWTests (Issue 730076). betty-arc64-paladin provides some test coverage for the CQ, but it would be nice to have a physical DUT to run tests too.

caroline-arc64-release is not running any HWTests either.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/6218fba7b05721054227ba1eb7a24fbdc69ef7d7

commit 6218fba7b05721054227ba1eb7a24fbdc69ef7d7
Author: Nicolas Norvez <norvez@chromium.org>
Date: Fri Sep 29 06:26:18 2017

Add betty-arc64 to Android PFQ

This adds betty-arc64 to the Android PFQ, so far arc64 wasn't covered in
the PFQ.

BUG= chromium:769808 
BUG=chromium:769334
TEST=chromeos_config_unittest

Change-Id: I867745960eb33d96620c49e65e16a21bad161eeb
Reviewed-on: https://chromium-review.googlesource.com/690744
Commit-Ready: Nicolas Norvez <norvez@chromium.org>
Tested-by: Nicolas Norvez <norvez@chromium.org>
Reviewed-by: Bernie Thompson <bhthompson@chromium.org>

[modify] https://crrev.com/6218fba7b05721054227ba1eb7a24fbdc69ef7d7/cbuildbot/config_dump.json
[modify] https://crrev.com/6218fba7b05721054227ba1eb7a24fbdc69ef7d7/cbuildbot/waterfall_layout_dump.txt
[modify] https://crrev.com/6218fba7b05721054227ba1eb7a24fbdc69ef7d7/cbuildbot/chromeos_config.py

https://chromium-review.googlesource.com/690744 has broken the Android PFQ, see comments on the CL:
"
https://uberchromegw.corp.google.com/i/chromeos/builders/master-nyc-android-pfq/builds/1055

betty-arc64-nyc-android-pfq: [status] COMPLETED [result] FAILURE [failure_reason] INVALID_BUILD_DEFINITION
"

xixuan@
Apparently this would be solved by a waterfall restart? Would it be possible to restart it this morning? Otherwise I'll revert the CL.

I am trying for a restart in https://bugs.chromium.org/p/chromium/issues/detail?id=770002

Huh, betty-arc64-nyc-android-pfq is failing:
https://uberchromegw.corp.google.com/i/chromeos/builders/betty-arc64-nyc-android-pfq/builds/0
The same tests are passing in the paladin:
https://uberchromegw.corp.google.com/i/chromeos/builders/betty-arc64-paladin/builds/535

Maybe the PFQ has caught a genuine bug in the last few revisions of ARC++, but that's not very likely. Is there a way to make that builder non-important? IIUC correctly reverting https://chromium-review.googlesource.com/690744 would require a waterfall restart, right?

The chromite code that put it in place would need modified but it should be possible to make it not important. 

Just reverting it would cause it to fail outright, I think that would not require a waterfall restart to unblock if the master would then be ignoring it, its kind of a heavier handed not important.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/caa5449e1c0185a6c304d7e5dd86fa75e37dee62

commit caa5449e1c0185a6c304d7e5dd86fa75e37dee62
Author: Hidehiko Abe <hidehiko@chromium.org>
Date: Mon Oct 02 10:03:46 2017

Revert "Add betty-arc64 to Android PFQ"

This reverts commit 6218fba7b05721054227ba1eb7a24fbdc69ef7d7.

Reason for revert: Continuously failing on bots and blocking the roll.
TBR=norvez@chromium.com
BUG=chromium:770609

Original change's description:
> Add betty-arc64 to Android PFQ
>
> This adds betty-arc64 to the Android PFQ, so far arc64 wasn't covered in
> the PFQ.
>
> BUG= chromium:769808 
> BUG=chromium:769334
> TEST=chromeos_config_unittest
>
> Change-Id: I867745960eb33d96620c49e65e16a21bad161eeb
> Reviewed-on: https://chromium-review.googlesource.com/690744
> Commit-Ready: Nicolas Norvez <norvez@chromium.org>
> Tested-by: Nicolas Norvez <norvez@chromium.org>
> Reviewed-by: Bernie Thompson <bhthompson@chromium.org>

Bug:  chromium:769808 , chromium:769334
Change-Id: I225a9c86d7d3073c575062d42c06750cf4515b32
Reviewed-on: https://chromium-review.googlesource.com/694325
Commit-Ready: Hidehiko Abe <hidehiko@chromium.org>
Tested-by: Hidehiko Abe <hidehiko@chromium.org>
Reviewed-by: Hidehiko Abe <hidehiko@chromium.org>

[modify] https://crrev.com/caa5449e1c0185a6c304d7e5dd86fa75e37dee62/cbuildbot/config_dump.json
[modify] https://crrev.com/caa5449e1c0185a6c304d7e5dd86fa75e37dee62/cbuildbot/waterfall_layout_dump.txt
[modify] https://crrev.com/caa5449e1c0185a6c304d7e5dd86fa75e37dee62/cbuildbot/chromeos_config.py

The betty-arc64-paladin just failed:
https://luci-milo.appspot.com/buildbot/chromeos/betty-arc64-paladin/

This was the first CQ run after the ARC++ uprev, so maybe the PFQ failure was indeed correct and the ARC++ container currently in the tree is bad.

Assuming it's not a fluke and the next CQ run fails again, we should:
- move betty-arc64-paladin to experimental to unblock the CQ (AI for sheriffs)
- fix ARC++ (I will try to repro locally and update the bug)
- re-enable betty-arc64-android-pfq once it's known to work

Still seems to be related to adb issue in http://crbug.com/769334

error logs:

/tmp/cbuildbotNNmiPj/smoke_suite/test_harness/all/SimpleTestVerify/1_autotest_tests/results-25-cheets_ContainerSmokeTest                                                                                  [  FAILED  ]
/tmp/cbuildbotNNmiPj/smoke_suite/test_harness/all/SimpleTestVerify/1_autotest_tests/results-25-cheets_ContainerSmokeTest                                                                                    FAIL: adb is not ready in 60 seconds.
/tmp/cbuildbotNNmiPj/smoke_suite/test_harness/all/SimpleTestVerify/1_autotest_tests/results-25-cheets_ContainerSmokeTest/cheets_ContainerSmokeTest                                                        [  FAILED  ]
/tmp/cbuildbotNNmiPj/smoke_suite/test_harness/all/SimpleTestVerify/1_autotest_tests/results-25-cheets_ContainerSmokeTest/cheets_ContainerSmokeTest                                                          FAIL: adb is not ready in 60 seconds.
/tmp/cbuildbotNNmiPj/smoke_suite/test_harness/all/SimpleTestVerify/1_autotest_tests/results-25-cheets_ContainerSmokeTest/cheets_ContainerSmokeTest                                                          10/02 12:52:29.670 ERROR|         

It's similar, the container is not coming up. I've attached the logcat.

"
10-02 19:24:36.048   630   630 W System  : ClassLoader referenced unknown path: /system/priv-app/DownloadProvider/lib/x86_64
10-02 19:24:36.085   630   630 W System  : ClassLoader referenced unknown path: /system/priv-app/MediaProvider/lib/x86_64
10-02 19:24:36.085   630   630 D ApplicationLoaders: ignored Vulkan layer search path /system/priv-app/MediaProvider/lib/x86_64:/system/lib64:/vendor/lib64 for namespace 0x7e030b2bf0f0
10-02 19:24:36.086   614   614 W System  : ClassLoader referenced unknown path: /system/priv-app/PrebuiltGmsCoreRelease/lib/x86
10-02 19:24:36.101   123   146 I ArcNetworkFactory: Updating mNetworkAgent with: [ Transports: WIFI Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN LinkUpBandwidth>=100000Kbps LinkDnBandwidth>=100000Kbps], [type: ETHERNET[], state: CONNECTED/CONNECTED, reason: (unspecified), extra: (none), failover: false, available: true, roaming: false, metered: false], {InterfaceName: arc0 LinkAddresses: [100.115.92.2/30,fe80::2ff:aaff:fe00:55/64,]  Routes: [100.115.92.0/30 -> 0.0.0.0 arc0,0.0.0.0/0 -> 100.115.92.1 arc0,] DnsAddresses: [10.0.2.3,] Domains: null MTU: 0}
10-02 19:24:36.101   123   199 D ConnectivityService: Invalid tcpBufferSizes string: null, using defaults
10-02 19:24:36.102   123   199 E ConnectivityService: Can't set TCP buffer sizes:java.io.FileNotFoundException: /sys/kernel/ipv4/tcp_rmem_min (Permission denied)
10-02 19:24:36.169   123   301 I ActivityManager: Start proc 647:com.google.process.gapps/u0a14 for content provider com.google.android.gsf/.gservices.GservicesProvider
10-02 19:24:36.216   123   610 I ActivityManager: Low on memory:
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N    80288: zygote (pid 31) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N    41597: zygote64 (pid 29) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N    27512: surfaceflinger (pid 26) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     4900: cameraserver (pid 33) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     4588: audioserver (pid 32) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     4072: mediaserver (pid 40) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     3414: mediadrmserver (pid 38) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     3253: media.extractor (pid 39) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     3218: vold (pid 7) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     2237: drmserver (pid 34) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     2113: /init (pid 1) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     2087: arcbridgeservice (pid 28) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     2016: arctraceservice (pid 4) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     1905: netd (pid 41) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     1880: media.codec (pid 37) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     1734: logd (pid 3) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     1432: keystore (pid 36) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     1262: adbd (pid 210) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     1255: ip6tables-restore (pid 58) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     1177: installd (pid 35) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     1142: iptables-restore (pid 57) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     1096: gatekeeperd (pid 42) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N      860: healthd (pid 23) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N      760: servicemanager (pid 25) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N      724: perfprofd (pid 43) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N      673: logcat (pid 110) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N      649: debuggerd (pid 5) native
10-02 19:24:36.216   123   610 I ActivityManager:   ntv   N     1585: (Other native)
10-02 19:24:36.216   123   610 I ActivityManager:   sys   P    88793: system (pid 123) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   pers  PU   30287: org.chromium.arc.home (pid 316) pers-top-activity
10-02 19:24:36.216   123   610 I ActivityManager:   pers  P    16490: com.android.systemui (pid 267) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   pers  P     8694: com.google.android.apps.work.clouddpc.arc (pid 520) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   pers  P     7116: org.chromium.arc.intent_helper (pid 536) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   pers  P     5968: org.chromium.arc.gms (pid 465) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   pers  P     5864: org.chromium.arc.applauncher (pid 507) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   pers  P     5797: org.chromium.arc.ime (pid 479) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   pers  P     5773: org.chromium.arc.tts (pid 493) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   pers  P     5756: org.chromium.arc.cast_receiver (pid 431) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   pers  P     5736: org.chromium.arc.file_system (pid 440) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   pers  P     5729: org.chromium.arc.crash_collector (pid 426) fixed
10-02 19:24:36.216   123   610 I ActivityManager:   fore  R    11696: com.google.android.marvin.talkback (pid 400) broadcast
10-02 19:24:36.216   123   610 I ActivityManager:   fore  IF    8754: com.android.printspooler (pid 574) service
10-02 19:24:36.216   123   610 I ActivityManager:                        com.android.printspooler/.model.PrintSpoolerService<=Proc{123:system/1000}
10-02 19:24:36.216   123   610 I ActivityManager:   fore  IF    6833: com.android.keychain (pid 569) service
10-02 19:24:36.216   123   610 I ActivityManager:                        com.android.keychain/.KeyChainService<=Proc{123:system/1000}
10-02 19:24:36.216   123   610 I ActivityManager:   vis   SB    6267: com.google.android.ext.services (pid 371) service
10-02 19:24:36.216   123   610 I ActivityManager:                        com.google.android.ext.services/android.ext.services.notification.Ranker<=Proc{123:system/1000}
10-02 19:24:36.216   123   610 I ActivityManager:             424982: TOTAL
10-02 19:24:36.216   123   610 I ActivityManager:   MemInfo:    89,520K slab,     7,372K shmem,         0K vm alloc,    21,748K page tables    11,520K kernel stack
10-02 19:24:36.216   123   610 I ActivityManager:              180,144K buffers,   994,148K cached,   461,408K mapped,   144,828K free
10-02 19:24:36.216   123   610 I ActivityManager:   ZRAM:     5,004K RAM, 2,997,256K swap total, 2,977,632K swap free
10-02 19:24:36.216   123   610 I ActivityManager:   Free RAM:   857,712K
10-02 19:24:36.216   123   610 I ActivityManager:   Used RAM:   555,142K
10-02 19:24:36.216   123   610 I ActivityManager:   Lost RAM:   628,270K
10-02 19:24:36.220   647   647 W System  : ClassLoader referenced unknown path: /system/priv-app/GoogleServicesFrameworkRelease/lib/x86_64
10-02 19:24:36.229   647   647 D AndroidRuntime: Shutting down VM
10-02 19:24:36.251    28    47 I ArcBridgeService: ArcBridgeHostProxy connection for process 520 established
--------- beginning of crash
10-02 19:24:36.254   647   647 E AndroidRuntime: FATAL EXCEPTION: main
10-02 19:24:36.254   647   647 E AndroidRuntime: Process: com.google.process.gapps, PID: 647
10-02 19:24:36.254   647   647 E AndroidRuntime: java.lang.RuntimeException: Unable to get provider com.google.android.gsf.settings.GoogleSettingsProvider: java.lang.ClassNotFoundException: Didn't find class "com.google.android.gsf.settings.GoogleSettingsProvider" on path: DexPathList[[zip file "/system/priv-app/GoogleServicesFrameworkRelease/GoogleServicesFrameworkRelease.apk"],nativeLibraryDirectories=[/system/priv-app/GoogleServicesFrameworkRelease/lib/x86_64, /system/priv-app/GoogleServicesFrameworkRelease/GoogleServicesFrameworkRelease.apk!/lib/x86, /system/lib, /vendor/lib, /system/lib, /vendor/lib]]
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at android.app.ActivityThread.installProvider(ActivityThread.java:6009)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at android.app.ActivityThread.installContentProviders(ActivityThread.java:5598)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5537)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at android.app.ActivityThread.-wrap2(ActivityThread.java)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1601)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at android.os.Handler.dispatchMessage(Handler.java:102)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at android.os.Looper.loop(Looper.java:154)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at android.app.ActivityThread.main(ActivityThread.java:6313)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at java.lang.reflect.Method.invoke(Native Method)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:886)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:776)
10-02 19:24:36.254   647   647 E AndroidRuntime: Caused by: java.lang.ClassNotFoundException: Didn't find class "com.google.android.gsf.settings.GoogleSettingsProvider" on path: DexPathList[[zip file "/system/priv-app/GoogleServicesFrameworkRelease/GoogleServicesFrameworkRelease.apk"],nativeLibraryDirectories=[/system/priv-app/GoogleServicesFrameworkRelease/lib/x86_64, /system/priv-app/GoogleServicesFrameworkRelease/GoogleServicesFrameworkRelease.apk!/lib/x86, /system/lib, /vendor/lib, /system/lib, /vendor/lib]]
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at java.lang.ClassLoader.loadClass(ClassLoader.java:380)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	at android.app.ActivityThread.installProvider(ActivityThread.java:5994)
10-02 19:24:36.254   647   647 E AndroidRuntime: 	... 10 more
10-02 19:24:36.254   647   647 E AndroidRuntime: 	Suppressed: java.io.IOException: No original dex files found for dex location /system/priv-app/GoogleServicesFrameworkRelease/GoogleServicesFrameworkRelease.apk
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at dalvik.system.DexFile.openDexFileNative(Native Method)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at dalvik.system.DexFile.openDexFile(DexFile.java:367)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at dalvik.system.DexFile.<init>(DexFile.java:112)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at dalvik.system.DexFile.<init>(DexFile.java:77)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at dalvik.system.DexPathList.loadDexFile(DexPathList.java:359)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at dalvik.system.DexPathList.makeElements(DexPathList.java:323)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at dalvik.system.DexPathList.makeDexElements(DexPathList.java:263)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at dalvik.system.DexPathList.<init>(DexPathList.java:126)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at dalvik.system.BaseDexClassLoader.<init>(BaseDexClassLoader.java:48)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at dalvik.system.PathClassLoader.<init>(PathClassLoader.java:64)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at com.android.internal.os.PathClassLoaderFactory.createClassLoader(PathClassLoaderFactory.java:43)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at android.app.ApplicationLoaders.getClassLoader(ApplicationLoaders.java:58)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at android.app.LoadedApk.createOrUpdateClassLoaderLocked(LoadedApk.java:520)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at android.app.LoadedApk.getClassLoader(LoadedApk.java:553)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at android.app.ActivityThread.getTopLevelResources(ActivityThread.java:1928)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at android.app.LoadedApk.getResources(LoadedApk.java:766)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at android.app.ContextImpl.<init>(ContextImpl.java:2064)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at android.app.ContextImpl.createAppContext(ContextImpl.java:2009)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5447)
10-02 19:24:36.254   647   647 E AndroidRuntime: 		... 8 more

"

Deleted: logcat_betty-arc64_4371321-r1.log 449 KB

logcat_betty-arc64_4371321-r1.log 449 KB View Download

cheets_ContainerSmokeTest passes locally with the previous version of the container (4366703). Attaching the logcat of the successful run.

Deleted: logcat_betty-arc64-4366703-r1.log 687 KB

logcat_betty-arc64-4366703-r1.log 687 KB View Download

The fact the the PFQ builder (https://luci-milo.appspot.com/buildbot/chromeos/betty-arc64-nyc-android-pfq/) started failing as soon as it was instantiated suggests that the bug is between builds 4366703 and 4368152

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/e90393a957c80b2512ab68ce5cbccb75c3b12ef0

commit e90393a957c80b2512ab68ce5cbccb75c3b12ef0
Author: Nicolas Norvez <norvez@chromium.org>
Date: Fri Oct 06 01:06:46 2017

Reland "Add betty-arc64 to Android PFQ"

This is a reland of 6218fba7b05721054227ba1eb7a24fbdc69ef7d7

ag/3008869 reverted the GmsCore uprev that originally broke betty-arc64.

However this CL should not be submitted until the fix for
crbug.com/771147 has propagated to the PFQ. Even then, the betty-arc64
PFQ builder should be kept experimental until it's been confirmed to be
stable. Setting the tree status to
EXPERIMENTAL=betty-arc64-nyc-android-pfq should do the trick.

Original change's description:
> Add betty-arc64 to Android PFQ
>
> This adds betty-arc64 to the Android PFQ, so far arc64 wasn't covered in
> the PFQ.
>
> BUG= chromium:769808 
> BUG=chromium:769334
> TEST=chromeos_config_unittest
>
> Change-Id: I867745960eb33d96620c49e65e16a21bad161eeb
> Reviewed-on: https://chromium-review.googlesource.com/690744
> Commit-Ready: Nicolas Norvez <norvez@chromium.org>
> Tested-by: Nicolas Norvez <norvez@chromium.org>
> Reviewed-by: Bernie Thompson <bhthompson@chromium.org>

BUG= chromium:769808 
BUG=chromium:769334
BUG=chromium:770609
TEST=chromeos_config_unittest
Change-Id: I85a7772786fb708e7d53a1f5043acf083c2a7949
Reviewed-on: https://chromium-review.googlesource.com/701421
Commit-Ready: Nicolas Norvez <norvez@chromium.org>
Tested-by: Nicolas Norvez <norvez@chromium.org>
Reviewed-by: Bernie Thompson <bhthompson@chromium.org>

[modify] https://crrev.com/e90393a957c80b2512ab68ce5cbccb75c3b12ef0/cbuildbot/config_dump.json
[modify] https://crrev.com/e90393a957c80b2512ab68ce5cbccb75c3b12ef0/cbuildbot/waterfall_layout_dump.txt
[modify] https://crrev.com/e90393a957c80b2512ab68ce5cbccb75c3b12ef0/cbuildbot/chromeos_config.py