+jam: didn't we add an Android bot that runs tests with PlzNavigate already?

I'm talking about Android _trybot_ that will help catch bugs before landing. I'd like to be able to detect any issues sooner than later.

We didn't have capacity to run it on the CQ automatically due to device constraints.
It runs on the Android waterfall which I'm told is sheriffed: https://build.chromium.org/p/chromium.android/builders/Marshmallow%2064%20bit%20Tester?numbuilds=200
and there are optional trybots: android_n5x_swarming_dbg

Thanks for letting me know. I'll drop the CL and resort to the optional try bot.

android_n5x_swarming_dbg doesn't even run due to errors in its trybot configuration. I'll fix it to be able to run it optionally.

android_n5x_swarming_dbg is not really in a good state. Many other seemingly random tests fail or are flaky. It may need substantial effort to put up to the level where I feel comfortable running to get reliable tests result on plzNavigate. 

I'll keep this Issue open for now. Will pick up some frequently failing tests and look into them whenever my bandwidth allows.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3f7c90c00159bbd0d3b56a5164206417b89b3bab

commit 3f7c90c00159bbd0d3b56a5164206417b89b3bab
Author: Jinsuk Kim <jinsukkim@chromium.org>
Date: Sun Jul 16 19:43:50 2017

Fix android_n5x_swarming_dbg trybot config

This CL is to take care of errors when running an optional
trybot android_n5x_swarming_dbg:

Traceback (most recent call last):
  File "/b/c/b/android_n5x_swarming_dbg/src/tools/mb/mb.py", line 61, in Main
    ret = self.args.func()
  File "/b/c/b/android_n5x_swarming_dbg/src/tools/mb/mb.py", line 263, in CmdAnalyze
    return self.RunGNAnalyze(vals)
  File "/b/c/b/android_n5x_swarming_dbg/src/tools/mb/mb.py", line 1275, in RunGNAnalyze
    raise MBErr(err)
MBErr: target "android_tools" not found in //testing/buildbot/gn_isolate_map.pyl
target "system_webview_shell_layout_test_apk" not found in //testing/buildbot/gn_isolate_map.pyl
step returned non-zero exit code: 1

The two targets are ignored for now. I'm not certain at this point
whether they also needs testing. 

The build is not green yet with this CL. There are still some flaky
tests and timed-out test suites. Will look into them in follow-up CLs.

BUG=737910

Change-Id: Ia0119ebe9494e4c3c8df5844754a95f75ff47957
Reviewed-on: https://chromium-review.googlesource.com/558418
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Jinsuk Kim <jinsukkim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#487020}
[modify] https://crrev.com/3f7c90c00159bbd0d3b56a5164206417b89b3bab/testing/buildbot/gn_isolate_map.pyl
[modify] https://crrev.com/3f7c90c00159bbd0d3b56a5164206417b89b3bab/testing/buildbot/manage.py

jbudorick@ is there any way you would recommend for me to do to make sure the target not running by default (like android_n5x_swarming_dbg here) stays unbroken up to the step 'compile'? I've fixed a build issue in the past https://codereview.chromium.org/2804163002/ and did it again. And it may be broken in the future. Looks like they mostly go unnoticed.

Just to be clear - I'm not talking about green build (i.e. all the tests being passed) but only up to the step compiling all the targets to test.

It's tricky as-is because of the builder that android_n5x_swarming_dbg is set to mirror, Android arm64 Builder (dbg) (https://build.chromium.org/p/chromium.android/builders/Android%20arm64%20Builder%20(dbg)). That builder is responsible for building the targets used by M64 (the tester that android_n5x_swarming_dbg mirrors) as well as the targets used by the N webview bot. The latter case has no trybot coverage (iirc).

I think the easiest way to ensure that android_n5x_swarming_dbg keeps building would be to split the waterfall builder s.t. android_n5x_swarming_dbg doesn't build the targets required for the N webview bot. That doesn't really help those targets stay green, unfortunately.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7ffeb9cc2f206d63942c7e6cf56037d5e26046a8

commit 7ffeb9cc2f206d63942c7e6cf56037d5e26046a8
Author: Jinsuk Kim <jinsukkim@chromium.org>
Date: Thu Jul 20 00:36:02 2017

Fix WebappModeTest.testWebappRequiresValidMac

The test passes fine on the release build but shows flakiness on
the debug build whose trybot result I'm trying to stablize. It
takes a bit longer to get the test page fully loaded on the debug
build, therefore the test often timed out before proceeding to
the next step. This CL ensures the scale factor gets matched
by waiting for the tab to be created before the test is performed. 
Local tests shows 100% pass, compared to about 50% failures before
the patch.

BUG=737910, 746476

Change-Id: If270b2c5dba122c899cba0f050ca808878773048
Reviewed-on: https://chromium-review.googlesource.com/559330
Reviewed-by: Tommy Nyquist <nyquist@chromium.org>
Reviewed-by: Changwan Ryu <changwan@chromium.org>
Commit-Queue: Jinsuk Kim <jinsukkim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#488063}
[modify] https://crrev.com/7ffeb9cc2f206d63942c7e6cf56037d5e26046a8/chrome/android/javatests/src/org/chromium/chrome/browser/TabsOpenedFromExternalAppTest.java
[modify] https://crrev.com/7ffeb9cc2f206d63942c7e6cf56037d5e26046a8/chrome/test/android/javatests/src/org/chromium/chrome/test/MultiActivityTestBase.java
[modify] https://crrev.com/7ffeb9cc2f206d63942c7e6cf56037d5e26046a8/chrome/test/android/javatests/src/org/chromium/chrome/test/MultiActivityTestCommon.java
[modify] https://crrev.com/7ffeb9cc2f206d63942c7e6cf56037d5e26046a8/chrome/test/android/javatests/src/org/chromium/chrome/test/MultiActivityTestRule.java
[modify] https://crrev.com/7ffeb9cc2f206d63942c7e6cf56037d5e26046a8/chrome/test/android/javatests/src/org/chromium/chrome/test/util/ApplicationTestUtils.java

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d22717f0a203466640c03dd32247548403e02996

commit d22717f0a203466640c03dd32247548403e02996
Author: Jinsuk Kim <jinsukkim@chromium.org>
Date: Thu Aug 10 23:36:33 2017

Tidy up some chrome tests

This CL addresses the nits which the reviewer had left
but I missed addressing in https://crrev.com/c/559330.

BUG=737910

Change-Id: I647a612d0cc9ddbdb09664596418f259e49478f8
Reviewed-on: https://chromium-review.googlesource.com/604968
Reviewed-by: Tommy Nyquist <nyquist@chromium.org>
Commit-Queue: Jinsuk Kim <jinsukkim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#493598}
[modify] https://crrev.com/d22717f0a203466640c03dd32247548403e02996/chrome/android/javatests/src/org/chromium/chrome/browser/webapps/WebappModeTest.java
[modify] https://crrev.com/d22717f0a203466640c03dd32247548403e02996/chrome/test/android/javatests/src/org/chromium/chrome/test/MultiActivityTestBase.java
[modify] https://crrev.com/d22717f0a203466640c03dd32247548403e02996/chrome/test/android/javatests/src/org/chromium/chrome/test/MultiActivityTestCommon.java
[modify] https://crrev.com/d22717f0a203466640c03dd32247548403e02996/chrome/test/android/javatests/src/org/chromium/chrome/test/MultiActivityTestRule.java
[modify] https://crrev.com/d22717f0a203466640c03dd32247548403e02996/chrome/test/android/javatests/src/org/chromium/chrome/test/util/ApplicationTestUtils.java

Moving Infra>Client>Android -> Infra>Client>Chrome+OS=Android