Can you help me route this to someone to look at what may have gone on with the Android try bot more closely? The build I reference above, which is the one rebaseline-cl claims to have pulled from, has layout test results but they do not match what I would expect. It's as if they were produced from a different binary that didn't include my patch. I wonder if something is up with swarming or otherwise w/ the bot config?

I don't really want to monkey with things more on off chance I make it harder to debug the issue.

The reason why Android passes here is because it only runs the smoke tests (https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/SmokeTests), and the set of smoke tests has no overlap with the set of failing tests for that change. That is, that change almost certainly affects results on Android, but we only run a very small number of tests on Android, so we don't discover the results that are different.

I'm surprised that it said that it said that android results were missing though, since they should be available (https://storage.googleapis.com/chromium-layout-test-archives/android_blink_rel/2310/layout-test-results/results.html). I expect that when you run rebaseline-cl it should log something like:
  Got 404 response from: https://storage.googleapis.com/chromium-layout-test-archives/android_blink_rel/2310/layout-test-results/failing_results.json
if it failed to get results. Do you see that?

Yes I saw something like what you said. Ok, hmm. What is my best next step? How do we produce, fetch and integrate Android changes?

BTW I will re-run script tonight/tomorrow and attach specific output so we have it for record.

Well, the status quo is that we're not covering Android currently, so for just that particular change, we could go ahead and ignore android for now.

In order to actually add more coverage in Android, we just need to add more tests to SmokeTests ( bug 412732 ). AFAIK there's no good reason why we didn't do this before; this would be done by https://chromium-review.googlesource.com/c/481042/.

You could add tests to SmokeTests yourself for that change in order to get android_blink_rel to run the tests -- or you could go ahead for now with rebaselining for other platforms, and when that change above is submitted then the relevant tests should be covered.

Ah, thanks. I thought we were running most/all paint-related layout tests on Android builder on waterfall, but it sounds like if we were, perhaps we aren't currently?

If we enable some we just need to make sure there is a trybot people can add to their change to see and somehow resolve failures before landing. We had an issue with this in past, see http://crbug.com/512856. And then I would have thought the new trybot added for use with rebaseline-cl would be another potential source but that one also seems to only run SmokeTests.

I am actually ok with the current status quo for now. Using more Android bot resources and increasing time to run, etc. doesn't seem warranted unless we're actually breaking functionality or burning dev time for some reason without enhancing smoke tests.

+pdr as he may have more background/opinion on layout tests on Android.

Yep, so the current status is that android_blink_rel also runs only smoke tests, and currently if you want to try your change on android on tests that aren't in smoke tests, you have to include a change to smoke tests as part of the patch. No new bot was added for rebaseline-cl for android, we just use android_blink_rel.

Update: I've now committed https://chromium-review.googlesource.com/c/481042/ which increases the number of smoke tests to about 1000, including many of the tests with different results in https://codereview.chromium.org/2820743003/.

For example, for fast/text/international/bidi-neutral-directionality-paragraph-start.html:

https://chromium-review.googlesource.com/c/481042/6/third_party/WebKit/LayoutTests/platform/android/fast/text/international/bidi-neutral-directionality-paragraph-start-expected.png