Set the correct component to get it triaged...

Technically I think this is "working as intended" -- the windows baseline directory is explicitly listed as the fallback if there are no linux-specific baselines at https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/port/linux.py?l=46.

See attached baseline fallback graph.

The original reason for this was to decrease the number of baselines required in the case where win and linux match but mac is different.

This could be changed, and the change would involve rebaselining lots of tests so that when win and linux match but mac doesn't, we would then store two separate copies of the same baseline file.

For cases where there are baselines in platform/win/... but results don't match on linux with layout NG, does it work to add failure expectation lines in FlagExpectations/enable-blink-features%3DLayoutNG?

Deleted: fallback-paths.png 44.0 KB

	fallback-paths.png 44.0 KB View Download

TIL the layout test fallback paths...

Without modifying the fallback path, is it possible to setup flag&platform-specific baselines for layout ng?

For LayoutNG, if a test already has flag-specific baseline, it's considered passing. I hope it can be distinguished from the other tons of tests LayoutNG is failing on, so I don't want to mark it as fail in FlagExpectations.

Quick note: When we talked in person I couldn't remember whether there were flag-specific platform-specific baselines, and I can't find any (there's no platform/ dir under the https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/flag-specific dirs, and no flag-specific/ dir under the platform dirs). I think skobes@ said that platform-specific flag-specific baselines should be supported though...

The test runner supports flag-specific/$FLAG/platform/..., though maybe no one is using it yet.

But this bug seems to be about the baseline search path for virtual test suites, which is different from flag-specific.

In particular it does seem surprising to use a Windows-specific baseline on Linux, in preference to a baseline outside of platform.  I think that means we have a lot of unnecessary duplication of baselines where only one platform diverges...

Reading comment #3 more carefully... it sounds like we made an informed trade-off of eliminating duplication when Win and Linux match (but diverge from Mac), at the cost of extra duplication when Win alone is different?

I guess "Win" really means "Win OR Linux".  Maybe we should rename it to "Aura".

+robertma, you've talked about the complex fallback paths before, any thoughts on this?

The discussion so far is quite accurate and comprehensive.

This is indeed WAI. I recently documented the details of the fallback mechanism in https://chromium.googlesource.com/chromium/src/+/master/docs/testing/layout_test_baseline_fallback.md

*Note: the fallback graph in comment #3 might have been outdated; it is configurable and is changing along the time as new platforms come out. The doc above explains how to look up the newest fallback path in the code.

The one last question, I think, is whether we should use win as a fallback of linux. The answer is not obvious. There are two considerations: making the fallback path logical/intuitive, and minimizing duplication. Someone (sorry I don't exactly remember who) told me the historic reason: Windows came out first and is the primary platform, so the later Linux release was something tagging along (Mac, apparently, is too different). I don't see a strong reason for immediate changes. The weird linux->win fallback is already known by many folks, and tools like rebaseline-cl hide most of these details (which I strongly suggest using).

Closing as WONTFIX, but we can definitely come back and revisit if more people are unhappy about this.