More specifically, let's add a step to do a manual test pass across CUJs on Beta channel to get an accurate representation of merges prior to shipping a feature. 

We've been working with cliffordcheng@ on this.

We only have covered that for Linux now.
Mac will be added once we land this CL.
https://chromium-review.googlesource.com/c/chromium/src/+/1296811

Doug, maybe we can ask David to also cover it until Windows and Mac are fully covered.
It shouldn't take too much of his time (maybe 30 mins?)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0cfecb52487066ad0cc55ccf18635e6d28bd9b5d

commit 0cfecb52487066ad0cc55ccf18635e6d28bd9b5d
Author: Clifford Cheng <cliffordcheng@chromium.org>
Date: Thu Nov 08 18:50:38 2018

[Media UX] Enable the pixel by pixel comparison test for Mac OS.

1. Tested on Mac and verify everything works as expected.
2. Removed the unnecessary call of GetNativeWindow().
3. The RGB for GPU enabled and GPU disabled test images is slightly different. Additional check is needed.

Bug: 898230
Change-Id: I85bf2450862361572597632159de60db6abf2c22
Reviewed-on: https://chromium-review.googlesource.com/c/1307062
Reviewed-by: Mounir Lamouri <mlamouri@chromium.org>
Commit-Queue: Clifford Cheng <cliffordcheng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#606552}
[modify] https://crrev.com/0cfecb52487066ad0cc55ccf18635e6d28bd9b5d/chrome/browser/picture_in_picture/picture_in_picture_window_controller_browsertest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ba0d48f8b3e768e554f6b30d7ab04e3f0e907e61

commit ba0d48f8b3e768e554f6b30d7ab04e3f0e907e61
Author: Findit <findit-for-me@appspot.gserviceaccount.com>
Date: Thu Nov 08 23:52:22 2018

Revert "[Media UX] Enable the pixel by pixel comparison test for Mac OS."

This reverts commit 0cfecb52487066ad0cc55ccf18635e6d28bd9b5d.

Reason for revert:

Findit (https://goo.gl/kROfz5) identified CL at revision 606552 as the
culprit for flakes in the build cycles as shown on:
https://findit-for-me.appspot.com/waterfall/flake/flake-culprit?key=ag9zfmZpbmRpdC1mb3ItbWVyQwsSDEZsYWtlQ3VscHJpdCIxY2hyb21pdW0vMGNmZWNiNTI0ODcwNjZhZDBjYzU1Y2NmMTg2MzVlNmQyOGJkOWI1ZAw

Sample Failed Build: https://ci.chromium.org/buildbot/chromium.mac/Mac10.12%20Tests/16586

Sample Failed Step: network_service_browser_tests on Intel GPU on Mac on Mac-10.12.6

Sample Flaky Test: PictureInPicturePixelComparisonBrowserTest.VideoPlay

Original change's description:
> [Media UX] Enable the pixel by pixel comparison test for Mac OS.
> 
> 1. Tested on Mac and verify everything works as expected.
> 2. Removed the unnecessary call of GetNativeWindow().
> 3. The RGB for GPU enabled and GPU disabled test images is slightly different. Additional check is needed.
> 
> Bug: 898230
> Change-Id: I85bf2450862361572597632159de60db6abf2c22
> Reviewed-on: https://chromium-review.googlesource.com/c/1307062
> Reviewed-by: Mounir Lamouri <mlamouri@chromium.org>
> Commit-Queue: Clifford Cheng <cliffordcheng@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#606552}

Change-Id: I10a133230cad3256bf4e5e0c3c925920484ce6a1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 898230,  903580 
Reviewed-on: https://chromium-review.googlesource.com/c/1327612
Cr-Commit-Position: refs/heads/master@{#606647}
[modify] https://crrev.com/ba0d48f8b3e768e554f6b30d7ab04e3f0e907e61/chrome/browser/picture_in_picture/picture_in_picture_window_controller_browsertest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bf310bd3ce97495602e136d25826dd2b68b7234b

commit bf310bd3ce97495602e136d25826dd2b68b7234b
Author: Clifford Cheng <cliffordcheng@chromium.org>
Date: Tue Nov 13 23:00:56 2018

[Media UX] Enable the pixel by pixel comparison test for Mac OS.

1. Tested on Mac and verify everything works as expected.
2. Removed the unnecessary call of GetNativeWindow().
3. Added a flag to disable GPU for the test. GPU makes an slight impact on image conversion and the output image color.
   It is turned off to prevent noise.

Bug: 898230
Change-Id: If69a900a2e1373075f58c80a2aaef526d856bb08
Reviewed-on: https://chromium-review.googlesource.com/c/1330297
Reviewed-by: Mounir Lamouri <mlamouri@chromium.org>
Commit-Queue: Clifford Cheng <cliffordcheng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#607794}
[modify] https://crrev.com/bf310bd3ce97495602e136d25826dd2b68b7234b/chrome/browser/picture_in_picture/picture_in_picture_window_controller_browsertest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/cd7f1f8e6b94495acc668869789bcae171bc0d77

commit cd7f1f8e6b94495acc668869789bcae171bc0d77
Author: Clifford Cheng <cliffordcheng@chromium.org>
Date: Fri Nov 30 18:55:33 2018

Enable pixel-by-pixel comparison test for Windows.

Fixed all the compile and runtime issues on Windows and enable the test on it.

Bug: 898230
Change-Id: I9e92e49583893898603bffa46f6fc6bdc1ef7e20
Reviewed-on: https://chromium-review.googlesource.com/c/1339240
Reviewed-by: Mounir Lamouri <mlamouri@chromium.org>
Commit-Queue: Clifford Cheng <cliffordcheng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#612714}
[modify] https://crrev.com/cd7f1f8e6b94495acc668869789bcae171bc0d77/chrome/browser/picture_in_picture/picture_in_picture_window_controller_browsertest.cc

This bug had an unsupported status. Updating to Untriaged so someone will reevaluate.

This issue has an owner, a component and a priority, but is still listed as untriaged or unconfirmed. By definition, this bug is triaged. Changing status to "assigned". Please reach out to me if you disagree with how I've done this.