WebKit bots seem to be compiled without software H.264 support |
||||||
Issue descriptionsee this revert and linked trybot runs: https://chromium-review.googlesource.com/c/chromium/src/+/1049088 WebKit Win7 https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Win7/63947 WebKit Win10 https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Win10/34576 WebKit Mac10.10 https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Mac10.10/46846 WebKit Mac10.11 https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Mac10.11/32883 WebKit Linux Trusty https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Linux%20Trusty/44403 (It passes on Mac 10.12.) Guess at cause: Possibly the bots named "WebKit" are compiled without software H.264 support, leading to no H.264 being announced when hardware support is lacking (I assume the pass on 10.12 is because that platform uses the platform's built-in H.264).
,
May 8 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e4d5b964d7b62222f4b840e60596e66f705646a1 commit e4d5b964d7b62222f4b840e60596e66f705646a1 Author: Harald Alvestrand <hta@chromium.org> Date: Tue May 08 10:34:45 2018 Disable virtual tests for webkit bots TBR=ksakamoto@chromium.org Bug: chromium:840659 Change-Id: Id25d5f8678296cf0e797fc668cdf384e01354433 Reviewed-on: https://chromium-review.googlesource.com/1049645 Reviewed-by: Harald Alvestrand <hta@chromium.org> Reviewed-by: Kunihiko Sakamoto <ksakamoto@chromium.org> Commit-Queue: Harald Alvestrand <hta@chromium.org> Cr-Commit-Position: refs/heads/master@{#556747} [modify] https://crrev.com/e4d5b964d7b62222f4b840e60596e66f705646a1/third_party/WebKit/LayoutTests/TestExpectations
,
May 8 2018
,
May 8 2018
,
May 8 2018
No, I'm not going to add H.264 to the webkit bots.
,
May 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1af763dfc71d7ca20b6615fd663c613e6013f1c2 commit 1af763dfc71d7ca20b6615fd663c613e6013f1c2 Author: Robert Ma <robertma@chromium.org> Date: Tue May 29 03:22:51 2018 Tweak expectations for webrtc video codecs WPT to cover all platforms and stop the importer from trying to rebaseline. TBR=hta No-Try: True Bug: 840659 Change-Id: Iff9036601cd45d97f39df1b55b45c82b243b7501 Reviewed-on: https://chromium-review.googlesource.com/1075770 Reviewed-by: Robert Ma <robertma@chromium.org> Commit-Queue: Robert Ma <robertma@chromium.org> Cr-Commit-Position: refs/heads/master@{#562322} [modify] https://crrev.com/1af763dfc71d7ca20b6615fd663c613e6013f1c2/third_party/WebKit/LayoutTests/TestExpectations
,
May 29 2018
@hta do you know what decides whether software H.264 is compiled in? At the time being, because we can't set different test expectations for different build configurations, we have to set the test as "flaky" on all platforms (while in fact they consistently pass/fail depending on whether software H.264 is compiled in), and we essentially lose coverage of the test.
,
May 29 2018
@robertma The reason for the selection of platforms is that there are no webkit bots for ChromeOS, Android or IOS, and those had no failures, so didn't need to be eliminated. Also, the Mac 10.12 seems to use platform H.264, and thus passed. I was wondering what you meant by "stop the importer from trying to rebaseline", and why that mattered. The flags in out/Debug/args.gn that matter are: proprietary_codecs = true ffmpeg_branding = "Chrome"
,
May 29 2018
First of all, the direct reason for my CL in 6# was to fix the `rebaseline-cl` tool and unblock wpt-importer. Here's an example failed import from yesterday: https://chromium-review.googlesource.com/c/chromium/src/+/1075565/1 The importer uses `rebaseline-cl` to rebaseline all tests failing on Blink try bots (usually mirrors of chromium.webkit). video-codecs.https.html (and its virtual variant) failed on two bots: https://ci.chromium.org/buildbot/tryserver.blink/mac10.12_retina_blink_rel/683 [ Retina ] https://ci.chromium.org/buildbot/tryserver.blink/mac10.13_blink_rel/878 [ Mac10.13 ] The importer then created a new baseline based on the results above: https://chromium-review.googlesource.com/c/chromium/src/+/1075565/1..2/third_party/WebKit/LayoutTests/external/wpt/webrtc/protocol/video-codecs.https-expected.txt However, since the CQ bots have software codecs, the CQ results wouldn't match the new baseline and the CL couldn't land. This is not limited to the importer. Essentially, because of the inconsistency between the Blink try bots and CQ bots, the rebaseline-cl tool, which many Blink engineers use, can't work properly unless these two tests are added to TestExpectations. Furthermore, as I just replied in the CL (https://crrev.com/c/1075770#message-07aa79e51344c0cfb42f2b3596c447229d676173), I believe the tests wouldn't pass on WebKit Mac 10.12, either. Because of issue 818301 , WebKit Mac 10.12 currently doesn't run layout tests and is almost a no-op bot. If layout tests were run on WebKit Mac 10.12, the two tests in question would also fail. Therefore, at least [ Retina Mac10.12 Mac10.13 ] were missing from the expectations, so at least we need [ Win Mac Linux ]. And I wasn't (and still am not) aware that other platforms run these tests (I don't think we run layout tests on ChromeOS or iOS at all; for Android, we run only a small subset listed in SmokeTests and these two tests are not in SmokeTests). Hence, I simply removed the platform specifiers altogether to cover all platforms. Please correct me if I'm mistaken about something and I'm totally fine to narrow down the specifiers to [ Win Mac Linux ].
,
May 29 2018
Thanks for the explanations! Adding @holmer to bug since this is relevant to H.264 management in general.
,
Aug 6
Ping from your friendly neighbourhood ecosystem infra rotation. This issue is more than 2 months old. hta@chromium.org and holmer@chromium.org, what are the risks of letting this issue get stale?
,
Nov 13
This issue is still present, but I'm downgrading to P3 given comment #5. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by bugdroid1@chromium.org
, May 8 2018