Thank you for the report!

Could you confirm that it's a clean ToT build without modifications?
And could you clarify what is the OS version? The report says OS X 10.13.6, but these are Win tests.
Also, could you describe your audio HW setup?

Thanks!

I can confirm that this is from a clean ToT build (no modifications).

Apologies for the OS version, it's from a Windows 10 machine running Windows 10 Pro Version 1809, OS build 17763.1. The machine is running on the Windows Insider Fast channel.

Regarding the audio setup it's using the built-in mic/speakers which are using the "High Definition Audio Device" driver, let me know if you need more detail around the specific hardware!

Thanks Andrew.

Marina, could you PTAL if it's related to sandboxing?

Checking.

Not sandbox related, tests crash with sandbox disabled as well.

Also, the tests are not run out of process - sandbox has no effect on them.

Has the same tests worked on the same device before; i.e, is the issue new in M71?

I didn't find it in waterfall, Mirko is checking if it runs on webrtc bots.

It runs in the //media:media_unittests binary.

Here is a recent run: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win%207%20Tests%20x64%20%281%29/43801 and it is passing  (https://chromium-swarm.appspot.com/task?id=405267b500238810&refresh=10&show_raw=1).

All these bots should be running it: https://ci.chromium.org/p/chromium/g/chromium.win/console.

andrew.macpherson@: are you building in Debug mode?

mbonadei@: I assume we build in Release mode on bots but with DCHECKs enabled. Not sure exactly what flags are needed to do that.

henrika@: Yes, building in Debug mode here.

Mirko just found that the tests are passing on this bot: https://chromium-swarm.appspot.com/task?id=405267b500238810&refresh=10&show_raw=1

Maybe this is a Win10 only issue?

Passing on win 10 bot as well: https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8933691825360279136/+/steps/media_unittests_on_Windows-10-15063/0/stdout

marinaciocea@: If I force ABORT_AUDIO_TEST_IF_NOT() to abort by appending "false &&" to the conditional on the failing tests they seem to report that they've passed but print a warning:

C:\src\chromium\src>.\out\Default\audio_unittests --gtest_filter="WinAudioInputTest.WASAPIAudioInputStreamLoopback*"
IMPORTANT DEBUGGING NOTE: batches of tests are run inside their
own process. For debugging a test inside a debugger, use the
--gtest_filter=<your_test_name> flag along with
--single-process-tests.
Using sharding settings from environment. This is shard 0/1
Using 1 parallel jobs.
Note: Google Test filter = WinAudioInputTest.WASAPIAudioInputStreamLoopback/0:WinAudioInputTest.WASAPIAudioInputStreamLoopback/1
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from WinAudioInputTest
[ RUN      ] WinAudioInputTest.WASAPIAudioInputStreamLoopback/0
[7880:12248:1003/134458.135:985655921:WARNING:audio_unittest_util.cc(20)] Requirement(s) not satisfied (false && device_info_accessor.HasAudioOutputDevices() && CoreAudioUtil::IsSupported())
[       OK ] WinAudioInputTest.WASAPIAudioInputStreamLoopback/0 (15 ms)
[ RUN      ] WinAudioInputTest.WASAPIAudioInputStreamLoopback/1
[7880:12248:1003/134458.135:985655921:WARNING:audio_unittest_util.cc(20)] Requirement(s) not satisfied (false && device_info_accessor.HasAudioOutputDevices() && CoreAudioUtil::IsSupported())
[       OK ] WinAudioInputTest.WASAPIAudioInputStreamLoopback/1 (7 ms)
[----------] 2 tests from WinAudioInputTest (27 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (33 ms total)
[  PASSED  ] 2 tests.
[1/2] WinAudioInputTest.WASAPIAudioInputStreamLoopback/0 (15 ms)
[2/2] WinAudioInputTest.WASAPIAudioInputStreamLoopback/1 (7 ms)
SUCCESS: all tests passed.
Tests took 0 seconds.

Could it be that the tests aren't really running because HasAudioOutputDevices() is false? And maybe the warning is lost in the logs because it's output to stderr and not stdout? Just guessing here though. :)

Re #11: This (https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20Tests%20x64%20(dbg)) runs the test compiled in Debug mode.

The flag to compile in Release mode with DCHECKS enabled is 'dcheck_always_on'.

It's not quite about DCHECKs: the tests will return silently if there is no audio HW and --require-audio-hardware-for-testing is not specified.

Can't explain how this test can work on some bots. To me it looks like it should always fail after this change:

https://chromium-review.googlesource.com/c/chromium/src/+/1065918

Not sure if the existing check is really needed or not.

What most likely happens on bots where this test passes is that

https://cs.chromium.org/chromium/src/media/audio/win/audio_manager_win.cc?q=audio_manager_win&sq=package:chromium&g=0&l=173

is hit and we never reach the this part

https://cs.chromium.org/chromium/src/media/audio/win/audio_manager_win.cc?q=audio_manager_win&sq=package:chromium&g=0&l=190

where audio parameters are set to a non-zero value. In any case, the test is flaky as is.

The first failure was introduces by my CL in comment #19.

The second (DCHECK hit) was introduced in https://chromium-review.googlesource.com/c/chromium/src/+/1181134. CC armax@ for that.

In both cases the tests need to be updated. armax@ please update
WinAudioTest.PCMWaveStreamOpenLimit
WinAudioTest.SanityOnMakeParams

It believe the bots don't have audio devices thus the code path is different when running there. It's unfortunate but the bot maintenance cost vs gain is very high. Devs need to run media unittests locally.

Yep, these tests doesn't run on any bots in any meaningful sense. Maybe we can add them to the WebRTC bots that also run some content_browsertests and such (which has devices, IIRC)?

The params.IsValid() DCHECK was added since we validate the parameters before they reach the AudioManager now (in the IPC layer). Any tests that pass invalid parameters can probably just be removed.

So, would it be the right approach to either fix the two tests to in fact create proper parameter objects (should they be mocked/faked?) or remove the both of them?

The point of the tests are to make sure invalid parameters are handled correctly. Since invalid parameters doesn't have to be handled by the AudioManager any longer, the tests can be removed.

In the loopback case, the experimental echo canceller shouldn't be available, so I'm removing that capability.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8d59498c5925b280b0950dd8f0fad010c4871136

commit 8d59498c5925b280b0950dd8f0fad010c4871136
Author: Armando Miraglia <armax@chromium.org>
Date: Fri Oct 05 13:28:52 2018

[Media Win] Removing crashing tests from windows media testing suite.

Since the parameters are now checked when passed through Mojo, this
test is now outdated as the parameters are never invalid. This is
verifyed by a DCHECK intorduced in crrev.com/c/1181134.

BUG= 891202 
TESTS=None

Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: Ibbda19ce4d6d99363e02136883709082383582c2
Reviewed-on: https://chromium-review.googlesource.com/c/1264676
Reviewed-by: Max Morin <maxmorin@chromium.org>
Reviewed-by: Henrik Grunell <grunell@chromium.org>
Commit-Queue: Armando Miraglia <armax@chromium.org>
Cr-Commit-Position: refs/heads/master@{#597089}
[modify] https://crrev.com/8d59498c5925b280b0950dd8f0fad010c4871136/media/audio/win/audio_output_win_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/21ed1ea972dba1ec80524ded075855f77cd4bdb4

commit 21ed1ea972dba1ec80524ded075855f77cd4bdb4
Author: Henrik Grunell <grunell@chromium.org>
Date: Fri Oct 05 14:31:03 2018

[Win] Don't support experimental system AEC for loopback devices.

Adds unit test for effects.

Bug:  891202 
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I16653e77faa5fc12d4462514b8cd73abbe74341b
Reviewed-on: https://chromium-review.googlesource.com/c/1264535
Reviewed-by: Max Morin <maxmorin@chromium.org>
Commit-Queue: Henrik Grunell <grunell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#597107}
[modify] https://crrev.com/21ed1ea972dba1ec80524ded075855f77cd4bdb4/media/audio/win/audio_low_latency_input_win_unittest.cc
[modify] https://crrev.com/21ed1ea972dba1ec80524ded075855f77cd4bdb4/media/audio/win/audio_manager_win.cc

Andrew, thanks for reporting! Can you verify that it's fixed for you?

Thanks grunell@, I can confirm that this is fixed now on ToT!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e833c5b9f864a0395a81d51c5a2e50036b9e4355

commit e833c5b9f864a0395a81d51c5a2e50036b9e4355
Author: Henrik Grunell <grunell@chromium.org>
Date: Mon Oct 08 08:32:59 2018

Fixed typo in comment.

Bug:  891202 
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I46c6df2e176a55a35a3afdcfec31d8652d78bb66
Reviewed-on: https://chromium-review.googlesource.com/c/1268015
Reviewed-by: Henrik Andreasson <henrika@chromium.org>
Commit-Queue: Henrik Grunell <grunell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#597496}
[modify] https://crrev.com/e833c5b9f864a0395a81d51c5a2e50036b9e4355/media/audio/win/audio_low_latency_input_win_unittest.cc

Thanks for verifying!