chameleon: Quality test is not strict enough |
|||||
Issue descriptionI found there is rattling noise in chell headphone result. However, the quality check used in audio_AudioBasicHeadphone.quality can not catch such rattling noise. I will check if the quality threshold is too loose. Or, if we should switch to the new anomaly detection algorithm.
,
Nov 11 2016
https://chromium-review.googlesource.com/410581 should solve the false alarm in the end of data.
,
Nov 11 2016
,
Nov 14 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/443d9083210bd0acd94e1e38153eb1d3a5312f90 commit 443d9083210bd0acd94e1e38153eb1d3a5312f90 Author: Cheng-Yi Chiang <cychiang@chromium.org> Date: Thu Nov 10 08:55:47 2016 [autotest] audio_data: Use numpy fromstring function Use numpy fromstring function to read samples from data string. This speeds up the time to read data from large file or large data string and parse data input channel samples. The speed boost is significant. For example, for 150 seconds long data, the time to read data dropped from 140 seconds to 0.2 seconds. Also, remove some unused functions. BUG= chromium:663616 TEST=Use check_quality tool, check the time to read samples becomes much faster. Change-Id: I6a9028445b3311cd01a3c75621eaf64b5b9d11b8 Reviewed-on: https://chromium-review.googlesource.com/409893 Commit-Ready: Cheng-Yi Chiang <cychiang@chromium.org> Tested-by: Cheng-Yi Chiang <cychiang@chromium.org> Reviewed-by: Kalin Stoyanov <kalin@chromium.org> [modify] https://crrev.com/443d9083210bd0acd94e1e38153eb1d3a5312f90/client/cros/audio/audio_data.py
,
Nov 15 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/6971b1811c477104bdcd440686757718893e341a commit 6971b1811c477104bdcd440686757718893e341a Author: Cheng-Yi Chiang <cychiang@chromium.org> Date: Thu Nov 10 09:26:49 2016 [autotest] Use artifact algorithm for 3.5mm quality test The new artifact algorithm can detect rattling noise. Use it for basic headphoen and external microphone quality test. BUG= chromium:663616 TEST=run the test on samus, can pass. On chell, can fail. Change-Id: I073324f8037f71cd6cbc3d491a06444280a39de7 Reviewed-on: https://chromium-review.googlesource.com/409894 Commit-Ready: Cheng-Yi Chiang <cychiang@chromium.org> Tested-by: Cheng-Yi Chiang <cychiang@chromium.org> Reviewed-by: Cheng-Yi Chiang <cychiang@chromium.org> [modify] https://crrev.com/6971b1811c477104bdcd440686757718893e341a/server/site_tests/audio_AudioBasicHeadphone/audio_AudioBasicHeadphone.py [modify] https://crrev.com/6971b1811c477104bdcd440686757718893e341a/server/site_tests/audio_AudioBasicExternalMicrophone/audio_AudioBasicExternalMicrophone.py
,
Nov 17 2016
,
Nov 21 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/93e9d9b04b2bea757953c1d1d7c5a4b4ade14b49 commit 93e9d9b04b2bea757953c1d1d7c5a4b4ade14b49 Author: Cheng-Yi Chiang <cychiang@chromium.org> Date: Fri Nov 11 10:55:31 2016 [autotest] audio_quality_measurement: Ignore detected noise at edge In the beginning and the end of data, there will be false alarm noise caused by edge effect of Hilbert transform. We should ignore them. See the docstring of NEAR_DATA_START_OR_END_SECS for detail and example. BUG= chromium:663616 TEST=use check_quality tool to check the quality of the file attached on the issue. Change-Id: Ie7cbc1dea3986923f2421275116344a47a430199 Reviewed-on: https://chromium-review.googlesource.com/410581 Commit-Ready: Cheng-Yi Chiang <cychiang@chromium.org> Tested-by: Cheng-Yi Chiang <cychiang@chromium.org> Reviewed-by: Wai-Hong Tam <waihong@google.com> [modify] https://crrev.com/93e9d9b04b2bea757953c1d1d7c5a4b4ade14b49/client/cros/audio/audio_quality_measurement.py
,
Dec 19 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/1cba94922d3218b614dba11e0fc3a3e878faea88 commit 1cba94922d3218b614dba11e0fc3a3e878faea88 Author: Cheng-Yi Chiang <cychiang@chromium.org> Date: Fri Nov 11 10:55:31 2016 [autotest] audio_quality_measurement: Ignore detected noise at edge In the beginning and the end of data, there will be false alarm noise caused by edge effect of Hilbert transform. We should ignore them. See the docstring of NEAR_DATA_START_OR_END_SECS for detail and example. BUG= chromium:663616 TEST=use check_quality tool to check the quality of the file attached on the issue. Change-Id: Ie7cbc1dea3986923f2421275116344a47a430199 Reviewed-on: https://chromium-review.googlesource.com/410581 Commit-Ready: Cheng-Yi Chiang <cychiang@chromium.org> Tested-by: Cheng-Yi Chiang <cychiang@chromium.org> Reviewed-by: Wai-Hong Tam <waihong@google.com> (cherry picked from commit 93e9d9b04b2bea757953c1d1d7c5a4b4ade14b49) Reviewed-on: https://chromium-review.googlesource.com/422121 Reviewed-by: Ting Shen <phoenixshen@chromium.org> Commit-Queue: Ting Shen <phoenixshen@chromium.org> Tested-by: Ting Shen <phoenixshen@chromium.org> [modify] https://crrev.com/1cba94922d3218b614dba11e0fc3a3e878faea88/client/cros/audio/audio_quality_measurement.py
,
Dec 19 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/6e1a9522d2e99c5eca153e0326b345fcfb22030c commit 6e1a9522d2e99c5eca153e0326b345fcfb22030c Author: Cheng-Yi Chiang <cychiang@chromium.org> Date: Thu Nov 10 08:55:47 2016 [autotest] audio_data: Use numpy fromstring function Use numpy fromstring function to read samples from data string. This speeds up the time to read data from large file or large data string and parse data input channel samples. The speed boost is significant. For example, for 150 seconds long data, the time to read data dropped from 140 seconds to 0.2 seconds. Also, remove some unused functions. BUG= chromium:663616 TEST=Use check_quality tool, check the time to read samples becomes much faster. Change-Id: I6a9028445b3311cd01a3c75621eaf64b5b9d11b8 Reviewed-on: https://chromium-review.googlesource.com/409893 Commit-Ready: Cheng-Yi Chiang <cychiang@chromium.org> Tested-by: Cheng-Yi Chiang <cychiang@chromium.org> Reviewed-by: Kalin Stoyanov <kalin@chromium.org> (cherry picked from commit 443d9083210bd0acd94e1e38153eb1d3a5312f90) Reviewed-on: https://chromium-review.googlesource.com/422122 Reviewed-by: Ting Shen <phoenixshen@chromium.org> Commit-Queue: Ting Shen <phoenixshen@chromium.org> Tested-by: Ting Shen <phoenixshen@chromium.org> [modify] https://crrev.com/6e1a9522d2e99c5eca153e0326b345fcfb22030c/client/cros/audio/audio_data.py
,
Feb 10 2017
Verified. https://wmatrix.googleplex.com/platform/unfiltered?hide_missing=True&tests=audio_AudioBasicHeadphone.quality&releases=tot&platforms=samus |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by cychiang@chromium.org
, Nov 11 2016I changed basic external mic test to use quality measurement module. But I found a case where quality test has incorrectly detect noise in the end of data. I will fix the quality measurement module. With the check_quality tool, we can easily check and debug quality measurement module: For attached samus_ext_mic_fa.wav ./client/cros/audio/check_quality.py ~/res/sample/samus_ext_mic_fa.wav --spectral --quality --debug shows: Channel 0 quality: {'artifacts': {'burst_during_playback': [], 'delay_during_playback': [], 'noise_after_playback': [(3.419979166666667, 6.249999999985434e-05)], 'noise_before_playback': []}, 'equivalent_noise_level': 0.00015422946018736123, 'volume_changes': []} while the data length is 3.42 seconds.320 KB
320 KB Download