New issue
Advanced search Search tips

Issue 663616 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Closed: Nov 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

chameleon: Quality test is not strict enough

Project Member Reported by cychiang@chromium.org, Nov 9 2016

Issue description

I found there is rattling noise in chell headphone result.
However, the quality check used in audio_AudioBasicHeadphone.quality can not catch such rattling noise.

I will check if the quality threshold is too loose.
Or, if we should switch to the new anomaly detection algorithm.
 
chell_hp.wav
6.5 MB Download
chell_rattling_noise.png
110 KB View Download
Status: Started (was: Assigned)
I changed basic external mic test to use quality measurement module.
But I found a case where quality test has incorrectly detect noise in the end of data.
I will fix the quality measurement module.

With the check_quality tool, we can easily check and debug quality measurement module:

For attached samus_ext_mic_fa.wav
./client/cros/audio/check_quality.py ~/res/sample/samus_ext_mic_fa.wav --spectral --quality --debug

shows:

Channel 0 quality:
{'artifacts': {'burst_during_playback': [],
               'delay_during_playback': [],
               'noise_after_playback': [(3.419979166666667,
                                         6.249999999985434e-05)],
               'noise_before_playback': []},
 'equivalent_noise_level': 0.00015422946018736123,
 'volume_changes': []}

while the data length is 3.42 seconds.
samus_ext_mic_fa.wav
320 KB Download
https://chromium-review.googlesource.com/410581 should solve the false alarm in the end of data.

Comment 3 by vsu...@chromium.org, Nov 11 2016

Cc: vsu...@chromium.org
Project Member

Comment 4 by bugdroid1@chromium.org, Nov 14 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/443d9083210bd0acd94e1e38153eb1d3a5312f90

commit 443d9083210bd0acd94e1e38153eb1d3a5312f90
Author: Cheng-Yi Chiang <cychiang@chromium.org>
Date: Thu Nov 10 08:55:47 2016

[autotest] audio_data: Use numpy fromstring function

Use numpy fromstring function to read samples from data string.
This speeds up the time to read data from large file or large data
string and parse data input channel samples.

The speed boost is significant. For example, for 150 seconds long data,
the time to read data dropped from 140 seconds to 0.2 seconds.

Also, remove some unused functions.

BUG= chromium:663616 
TEST=Use check_quality tool, check the time to read samples becomes much
faster.

Change-Id: I6a9028445b3311cd01a3c75621eaf64b5b9d11b8
Reviewed-on: https://chromium-review.googlesource.com/409893
Commit-Ready: Cheng-Yi Chiang <cychiang@chromium.org>
Tested-by: Cheng-Yi Chiang <cychiang@chromium.org>
Reviewed-by: Kalin Stoyanov <kalin@chromium.org>

[modify] https://crrev.com/443d9083210bd0acd94e1e38153eb1d3a5312f90/client/cros/audio/audio_data.py

Project Member

Comment 5 by bugdroid1@chromium.org, Nov 15 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/6971b1811c477104bdcd440686757718893e341a

commit 6971b1811c477104bdcd440686757718893e341a
Author: Cheng-Yi Chiang <cychiang@chromium.org>
Date: Thu Nov 10 09:26:49 2016

[autotest] Use artifact algorithm for 3.5mm quality test

The new artifact algorithm can detect rattling noise.
Use it for basic headphoen and external microphone quality test.

BUG= chromium:663616 
TEST=run the test on samus, can pass. On chell, can fail.

Change-Id: I073324f8037f71cd6cbc3d491a06444280a39de7
Reviewed-on: https://chromium-review.googlesource.com/409894
Commit-Ready: Cheng-Yi Chiang <cychiang@chromium.org>
Tested-by: Cheng-Yi Chiang <cychiang@chromium.org>
Reviewed-by: Cheng-Yi Chiang <cychiang@chromium.org>

[modify] https://crrev.com/6971b1811c477104bdcd440686757718893e341a/server/site_tests/audio_AudioBasicHeadphone/audio_AudioBasicHeadphone.py
[modify] https://crrev.com/6971b1811c477104bdcd440686757718893e341a/server/site_tests/audio_AudioBasicExternalMicrophone/audio_AudioBasicExternalMicrophone.py

Status: Fixed (was: Started)
Project Member

Comment 7 by bugdroid1@chromium.org, Nov 21 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/93e9d9b04b2bea757953c1d1d7c5a4b4ade14b49

commit 93e9d9b04b2bea757953c1d1d7c5a4b4ade14b49
Author: Cheng-Yi Chiang <cychiang@chromium.org>
Date: Fri Nov 11 10:55:31 2016

[autotest] audio_quality_measurement: Ignore detected noise at edge

In the beginning and the end of data, there will be false alarm noise
caused by edge effect of Hilbert transform. We should ignore them.
See the docstring of NEAR_DATA_START_OR_END_SECS for detail and example.

BUG= chromium:663616 
TEST=use check_quality tool to check the quality of the file attached on
the issue.

Change-Id: Ie7cbc1dea3986923f2421275116344a47a430199
Reviewed-on: https://chromium-review.googlesource.com/410581
Commit-Ready: Cheng-Yi Chiang <cychiang@chromium.org>
Tested-by: Cheng-Yi Chiang <cychiang@chromium.org>
Reviewed-by: Wai-Hong Tam <waihong@google.com>

[modify] https://crrev.com/93e9d9b04b2bea757953c1d1d7c5a4b4ade14b49/client/cros/audio/audio_quality_measurement.py

Project Member

Comment 8 by bugdroid1@chromium.org, Dec 19 2016

Labels: merge-merged-factory-reef-8811.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/1cba94922d3218b614dba11e0fc3a3e878faea88

commit 1cba94922d3218b614dba11e0fc3a3e878faea88
Author: Cheng-Yi Chiang <cychiang@chromium.org>
Date: Fri Nov 11 10:55:31 2016

[autotest] audio_quality_measurement: Ignore detected noise at edge

In the beginning and the end of data, there will be false alarm noise
caused by edge effect of Hilbert transform. We should ignore them.
See the docstring of NEAR_DATA_START_OR_END_SECS for detail and example.

BUG= chromium:663616 
TEST=use check_quality tool to check the quality of the file attached on
the issue.

Change-Id: Ie7cbc1dea3986923f2421275116344a47a430199
Reviewed-on: https://chromium-review.googlesource.com/410581
Commit-Ready: Cheng-Yi Chiang <cychiang@chromium.org>
Tested-by: Cheng-Yi Chiang <cychiang@chromium.org>
Reviewed-by: Wai-Hong Tam <waihong@google.com>
(cherry picked from commit 93e9d9b04b2bea757953c1d1d7c5a4b4ade14b49)
Reviewed-on: https://chromium-review.googlesource.com/422121
Reviewed-by: Ting Shen <phoenixshen@chromium.org>
Commit-Queue: Ting Shen <phoenixshen@chromium.org>
Tested-by: Ting Shen <phoenixshen@chromium.org>

[modify] https://crrev.com/1cba94922d3218b614dba11e0fc3a3e878faea88/client/cros/audio/audio_quality_measurement.py

Project Member

Comment 9 by bugdroid1@chromium.org, Dec 19 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/6e1a9522d2e99c5eca153e0326b345fcfb22030c

commit 6e1a9522d2e99c5eca153e0326b345fcfb22030c
Author: Cheng-Yi Chiang <cychiang@chromium.org>
Date: Thu Nov 10 08:55:47 2016

[autotest] audio_data: Use numpy fromstring function

Use numpy fromstring function to read samples from data string.
This speeds up the time to read data from large file or large data
string and parse data input channel samples.

The speed boost is significant. For example, for 150 seconds long data,
the time to read data dropped from 140 seconds to 0.2 seconds.

Also, remove some unused functions.

BUG= chromium:663616 
TEST=Use check_quality tool, check the time to read samples becomes much
faster.

Change-Id: I6a9028445b3311cd01a3c75621eaf64b5b9d11b8
Reviewed-on: https://chromium-review.googlesource.com/409893
Commit-Ready: Cheng-Yi Chiang <cychiang@chromium.org>
Tested-by: Cheng-Yi Chiang <cychiang@chromium.org>
Reviewed-by: Kalin Stoyanov <kalin@chromium.org>
(cherry picked from commit 443d9083210bd0acd94e1e38153eb1d3a5312f90)
Reviewed-on: https://chromium-review.googlesource.com/422122
Reviewed-by: Ting Shen <phoenixshen@chromium.org>
Commit-Queue: Ting Shen <phoenixshen@chromium.org>
Tested-by: Ting Shen <phoenixshen@chromium.org>

[modify] https://crrev.com/6e1a9522d2e99c5eca153e0326b345fcfb22030c/client/cros/audio/audio_data.py

Sign in to add a comment