Project: webrtc Issues People Development process History Sign in
New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Issue 2073 Ideas for more automated audio NACK tests
Starred by 9 users Project Member Reported by tnakamura@webrtc.org, Jul 11 2013 Back to list
Status: Assigned
Owner:
Cc:
Components:
OS: ----
Pri: 2
Type: Other



Sign in to add a comment
This is a follow up to  bug 1601 . See also bug 2072 for ideas for future manual testing.

Turaj added unit tests as part of his audio NACK implementation. We brainstormed about other ideas for automated tests. I'm not sure if these tests could live somewhere in the VoE.

a) drop some audio packets, call the NACK API, and verify that those dropped packets are in the list the API returns
b) drop an audio packet, NACK the packet and say that you need the packet in X ms. Have the sender resend/inject the packet in time, then query NetEQ for packet loss metrics. NetEQ should report that there is zero packet loss.
c) drop an audio packet, NACK the packet and say that you need the packet in X ms. Have the sender resend/inject the packet too late, then query NetEQ for packet loss metrics. NetEQ should report that there is non-zero packet loss, and if it supports it, NetEQ should also report that a non-zero dropped packet value.

Andrew, let's talk about these ideas when you're back.
 
Project Member Comment 1 by andrew@webrtc.org, Jul 15 2013
Cc: pwestin@webrtc.org mikhal@webrtc.org stefan@webrtc.org
I think it would be instructive to know what kind of testing (if any) is done for video NACK. Added Patrik, Mikhal and Stefan for details.
Comment 2 by holmer@google.com, Oct 15 2013
For video we have several unittests verifying that nack lists contain the right packets and that only complete frames are decoded. We also try to make sure that we verify corner cases, such as sequence number wraps.

We also have fairly simple integration tests. (https://code.google.com/p/webrtc/source/browse/trunk/webrtc/video_engine/test/call_tests.cc#284)

I would recommend that you unittest ACM with nack enabled to be able to also verify NetEq behavior. I would also add a voice engine integration test.
Comment 3 by vrk@webrtc.org, Oct 15 2014
Labels: Area-Audio
Project Member Comment 4 by tina.legrand@webrtc.org, Oct 30 2014
Cc: tina.legrand@webrtc.org
Labels: EngTriaged IceBox
Project Member Comment 5 by tina.legrand@webrtc.org, Apr 20 2015
Cc: kwiberg@webrtc.org jmarusic@webrtc.org
Project Member Comment 6 by tina.legrand@webrtc.org, Nov 24 2015
Owner: tina.legrand@webrtc.org
Project Member Comment 7 by kjellander@webrtc.org, Dec 1
Cc: -jmarusic@webrtc.org
Project Member Comment 8 by anatolid@webrtc.org, Dec 14
Status: Assigned
Sign in to add a comment