New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 9 users
Status: Archived
Owner:
Last visit 19 days ago
Closed: Nov 7
Cc:
Components:
NextAction: ----
OS: ----
Pri: 2
Type: Other



Sign in to add a comment
Ideas for more automated audio NACK tests
Project Member Reported by tnakamura@webrtc.org, Jul 11 2013 Back to list
This is a follow up to  bug 1601 . See also  bug 2072  for ideas for future manual testing.

Turaj added unit tests as part of his audio NACK implementation. We brainstormed about other ideas for automated tests. I'm not sure if these tests could live somewhere in the VoE.

a) drop some audio packets, call the NACK API, and verify that those dropped packets are in the list the API returns
b) drop an audio packet, NACK the packet and say that you need the packet in X ms. Have the sender resend/inject the packet in time, then query NetEQ for packet loss metrics. NetEQ should report that there is zero packet loss.
c) drop an audio packet, NACK the packet and say that you need the packet in X ms. Have the sender resend/inject the packet too late, then query NetEQ for packet loss metrics. NetEQ should report that there is non-zero packet loss, and if it supports it, NetEQ should also report that a non-zero dropped packet value.

Andrew, let's talk about these ideas when you're back.
 
Project Member Comment 1 by andrew@webrtc.org, Jul 15 2013
Cc: pwestin@webrtc.org mikhal@webrtc.org stefan@webrtc.org
I think it would be instructive to know what kind of testing (if any) is done for video NACK. Added Patrik, Mikhal and Stefan for details.
Comment 2 by holmer@google.com, Oct 15 2013
For video we have several unittests verifying that nack lists contain the right packets and that only complete frames are decoded. We also try to make sure that we verify corner cases, such as sequence number wraps.

We also have fairly simple integration tests. (https://code.google.com/p/webrtc/source/browse/trunk/webrtc/video_engine/test/call_tests.cc#284)

I would recommend that you unittest ACM with nack enabled to be able to also verify NetEq behavior. I would also add a voice engine integration test.
Comment 3 by vrk@webrtc.org, Oct 15 2014
Labels: Area-Audio
Project Member Comment 4 by tina.legrand@webrtc.org, Oct 30 2014
Cc: tina.legrand@webrtc.org
Labels: EngTriaged IceBox
Project Member Comment 5 by tina.legrand@webrtc.org, Apr 20 2015
Cc: kwiberg@webrtc.org jmarusic@webrtc.org
Project Member Comment 6 by tina.legrand@webrtc.org, Nov 24 2015
Owner: tina.legrand@webrtc.org
Project Member Comment 7 by kjellander@webrtc.org, Dec 1 2016
Cc: -jmarusic@webrtc.org
Project Member Comment 8 by anatolid@webrtc.org, Dec 14 2016
Status: Assigned
Project Member Comment 9 by tina.legrand@webrtc.org, Nov 7
Status: Archived
[Bulk edit] This issue hasn't been modified the last twelve months -> archiving.

If this is still a valid issue that should be open, please reopen again.
Sign in to add a comment