New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 9 users

Issue metadata

Status: Archived
Owner:
Last visit > 30 days ago
Closed: Nov 2017
Cc:
Components:
NextAction: ----
OS: ----
Pri: 2
Type: Other



Sign in to add a comment

Ideas for more automated audio NACK tests

Project Member Reported by tnakamura@webrtc.org, Jul 11 2013

Issue description

This is a follow up to  bug 1601 . See also  bug 2072  for ideas for future manual testing.

Turaj added unit tests as part of his audio NACK implementation. We brainstormed about other ideas for automated tests. I'm not sure if these tests could live somewhere in the VoE.

a) drop some audio packets, call the NACK API, and verify that those dropped packets are in the list the API returns
b) drop an audio packet, NACK the packet and say that you need the packet in X ms. Have the sender resend/inject the packet in time, then query NetEQ for packet loss metrics. NetEQ should report that there is zero packet loss.
c) drop an audio packet, NACK the packet and say that you need the packet in X ms. Have the sender resend/inject the packet too late, then query NetEQ for packet loss metrics. NetEQ should report that there is non-zero packet loss, and if it supports it, NetEQ should also report that a non-zero dropped packet value.

Andrew, let's talk about these ideas when you're back.
 
Project Member

Comment 1 by andrew@webrtc.org, Jul 15 2013

Cc: pwestin@webrtc.org mikhal@webrtc.org stefan@webrtc.org
I think it would be instructive to know what kind of testing (if any) is done for video NACK. Added Patrik, Mikhal and Stefan for details.

Comment 2 by holmer@google.com, Oct 15 2013

For video we have several unittests verifying that nack lists contain the right packets and that only complete frames are decoded. We also try to make sure that we verify corner cases, such as sequence number wraps.

We also have fairly simple integration tests. (https://code.google.com/p/webrtc/source/browse/trunk/webrtc/video_engine/test/call_tests.cc#284)

I would recommend that you unittest ACM with nack enabled to be able to also verify NetEq behavior. I would also add a voice engine integration test.

Comment 3 by vrk@webrtc.org, Oct 15 2014

Labels: Area-Audio
Project Member

Comment 4 by tina.legrand@webrtc.org, Oct 30 2014

Cc: tina.legrand@webrtc.org
Labels: EngTriaged IceBox
Project Member

Comment 5 by tina.legrand@webrtc.org, Apr 20 2015

Cc: kwiberg@webrtc.org jmarusic@webrtc.org
Project Member

Comment 6 by tina.legrand@webrtc.org, Nov 24 2015

Owner: tina.legrand@webrtc.org
Project Member

Comment 7 by kjellander@webrtc.org, Dec 1 2016

Cc: -jmarusic@webrtc.org
Project Member

Comment 8 by anatolid@webrtc.org, Dec 14 2016

Status: Assigned (was: Available)
Project Member

Comment 9 by tina.legrand@webrtc.org, Nov 7 2017

Status: Archived (was: Assigned)
[Bulk edit] This issue hasn't been modified the last twelve months -> archiving.

If this is still a valid issue that should be open, please reopen again.

Sign in to add a comment