New issue
Advanced search Search tips

Issue 794722 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Refactor MediaCodecBridge and cover more logic in tests

Project Member Reported by sanfin@chromium.org, Dec 13 2017

Issue description

Background: I have tried several times to upstream a patch from the Chromecast fork, and each time, have found it difficult to cleanly add the new logic. The new feature in question is a followup to internal bug b/30515215, where it turned out that the conservative estimate for MediaFormat.KEY_MAX_WIDTH and MediaFormat.KEY_MAX_HEIGHT was problematic for some decoder implementations on Android TV. I could submit a hack that jams in the new feature in the conventional way, with lots of plumbing, but it would probably not be well-received in review, or worse, would be accepted without complaint and allowed to increase the entropy of MediaCodecBridge even further. Something as simple as plumbing in a strategy class for determining these keys turns out to be hard because of the convoluted setup logic for MediaCodecBridge. This convoluted setup logic includes:

1) Several JNI setter methods that are only supposed to be called before start() (though they are not explicitly documented as such)
2) Various static and non-static Java helper methods that set keys on a MediaFormat. Some are invoked in Java as side effects of other Java methods, and others are called from native code through JNI.
3) Parameters that are redundant depending on the values of other parameters (such as the "direction" parameter in MediaCodecBridge#create()).
4) Nested if statements that are hard to follow.

This makes it really hard to test, and therefore hard to know if a change broke any of the many combinations of platform exceptions, decoder types, and other parameters. There are unittests for MediaCodecBridgeImpl in native code, but it is hard to tell what Java logic is covered.

MediaCodecBridge, in Java and C++, can be refactored to better fit the single-responsibility principle. Here are some ideas on how to do this:

1) Move construction/configuration logic out of MediaCodecBridge.java and into a factory/builder class. The constructor of MediaCodecBridge can just take an already-configured MediaCodec instance
2) Split MediaCodecUtil into multiple helper classes to disambiguate between helper methods for native code (through media_codec_util.cc) and helper methods for Java code.
3) Extract concerns of configuring the MediaFormat and MediaCodec and encapsulate them in methods and classes that don't take redundant arguments.
4) Encapsulate SDK version checks underneath delegate classes or some other means, to make it less necessary to constantly keep track of what OS version we're running on.

As a result, MediaCodecBridge and MediaCodecUtil should be more readable and testable. JNI classes (i.e. the pair of C++ and Java classes with the same name) should be made as small as possible, and can be tested in native code, and helper methods on the Java side can be tested with JUnit tests that can easily mock dependencies. This would allow us to, for instance, test logic that depends on hard-coded whitelists, like the Build.MODEL check in MediaCodecBridge#maybeSetMaxInputSize(), in any JVM, providing much better test coverage.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Jan 6 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/96381699b93319e8eb824bb506b44572f5c046e9

commit 96381699b93319e8eb824bb506b44572f5c046e9
Author: Simeon Anfinrud <sanfin@chromium.org>
Date: Sat Jan 06 00:16:27 2018

[media] Extract builder class from MediaCodecBridge.

This detangles a lot of the setup logic in MediaCodecBridge,
clearly delineating logic that should happen before start().

The main goal of this CL is to reduce the number of JNI back-
and-forth in the create*coder static MediaCodecBridge methods.
Now all the native information is passed to Java code in one go,
while dedicated methods for each MediaCodec type help reduce
redundant arguments.

Bug: 794722
Bug: Internal b/31597555
Test: media_unittests
Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I17bfe37191e4c2bd03cd895dd0ba605c42b1b06b
Reviewed-on: https://chromium-review.googlesource.com/825986
Commit-Queue: Simeon Anfinrud <sanfin@chromium.org>
Reviewed-by: Frank Liberato <liberato@chromium.org>
Cr-Commit-Position: refs/heads/master@{#527444}
[modify] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/BUILD.gn
[add] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/java/src/org/chromium/media/HdrMetadata.java
[modify] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/java/src/org/chromium/media/MediaCodecBridge.java
[add] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/java/src/org/chromium/media/MediaCodecBridgeBuilder.java
[modify] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/java/src/org/chromium/media/MediaCodecEncoder.java
[modify] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/java/src/org/chromium/media/MediaCodecUtil.java
[add] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/jni_hdr_metadata.cc
[add] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/jni_hdr_metadata.h
[modify] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/media_codec_bridge.h
[modify] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/media_codec_bridge_impl.cc
[modify] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/base/android/media_codec_bridge_impl.h
[modify] https://crrev.com/96381699b93319e8eb824bb506b44572f5c046e9/media/filters/audio_decoder_unittest.cc

Project Member

Comment 2 by bugdroid1@chromium.org, Jan 8 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/feeebadab2709f63968b2ef5c977fb8e3f119dca

commit feeebadab2709f63968b2ef5c977fb8e3f119dca
Author: Simeon Anfinrud <sanfin@chromium.org>
Date: Mon Jan 08 19:40:24 2018

[media] Move behavior into BitrateAdjustmentTypes.

This isolates a concern that was spread across MediaCodecUtil
and MediaCodecBridge inside a single enum class.

Bug: 794722
Test: media_unittests

Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I49e46c404c314bc64509149bb8198ac32922f939
Reviewed-on: https://chromium-review.googlesource.com/853283
Commit-Queue: Simeon Anfinrud <sanfin@chromium.org>
Reviewed-by: Frank Liberato <liberato@chromium.org>
Cr-Commit-Position: refs/heads/master@{#527714}
[modify] https://crrev.com/feeebadab2709f63968b2ef5c977fb8e3f119dca/media/base/android/BUILD.gn
[add] https://crrev.com/feeebadab2709f63968b2ef5c977fb8e3f119dca/media/base/android/java/src/org/chromium/media/BitrateAdjuster.java
[modify] https://crrev.com/feeebadab2709f63968b2ef5c977fb8e3f119dca/media/base/android/java/src/org/chromium/media/MediaCodecBridge.java
[modify] https://crrev.com/feeebadab2709f63968b2ef5c977fb8e3f119dca/media/base/android/java/src/org/chromium/media/MediaCodecBridgeBuilder.java
[modify] https://crrev.com/feeebadab2709f63968b2ef5c977fb8e3f119dca/media/base/android/java/src/org/chromium/media/MediaCodecEncoder.java
[modify] https://crrev.com/feeebadab2709f63968b2ef5c977fb8e3f119dca/media/base/android/java/src/org/chromium/media/MediaCodecUtil.java
[add] https://crrev.com/feeebadab2709f63968b2ef5c977fb8e3f119dca/media/base/android/java/src/test/org/chromium/media/BitrateAdjusterTest.java

Project Member

Comment 3 by bugdroid1@chromium.org, Jan 17 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4cc15c26431a824da3d3485a92b526d0147b0e6d

commit 4cc15c26431a824da3d3485a92b526d0147b0e6d
Author: Simeon Anfinrud <sanfin@chromium.org>
Date: Wed Jan 17 19:09:41 2018

[media] Extract builder class for MediaFormat.

This helps separate logic between video and audio decoders and
encoders. It also allows more lightweight testing of internal
logic through JUnit tests.

Bug: 794722
Test: media_unittests
Test: media_base_junit_tests
Change-Id: If4741b46e557e384dd638907e4a86c4b0dbb3792
Reviewed-on: https://chromium-review.googlesource.com/868848
Reviewed-by: Frank Liberato <liberato@chromium.org>
Commit-Queue: Simeon Anfinrud <sanfin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#529838}
[modify] https://crrev.com/4cc15c26431a824da3d3485a92b526d0147b0e6d/media/base/android/BUILD.gn
[modify] https://crrev.com/4cc15c26431a824da3d3485a92b526d0147b0e6d/media/base/android/java/src/org/chromium/media/HdrMetadata.java
[modify] https://crrev.com/4cc15c26431a824da3d3485a92b526d0147b0e6d/media/base/android/java/src/org/chromium/media/MediaCodecBridge.java
[modify] https://crrev.com/4cc15c26431a824da3d3485a92b526d0147b0e6d/media/base/android/java/src/org/chromium/media/MediaCodecBridgeBuilder.java
[add] https://crrev.com/4cc15c26431a824da3d3485a92b526d0147b0e6d/media/base/android/java/src/org/chromium/media/MediaFormatBuilder.java
[add] https://crrev.com/4cc15c26431a824da3d3485a92b526d0147b0e6d/media/base/android/java/src/test/org/chromium/media/MediaFormatBuilderTest.java

Sign in to add a comment