New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 9642 link

Starred by 10 users

Issue metadata

Status: Assigned
Owner:
Last visit > 30 days ago
Cc:
Components:
NextAction: ----
OS: ----
Pri: 2
Type: Feature



Sign in to add a comment

Improve video quality analysis tool

Project Member Reported by magjed@webrtc.org, Aug 20

Issue description

We are working on improving the video quality analysis tools in WebRTC. Specifically, we are exploring using more sophisticated video quality metrics than PSNR or SSIM, such as MOtion-tuned Video Integrity Evaluation (MOVIE) index (https://ieeexplore.ieee.org/abstract/document/5290142/) or Video Multimethod Assessment Fusion (VMAF) https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652.

These more sophisticated video quality metrics typically require to have an aligned reference and test video, i.e. two videos with the same number of frames and where the frames match up to each other. Our current tool don't have the ability to align videos, and instead we find corresponding reference frames by inserting bar codes into the video. We would like to avoid inserting bar codes into the video since it's intrusive and actually alters the score, and it also requires pre- and post- processing of the videos which is a bit tedious. We would like to develop a tool that can align generic videos without the need for bar codes.

Eng owner: phensman

 
Project Member

Comment 1 by bugdroid1@chromium.org, Aug 23

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/404be7f302358e6be16aadeba8bc8f8aba348c0f

commit 404be7f302358e6be16aadeba8bc8f8aba348c0f
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Thu Aug 23 09:56:02 2018

Add Y4mFileReader

Encapsulate logic for reading .y4m video files in a single class. We
currently have spread out logic for opening .y4m files with partial
parsing. This CL consolidates this logic into a single class with a well
defined interface.

Change-Id: Id61673b3c95a0053b30e95b4cf382e1c6b05fc30
Bug: webrtc:9642
Reviewed-on: https://webrtc-review.googlesource.com/94772
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Paulina Hensman <phensman@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24398}
[delete] https://crrev.com/63af828a1baa96d67214861d7164b31c68b1f013/resources/video_quality_analysis_frame.txt.sha1
[modify] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/BUILD.gn
[modify] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/frame_analyzer/reference_less_video_analysis_lib.cc
[modify] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/frame_analyzer/reference_less_video_analysis_lib.h
[modify] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/frame_analyzer/reference_less_video_analysis_unittest.cc
[modify] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/frame_analyzer/video_quality_analysis.cc
[modify] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/frame_analyzer/video_quality_analysis.h
[modify] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/frame_analyzer/video_quality_analysis_unittest.cc
[modify] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/psnr_ssim_analyzer/psnr_ssim_analyzer.cc
[add] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/y4m_file_reader.cc
[add] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/y4m_file_reader.h
[add] https://crrev.com/404be7f302358e6be16aadeba8bc8f8aba348c0f/rtc_tools/y4m_file_reader_unittest.cc

Project Member

Comment 2 by bugdroid1@chromium.org, Aug 23

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/1b5e5db842971340eb9128985ddbaf0225a9d0b1

commit 1b5e5db842971340eb9128985ddbaf0225a9d0b1
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Thu Aug 23 11:04:27 2018

Remove SequencedTaskChecker from Y4mFileReader

SequencedTaskChecker is not part of rtc_base_approved and will not work
in Chromium. This CL simply removes it since it was just a precaution
and is not necessary for the tool. The thread assumptions are stated in
the class comment.

TBR=phensman@webrtc.org

Bug: webrtc:9642
Change-Id: I871ac361975595d8ed07b2e2447e3581c9ba9968
Reviewed-on: https://webrtc-review.googlesource.com/95648
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24401}
[modify] https://crrev.com/1b5e5db842971340eb9128985ddbaf0225a9d0b1/rtc_tools/BUILD.gn
[modify] https://crrev.com/1b5e5db842971340eb9128985ddbaf0225a9d0b1/rtc_tools/y4m_file_reader.cc
[modify] https://crrev.com/1b5e5db842971340eb9128985ddbaf0225a9d0b1/rtc_tools/y4m_file_reader.h

Project Member

Comment 3 by bugdroid1@chromium.org, Aug 23

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/b2c0e8f60fad10e2786e5e131136a0da1299d883

commit b2c0e8f60fad10e2786e5e131136a0da1299d883
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Thu Aug 23 15:35:28 2018

Add tool for aliging video files

This class adds logic for aligning a test video to a reference video
by an algorithm that maximizes SSIM between them. Aligned videos will be
easier to run video quality metrics on. This is a generic way of
aligning videos and can be replace the intrusive barcode stamping that
we currently use. This will be done in a follow-up CL.

Change-Id: I71cf1e2179c0f1e03eff9e4d8fc492fd5cfbbb1c
Bug: webrtc:9642
Reviewed-on: https://webrtc-review.googlesource.com/94773
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Paulina Hensman <phensman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24407}
[modify] https://crrev.com/b2c0e8f60fad10e2786e5e131136a0da1299d883/rtc_tools/BUILD.gn
[add] https://crrev.com/b2c0e8f60fad10e2786e5e131136a0da1299d883/rtc_tools/frame_analyzer/video_temporal_aligner.cc
[add] https://crrev.com/b2c0e8f60fad10e2786e5e131136a0da1299d883/rtc_tools/frame_analyzer/video_temporal_aligner.h
[add] https://crrev.com/b2c0e8f60fad10e2786e5e131136a0da1299d883/rtc_tools/frame_analyzer/video_temporal_aligner_unittest.cc

Project Member

Comment 4 by bugdroid1@chromium.org, Aug 24

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/d65e143801a7aaa9affdb939ea836aec1955cdcc

commit d65e143801a7aaa9affdb939ea836aec1955cdcc
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Fri Aug 24 09:26:14 2018

Update video_quality_analysis to align videos instead of using barcodes

This CL is a follow-up to the previous CL
https://webrtc-review.googlesource.com/c/src/+/94773 that added generic
logic for aligning videos. This will allow us to easily extend
video_quality_analysis with new sophisticated video quality metrics.
Also, we can use any kind of video that does not necessarily need to
contain bar codes. Removing the need to decode barcodes also leads to a
big speedup for the tests.

Bug: webrtc:9642
Change-Id: I74b0d630b3e1ed44781ad024115ded3143e28f50
Reviewed-on: https://webrtc-review.googlesource.com/94845
Reviewed-by: Paulina Hensman <phensman@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24423}
[modify] https://crrev.com/d65e143801a7aaa9affdb939ea836aec1955cdcc/examples/BUILD.gn
[delete] https://crrev.com/523b4c43303951cff837bf0269bde00b7b48acbe/rtc_tools/barcode_tools/DEPS
[delete] https://crrev.com/523b4c43303951cff837bf0269bde00b7b48acbe/rtc_tools/barcode_tools/README
[delete] https://crrev.com/523b4c43303951cff837bf0269bde00b7b48acbe/rtc_tools/barcode_tools/barcode_decoder.py
[delete] https://crrev.com/523b4c43303951cff837bf0269bde00b7b48acbe/rtc_tools/barcode_tools/barcode_encoder.py
[delete] https://crrev.com/523b4c43303951cff837bf0269bde00b7b48acbe/rtc_tools/barcode_tools/build_zxing.py
[delete] https://crrev.com/523b4c43303951cff837bf0269bde00b7b48acbe/rtc_tools/barcode_tools/helper_functions.py
[delete] https://crrev.com/523b4c43303951cff837bf0269bde00b7b48acbe/rtc_tools/barcode_tools/yuv_cropper.py
[modify] https://crrev.com/d65e143801a7aaa9affdb939ea836aec1955cdcc/rtc_tools/compare_videos.py
[modify] https://crrev.com/d65e143801a7aaa9affdb939ea836aec1955cdcc/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/d65e143801a7aaa9affdb939ea836aec1955cdcc/rtc_tools/frame_analyzer/video_quality_analysis.cc
[modify] https://crrev.com/d65e143801a7aaa9affdb939ea836aec1955cdcc/rtc_tools/frame_analyzer/video_quality_analysis.h
[modify] https://crrev.com/d65e143801a7aaa9affdb939ea836aec1955cdcc/rtc_tools/frame_analyzer/video_quality_analysis_unittest.cc
[modify] https://crrev.com/d65e143801a7aaa9affdb939ea836aec1955cdcc/rtc_tools/video_analysis.py

Project Member

Comment 5 by bugdroid1@chromium.org, Aug 24

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/3e169ac18caed5ce24ccb01c1614ebf394feee8c

commit 3e169ac18caed5ce24ccb01c1614ebf394feee8c
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Fri Aug 24 12:45:13 2018

Revert "Update video_quality_analysis to align videos instead of using barcodes"

This reverts commit d65e143801a7aaa9affdb939ea836aec1955cdcc.

Reason for revert: Breaks perf bots. frame_analyzer is a prebuilt binary, so it won't automatically pick up changes in the .cc file.

Original change's description:
> Update video_quality_analysis to align videos instead of using barcodes
> 
> This CL is a follow-up to the previous CL
> https://webrtc-review.googlesource.com/c/src/+/94773 that added generic
> logic for aligning videos. This will allow us to easily extend
> video_quality_analysis with new sophisticated video quality metrics.
> Also, we can use any kind of video that does not necessarily need to
> contain bar codes. Removing the need to decode barcodes also leads to a
> big speedup for the tests.
> 
> Bug: webrtc:9642
> Change-Id: I74b0d630b3e1ed44781ad024115ded3143e28f50
> Reviewed-on: https://webrtc-review.googlesource.com/94845
> Reviewed-by: Paulina Hensman <phensman@webrtc.org>
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24423}

TBR=phoglund@webrtc.org,magjed@webrtc.org,phensman@webrtc.org

Change-Id: Ia590b465687b861fe37ed1b14756d4607ca90da1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:9642
Reviewed-on: https://webrtc-review.googlesource.com/95946
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24428}
[modify] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/examples/BUILD.gn
[add] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/barcode_tools/DEPS
[add] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/barcode_tools/README
[add] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/barcode_tools/barcode_decoder.py
[add] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/barcode_tools/barcode_encoder.py
[add] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/barcode_tools/build_zxing.py
[add] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/barcode_tools/helper_functions.py
[add] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/barcode_tools/yuv_cropper.py
[modify] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/compare_videos.py
[modify] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/frame_analyzer/video_quality_analysis.cc
[modify] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/frame_analyzer/video_quality_analysis.h
[modify] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/frame_analyzer/video_quality_analysis_unittest.cc
[modify] https://crrev.com/3e169ac18caed5ce24ccb01c1614ebf394feee8c/rtc_tools/video_analysis.py

Project Member

Comment 6 by bugdroid1@chromium.org, Aug 24

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a

commit 9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Fri Aug 24 13:21:18 2018

Reland "Update video_quality_analysis to align videos instead of using barcodes"

This is a reland of d65e143801a7aaa9affdb939ea836aec1955cdcc

The binary for frame_analyzer.cpp is precompiled and stored in the cloud, so it
won't automatically pick up change to the source file. Therefore, restore all
old code to be backwards compatible.

Original change's description:
> Update video_quality_analysis to align videos instead of using barcodes
>
> This CL is a follow-up to the previous CL
> https://webrtc-review.googlesource.com/c/src/+/94773 that added generic
> logic for aligning videos. This will allow us to easily extend
> video_quality_analysis with new sophisticated video quality metrics.
> Also, we can use any kind of video that does not necessarily need to
> contain bar codes. Removing the need to decode barcodes also leads to a
> big speedup for the tests.
>
> Bug: webrtc:9642
> Change-Id: I74b0d630b3e1ed44781ad024115ded3143e28f50
> Reviewed-on: https://webrtc-review.googlesource.com/94845
> Reviewed-by: Paulina Hensman <phensman@webrtc.org>
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24423}

TBR=phensman@webrtc.org,phoglund@webrtc.org

Bug: webrtc:9642
Change-Id: Id8d129ce103284504c67690f8363c03eaae3eee7
Reviewed-on: https://webrtc-review.googlesource.com/96000
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24429}
[modify] https://crrev.com/9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a/examples/androidtests/video_quality_loopback_test.py
[modify] https://crrev.com/9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a/rtc_tools/frame_analyzer/video_quality_analysis.cc
[modify] https://crrev.com/9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a/rtc_tools/frame_analyzer/video_quality_analysis.h
[modify] https://crrev.com/9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a/rtc_tools/frame_analyzer/video_quality_analysis_unittest.cc

Project Member

Comment 7 by bugdroid1@chromium.org, Aug 24

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/5c2de6b3ce079cff52c411a2c02ce6553a38dc79

commit 5c2de6b3ce079cff52c411a2c02ce6553a38dc79
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Fri Aug 24 15:36:46 2018

Fix a bug in barcode_decoder.py

When converting from a .y4m file, it's illegal to pass a video_size
option since the resolution is already contained in the .y4m file.

TBR=phoglund@webrtc.org
NOTRY=TRUE

Bug: webrtc:9642
Change-Id: Iee7d2ba1332c45a1669af0fba43b0c3e7ce5846b
Reviewed-on: https://webrtc-review.googlesource.com/95949
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24431}
[modify] https://crrev.com/5c2de6b3ce079cff52c411a2c02ce6553a38dc79/rtc_tools/barcode_tools/barcode_decoder.py

Project Member

Comment 8 by bugdroid1@chromium.org, Aug 27

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/0673bc9204928f9c5566e8e816e22689c866681b

commit 0673bc9204928f9c5566e8e816e22689c866681b
Author: Sami Kalliomäki <sakal@webrtc.org>
Date: Mon Aug 27 16:50:54 2018

Revert CLs affecting video quality toolchain.

Speculatively fixes Chromium test for cut:  crbug.com/877968 

Reverts CLs:
https://webrtc-review.googlesource.com/c/src/+/94772
https://webrtc-review.googlesource.com/c/src/+/95648
https://webrtc-review.googlesource.com/c/src/+/94773
https://webrtc-review.googlesource.com/c/src/+/96000
https://webrtc-review.googlesource.com/c/src/+/95949

Revert "Add Y4mFileReader"

This reverts commit 404be7f302358e6be16aadeba8bc8f8aba348c0f.

Revert "Remove SequencedTaskChecker from Y4mFileReader"

This reverts commit 1b5e5db842971340eb9128985ddbaf0225a9d0b1.

Revert "Add tool for aliging video files"

This reverts commit b2c0e8f60fad10e2786e5e131136a0da1299d883.

Revert "Reland "Update video_quality_analysis to align videos instead of using barcodes""

This reverts commit 9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a.

Revert "Fix a bug in barcode_decoder.py"

This reverts commit 5c2de6b3ce079cff52c411a2c02ce6553a38dc79.

TBR=magjed@webrtc.org, phoglund@webrtc.org, phensman@webrtc.org

Bug:  chromium:877968 , webrtc:9642
Change-Id: I784d0598fd0370eec38d758b9fa0b38e4b3423be
Reviewed-on: https://webrtc-review.googlesource.com/96320
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Commit-Queue: Sami Kalliomäki <sakal@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24458}
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/examples/androidtests/video_quality_loopback_test.py
[add] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/resources/video_quality_analysis_frame.txt.sha1
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/BUILD.gn
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/barcode_tools/barcode_decoder.py
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/reference_less_video_analysis_lib.cc
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/reference_less_video_analysis_lib.h
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/reference_less_video_analysis_unittest.cc
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/video_quality_analysis.cc
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/video_quality_analysis.h
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/video_quality_analysis_unittest.cc
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/frame_analyzer/video_temporal_aligner.cc
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/frame_analyzer/video_temporal_aligner.h
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/frame_analyzer/video_temporal_aligner_unittest.cc
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/psnr_ssim_analyzer/psnr_ssim_analyzer.cc
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/y4m_file_reader.cc
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/y4m_file_reader.h
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/y4m_file_reader_unittest.cc

Project Member

Comment 9 by bugdroid1@chromium.org, Sep 4

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/5292654d7b7cbe4a7abe408a049aeca373cd856f

commit 5292654d7b7cbe4a7abe408a049aeca373cd856f
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Tue Sep 04 07:39:13 2018

Reland "Fix a bug in barcode_decoder.py"

This is a reland of 5c2de6b3ce079cff52c411a2c02ce6553a38dc79

Original change's description:
> Fix a bug in barcode_decoder.py
>
> When converting from a .y4m file, it's illegal to pass a video_size
> option since the resolution is already contained in the .y4m file.
>
> Bug: webrtc:9642
> Change-Id: Iee7d2ba1332c45a1669af0fba43b0c3e7ce5846b
> Reviewed-on: https://webrtc-review.googlesource.com/95949
> Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
> Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24431}

Bug: webrtc:9642
Change-Id: Iea6aad249839f9b1dad830bdf194cef2cc7dcfa6
Reviewed-on: https://webrtc-review.googlesource.com/97441
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24542}
[modify] https://crrev.com/5292654d7b7cbe4a7abe408a049aeca373cd856f/rtc_tools/barcode_tools/barcode_decoder.py

Project Member

Comment 10 by bugdroid1@chromium.org, Sep 4

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/5292654d7b7cbe4a7abe408a049aeca373cd856f

commit 5292654d7b7cbe4a7abe408a049aeca373cd856f
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Tue Sep 04 07:39:13 2018

Reland "Fix a bug in barcode_decoder.py"

This is a reland of 5c2de6b3ce079cff52c411a2c02ce6553a38dc79

Original change's description:
> Fix a bug in barcode_decoder.py
>
> When converting from a .y4m file, it's illegal to pass a video_size
> option since the resolution is already contained in the .y4m file.
>
> Bug: webrtc:9642
> Change-Id: Iee7d2ba1332c45a1669af0fba43b0c3e7ce5846b
> Reviewed-on: https://webrtc-review.googlesource.com/95949
> Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
> Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24431}

Bug: webrtc:9642
Change-Id: Iea6aad249839f9b1dad830bdf194cef2cc7dcfa6
Reviewed-on: https://webrtc-review.googlesource.com/97441
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24542}
[modify] https://crrev.com/5292654d7b7cbe4a7abe408a049aeca373cd856f/rtc_tools/barcode_tools/barcode_decoder.py

Project Member

Comment 11 by bugdroid1@chromium.org, Sep 4

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/5292654d7b7cbe4a7abe408a049aeca373cd856f

commit 5292654d7b7cbe4a7abe408a049aeca373cd856f
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Tue Sep 04 07:39:13 2018

Reland "Fix a bug in barcode_decoder.py"

This is a reland of 5c2de6b3ce079cff52c411a2c02ce6553a38dc79

Original change's description:
> Fix a bug in barcode_decoder.py
>
> When converting from a .y4m file, it's illegal to pass a video_size
> option since the resolution is already contained in the .y4m file.
>
> Bug: webrtc:9642
> Change-Id: Iee7d2ba1332c45a1669af0fba43b0c3e7ce5846b
> Reviewed-on: https://webrtc-review.googlesource.com/95949
> Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
> Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24431}

Bug: webrtc:9642
Change-Id: Iea6aad249839f9b1dad830bdf194cef2cc7dcfa6
Reviewed-on: https://webrtc-review.googlesource.com/97441
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24542}
[modify] https://crrev.com/5292654d7b7cbe4a7abe408a049aeca373cd856f/rtc_tools/barcode_tools/barcode_decoder.py

Project Member

Comment 12 by bugdroid1@chromium.org, Sep 5

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/10e829a2082339759947885575555c7df955d868

commit 10e829a2082339759947885575555c7df955d868
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Wed Sep 05 09:30:08 2018

Reland "Add Y4mFileReader"

This is a reland of 404be7f302358e6be16aadeba8bc8f8aba348c0f
It adds support for reading .yuv files as well to not break anything.

Original change's description:
> Add Y4mFileReader
>
> Encapsulate logic for reading .y4m video files in a single class. We
> currently have spread out logic for opening .y4m files with partial
> parsing. This CL consolidates this logic into a single class with a well
> defined interface.
>
> Change-Id: Id61673b3c95a0053b30e95b4cf382e1c6b05fc30
> Bug: webrtc:9642
> Reviewed-on: https://webrtc-review.googlesource.com/94772
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Reviewed-by: Paulina Hensman <phensman@webrtc.org>
> Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24398}

TBR=phensman,phoglund

Bug: webrtc:9642
Change-Id: Idecc5ec5da767221a5f5b439989f4fe07e3b3615
Reviewed-on: https://webrtc-review.googlesource.com/97983
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24571}
[delete] https://crrev.com/8f03216bbd254e90e86e366e2949848431e5d4f4/resources/video_quality_analysis_frame.txt.sha1
[modify] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/BUILD.gn
[modify] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/frame_analyzer/reference_less_video_analysis_lib.cc
[modify] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/frame_analyzer/reference_less_video_analysis_lib.h
[modify] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/frame_analyzer/reference_less_video_analysis_unittest.cc
[modify] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/frame_analyzer/video_quality_analysis.cc
[modify] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/frame_analyzer/video_quality_analysis.h
[modify] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/frame_analyzer/video_quality_analysis_unittest.cc
[modify] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/psnr_ssim_analyzer/psnr_ssim_analyzer.cc
[add] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/video_file_reader.cc
[add] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/video_file_reader.h
[add] https://crrev.com/10e829a2082339759947885575555c7df955d868/rtc_tools/video_file_reader_unittest.cc

Project Member

Comment 13 by bugdroid1@chromium.org, Sep 5

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/62228c41ea17b1f33ab85e80962a931ef74aa593

commit 62228c41ea17b1f33ab85e80962a931ef74aa593
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Wed Sep 05 13:30:16 2018

Reland "Add tool for aliging video files"

This is a reland of b2c0e8f60fad10e2786e5e131136a0da1299d883

Original change's description:
> Add tool for aliging video files
>
> This class adds logic for aligning a test video to a reference video
> by an algorithm that maximizes SSIM between them. Aligned videos will be
> easier to run video quality metrics on. This is a generic way of
> aligning videos and can be replace the intrusive barcode stamping that
> we currently use. This will be done in a follow-up CL.
>
> Change-Id: I71cf1e2179c0f1e03eff9e4d8fc492fd5cfbbb1c
> Bug: webrtc:9642
> Reviewed-on: https://webrtc-review.googlesource.com/94773
> Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Reviewed-by: Paulina Hensman <phensman@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24407}

TBR=phensman,phoglund

Bug: webrtc:9642
Change-Id: I35d6b0e598335b8d80fbfa37ba06d5c651bda4f6
Reviewed-on: https://webrtc-review.googlesource.com/98040
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24580}
[modify] https://crrev.com/62228c41ea17b1f33ab85e80962a931ef74aa593/rtc_tools/BUILD.gn
[add] https://crrev.com/62228c41ea17b1f33ab85e80962a931ef74aa593/rtc_tools/frame_analyzer/video_temporal_aligner.cc
[add] https://crrev.com/62228c41ea17b1f33ab85e80962a931ef74aa593/rtc_tools/frame_analyzer/video_temporal_aligner.h
[add] https://crrev.com/62228c41ea17b1f33ab85e80962a931ef74aa593/rtc_tools/frame_analyzer/video_temporal_aligner_unittest.cc

Project Member

Comment 14 by bugdroid1@chromium.org, Sep 5

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/b468aced4b02ec755fc581233a2d12f42d9c9650

commit b468aced4b02ec755fc581233a2d12f42d9c9650
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Wed Sep 05 14:41:15 2018

Reland "Reland "Update video_quality_analysis to align videos instead of using barcodes""

This is a reland of 9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a

Original change's description:
> Reland "Update video_quality_analysis to align videos instead of using barcodes"
>
> This is a reland of d65e143801a7aaa9affdb939ea836aec1955cdcc
>
> The binary for frame_analyzer.cpp is precompiled and stored in the cloud, so it
> won't automatically pick up change to the source file. Therefore, restore all
> old code to be backwards compatible.
>
> Original change's description:
> > Update video_quality_analysis to align videos instead of using barcodes
> >
> > This CL is a follow-up to the previous CL
> > https://webrtc-review.googlesource.com/c/src/+/94773 that added generic
> > logic for aligning videos. This will allow us to easily extend
> > video_quality_analysis with new sophisticated video quality metrics.
> > Also, we can use any kind of video that does not necessarily need to
> > contain bar codes. Removing the need to decode barcodes also leads to a
> > big speedup for the tests.
> >
> > Bug: webrtc:9642
> > Change-Id: I74b0d630b3e1ed44781ad024115ded3143e28f50
> > Reviewed-on: https://webrtc-review.googlesource.com/94845
> > Reviewed-by: Paulina Hensman <phensman@webrtc.org>
> > Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> > Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> > Cr-Commit-Position: refs/heads/master@{#24423}
>
> TBR=phensman@webrtc.org,phoglund@webrtc.org
>
> Bug: webrtc:9642
> Change-Id: Id8d129ce103284504c67690f8363c03eaae3eee7
> Reviewed-on: https://webrtc-review.googlesource.com/96000
> Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24429}

TBR=phensman,phoglund

Bug: webrtc:9642
Change-Id: Ic248b7831ae148251a1a4ebeec5d154286f91a0a
Reviewed-on: https://webrtc-review.googlesource.com/98080
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24583}
[modify] https://crrev.com/b468aced4b02ec755fc581233a2d12f42d9c9650/rtc_tools/BUILD.gn
[modify] https://crrev.com/b468aced4b02ec755fc581233a2d12f42d9c9650/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/b468aced4b02ec755fc581233a2d12f42d9c9650/rtc_tools/frame_analyzer/video_quality_analysis.cc
[modify] https://crrev.com/b468aced4b02ec755fc581233a2d12f42d9c9650/rtc_tools/frame_analyzer/video_quality_analysis.h
[modify] https://crrev.com/b468aced4b02ec755fc581233a2d12f42d9c9650/rtc_tools/frame_analyzer/video_quality_analysis_unittest.cc
[modify] https://crrev.com/b468aced4b02ec755fc581233a2d12f42d9c9650/rtc_tools/video_file_reader.cc

Project Member

Comment 15 by bugdroid1@chromium.org, Sep 24

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/b671d46f913d15bff3095b8b692d58d85bc7b551

commit b671d46f913d15bff3095b8b692d58d85bc7b551
Author: Paulina Hensman <phensman@webrtc.org>
Date: Mon Sep 24 08:03:10 2018

Add WriteVideoToFile to video_file_reader.

The function checks the file extension to determine YUV or Y4M format.

Also adds a flag aligned_output_file to compare_videos.py, which allows
saving the aligned reference video to a file.

Bug: webrtc:9642
Change-Id: Ia59f5c123a1e41104756eb6b235b6581c4ffbd77
Reviewed-on: https://webrtc-review.googlesource.com/99503
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Paulina Hensman <phensman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24787}
[modify] https://crrev.com/b671d46f913d15bff3095b8b692d58d85bc7b551/rtc_tools/BUILD.gn
[modify] https://crrev.com/b671d46f913d15bff3095b8b692d58d85bc7b551/rtc_tools/compare_videos.py
[modify] https://crrev.com/b671d46f913d15bff3095b8b692d58d85bc7b551/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/b671d46f913d15bff3095b8b692d58d85bc7b551/rtc_tools/frame_analyzer/video_temporal_aligner.cc
[modify] https://crrev.com/b671d46f913d15bff3095b8b692d58d85bc7b551/rtc_tools/frame_analyzer/video_temporal_aligner.h
[add] https://crrev.com/b671d46f913d15bff3095b8b692d58d85bc7b551/rtc_tools/video_file_writer.cc
[add] https://crrev.com/b671d46f913d15bff3095b8b692d58d85bc7b551/rtc_tools/video_file_writer.h
[add] https://crrev.com/b671d46f913d15bff3095b8b692d58d85bc7b551/rtc_tools/video_file_writer_unittest.cc

Project Member

Comment 16 by bugdroid1@chromium.org, Sep 24

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/dd8de18784ad73e617c5715bde74baf71c4853e3

commit dd8de18784ad73e617c5715bde74baf71c4853e3
Author: Paulina Hensman <phensman@webrtc.org>
Date: Mon Sep 24 11:18:41 2018

Include stringutils to allow build on chromium

Bug: webrtc:9642
Change-Id: Idad8d7a61f8b289c185590b64c79974f81a414e3
Reviewed-on: https://webrtc-review.googlesource.com/101541
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Paulina Hensman <phensman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24797}
[modify] https://crrev.com/dd8de18784ad73e617c5715bde74baf71c4853e3/rtc_tools/video_file_writer.cc

Project Member

Comment 17 by bugdroid1@chromium.org, Sep 28

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/e307d56bd7e192c354871a739bc0133d88cb5379

commit e307d56bd7e192c354871a739bc0133d88cb5379
Author: Paulina Hensman <phensman@webrtc.org>
Date: Fri Sep 28 07:02:23 2018

Add option to call VMAF in compare_videos.py.

VMAF compares videos on several metrics and produces a unified score.

Calling it from compare_videos required passing in a path to a VMAF
directory, where there should be a C++ wrapper executable and a model.
For now, the relative paths to those are constant.

VMAF needs to compare aligned videos in YUV format, so two videos
(ref and test) will be saved by frame_analyzer after it has aligned
them.

Bug: webrtc:9642
Change-Id: Idddfcf6b1b235e7f925696ffc38938fb84c4ff9e
Reviewed-on: https://webrtc-review.googlesource.com/102140
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Commit-Queue: Paulina Hensman <phensman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24876}
[modify] https://crrev.com/e307d56bd7e192c354871a739bc0133d88cb5379/rtc_tools/compare_videos.py
[modify] https://crrev.com/e307d56bd7e192c354871a739bc0133d88cb5379/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/e307d56bd7e192c354871a739bc0133d88cb5379/rtc_tools/video_file_writer.cc

Project Member

Comment 18 by bugdroid1@chromium.org, Sep 28

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/371781435a027d6848b18e03caddcba2192948cd

commit 371781435a027d6848b18e03caddcba2192948cd
Author: Sami Kalliomäki <sakal@webrtc.org>
Date: Fri Sep 28 09:19:48 2018

Revert "Add option to call VMAF in compare_videos.py."

This reverts commit e307d56bd7e192c354871a739bc0133d88cb5379.

Reason for revert:
Breaks client.webrtc.perf bots. Example failure:
https://ci.chromium.org/buildbot/client.webrtc.perf/Android32%20Tests%20(L%20Nexus7.2)/8635

AttributeError: Values instance has no attribute 'yuv_directory'

Original change's description:
> Add option to call VMAF in compare_videos.py.
> 
> VMAF compares videos on several metrics and produces a unified score.
> 
> Calling it from compare_videos required passing in a path to a VMAF
> directory, where there should be a C++ wrapper executable and a model.
> For now, the relative paths to those are constant.
> 
> VMAF needs to compare aligned videos in YUV format, so two videos
> (ref and test) will be saved by frame_analyzer after it has aligned
> them.
> 
> Bug: webrtc:9642
> Change-Id: Idddfcf6b1b235e7f925696ffc38938fb84c4ff9e
> Reviewed-on: https://webrtc-review.googlesource.com/102140
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
> Commit-Queue: Paulina Hensman <phensman@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24876}

TBR=phoglund@webrtc.org,sakal@webrtc.org,phensman@webrtc.org

Change-Id: I3e1dc98d7dfc0309ee2934cb3a978eecf274c477
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:9642
Reviewed-on: https://webrtc-review.googlesource.com/102561
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Commit-Queue: Sami Kalliomäki <sakal@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24883}
[modify] https://crrev.com/371781435a027d6848b18e03caddcba2192948cd/rtc_tools/compare_videos.py
[modify] https://crrev.com/371781435a027d6848b18e03caddcba2192948cd/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/371781435a027d6848b18e03caddcba2192948cd/rtc_tools/video_file_writer.cc

Project Member

Comment 19 by bugdroid1@chromium.org, Sep 28

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/12c62b922b40fb01e1d314442cfae769345f1209

commit 12c62b922b40fb01e1d314442cfae769345f1209
Author: Paulina Hensman <phensman@webrtc.org>
Date: Fri Sep 28 14:08:10 2018

Reland "Add option to call VMAF in compare_videos.py."

This is a reland of e307d56bd7e192c354871a739bc0133d88cb5379

options.yuv_directory would be unset if vmaf was not used.
It now gets set to None.

Also adds a try-finally around the temp directory for YUV files.

Original change's description:
> Add option to call VMAF in compare_videos.py.
>
> VMAF compares videos on several metrics and produces a unified score.
>
> Calling it from compare_videos required passing in a path to a VMAF
> executable and a model.
>
> VMAF needs to compare aligned videos in YUV format, so two videos
> (ref and test) will be saved by frame_analyzer after it has aligned
> them.
>
> Bug: webrtc:9642
> Change-Id: Idddfcf6b1b235e7f925696ffc38938fb84c4ff9e
> Reviewed-on: https://webrtc-review.googlesource.com/102140
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
> Commit-Queue: Paulina Hensman <phensman@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24876}

Bug: webrtc:9642
Change-Id: I1d04a56090e68df47dc3e6b7e710384244470d0c
TBR: phoglund
Reviewed-on: https://webrtc-review.googlesource.com/102544
Commit-Queue: Paulina Hensman <phensman@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24896}
[modify] https://crrev.com/12c62b922b40fb01e1d314442cfae769345f1209/rtc_tools/compare_videos.py
[modify] https://crrev.com/12c62b922b40fb01e1d314442cfae769345f1209/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/12c62b922b40fb01e1d314442cfae769345f1209/rtc_tools/video_file_writer.cc

Project Member

Comment 20 by bugdroid1@chromium.org, Oct 1

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/1c60ff521eda26c80fa53097d9c614f10200f651

commit 1c60ff521eda26c80fa53097d9c614f10200f651
Author: Paulina Hensman <phensman@webrtc.org>
Date: Mon Oct 01 12:34:49 2018

Tidy up and increase exception handling in compare_videos

Bug: webrtc:9642
Change-Id: I5c8b252de3b285f81a5437af99d789b5a28ce646
Reviewed-on: https://webrtc-review.googlesource.com/102880
Commit-Queue: Paulina Hensman <phensman@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24909}
[modify] https://crrev.com/1c60ff521eda26c80fa53097d9c614f10200f651/rtc_tools/BUILD.gn
[modify] https://crrev.com/1c60ff521eda26c80fa53097d9c614f10200f651/rtc_tools/compare_videos.py
[modify] https://crrev.com/1c60ff521eda26c80fa53097d9c614f10200f651/rtc_tools/frame_analyzer/frame_analyzer.cc

Project Member

Comment 21 by bugdroid1@chromium.org, Oct 1

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/3d3e08b2b122b2a7152a06695a65a72b69ee7079

commit 3d3e08b2b122b2a7152a06695a65a72b69ee7079
Author: Oleh Prypin <oprypin@webrtc.org>
Date: Mon Oct 01 13:21:31 2018

Revert "Tidy up and increase exception handling in compare_videos"

This reverts commit 1c60ff521eda26c80fa53097d9c614f10200f651.

Reason for revert: Breaks downstream tests:
non-test target compare_videos depends on testonly target frame_analyzer

Original change's description:
> Tidy up and increase exception handling in compare_videos
> 
> Bug: webrtc:9642
> Change-Id: I5c8b252de3b285f81a5437af99d789b5a28ce646
> Reviewed-on: https://webrtc-review.googlesource.com/102880
> Commit-Queue: Paulina Hensman <phensman@webrtc.org>
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24909}

TBR=phoglund@webrtc.org,sakal@webrtc.org,phensman@webrtc.org

Change-Id: I69c94248faf7d448b871b91548336ff681e4d139
No-Try: true
Bug: webrtc:9642
Reviewed-on: https://webrtc-review.googlesource.com/102921
Reviewed-by: Oleh Prypin <oprypin@webrtc.org>
Commit-Queue: Oleh Prypin <oprypin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24911}
[modify] https://crrev.com/3d3e08b2b122b2a7152a06695a65a72b69ee7079/rtc_tools/BUILD.gn
[modify] https://crrev.com/3d3e08b2b122b2a7152a06695a65a72b69ee7079/rtc_tools/compare_videos.py
[modify] https://crrev.com/3d3e08b2b122b2a7152a06695a65a72b69ee7079/rtc_tools/frame_analyzer/frame_analyzer.cc

Project Member

Comment 22 by bugdroid1@chromium.org, Oct 5

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/a6471eb58949b9d084ac585f7dc4abf965f60ada

commit a6471eb58949b9d084ac585f7dc4abf965f60ada
Author: Paulina Hensman <phensman@webrtc.org>
Date: Fri Oct 05 15:10:17 2018

Reland "Tidy up and increase exception handling in compare_videos"

This is a reland of 1c60ff521eda26c80fa53097d9c614f10200f651

Original change's description:
> Tidy up and increase exception handling in compare_videos
> 
> Bug: webrtc:9642
> Change-Id: I5c8b252de3b285f81a5437af99d789b5a28ce646
> Reviewed-on: https://webrtc-review.googlesource.com/102880
> Commit-Queue: Paulina Hensman <phensman@webrtc.org>
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#24909}

Bug: webrtc:9642
Change-Id: I11078a358297ae5046991ac3b0680df468bb413b
Reviewed-on: https://webrtc-review.googlesource.com/c/102941
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Paulina Hensman <phensman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25024}
[modify] https://crrev.com/a6471eb58949b9d084ac585f7dc4abf965f60ada/rtc_tools/compare_videos.py
[modify] https://crrev.com/a6471eb58949b9d084ac585f7dc4abf965f60ada/rtc_tools/frame_analyzer/frame_analyzer.cc

Project Member

Comment 23 by bugdroid1@chromium.org, Oct 10

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/ede87964ba7a381ff051fb7ef98cf9b5eb06f23b

commit ede87964ba7a381ff051fb7ef98cf9b5eb06f23b
Author: Paulina Hensman <phensman@webrtc.org>
Date: Wed Oct 10 14:31:00 2018

Print per-frame VMAF score instead of average.

compare_videos.py will now print the VMAF score for each frame.
The CL also removes some stale comments.

Bug: webrtc:9642
Change-Id: I5623588580dea06dd487d7763dc3a2511bd2cd3c
Reviewed-on: https://webrtc-review.googlesource.com/c/105103
Commit-Queue: Paulina Hensman <phensman@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25092}
[modify] https://crrev.com/ede87964ba7a381ff051fb7ef98cf9b5eb06f23b/rtc_tools/compare_videos.py

Project Member

Comment 24 by bugdroid1@chromium.org, Oct 16

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/1927dfafabc06a7eced67d1d3fd60f2c0c88c152

commit 1927dfafabc06a7eced67d1d3fd60f2c0c88c152
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Tue Oct 16 07:55:37 2018

Add tool for aligning color space of video files

This class adds logic for aligning color space of a test video compared
to a reference video. If there is a color space mismatch, it typically
does not have much impact on human perception, but it has a big impact
on PSNR and SSIM calculations. For example, aligning a test run with VP8
improves PSNR and SSIM from:
Average PSNR: 29.142818, average SSIM: 0.946026
to:
Average PSNR: 38.146229, average SSIM: 0.965388.

The optiomal color transformation between the two videos were:
0.86 0.01 0.00 14.37
0.00 0.88 0.00 15.32
0.00 0.00 0.88 15.74
which is converting YUV full range to YUV limited range. There is
already a CL out for fixing this discrepancy here:
https://webrtc-review.googlesource.com/c/src/+/94543

After that, hopefully there is no color space mismatch when saving the
raw YUV values. It's good that the video quality tool is color space
agnostic anyway, and can compensate for differences when the test
video is obtained by e.g. filming a physical device screen.

Also, the linear least square logic will be used for compensating
geometric distorisions in a follow-up CL.

Bug: webrtc:9642
Change-Id: I499713960a0544d8e45c5d09886e68ec829b28a7
Reviewed-on: https://webrtc-review.googlesource.com/c/95950
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25193}
[modify] https://crrev.com/1927dfafabc06a7eced67d1d3fd60f2c0c88c152/rtc_tools/BUILD.gn
[add] https://crrev.com/1927dfafabc06a7eced67d1d3fd60f2c0c88c152/rtc_tools/frame_analyzer/linear_least_squares.cc
[add] https://crrev.com/1927dfafabc06a7eced67d1d3fd60f2c0c88c152/rtc_tools/frame_analyzer/linear_least_squares.h
[add] https://crrev.com/1927dfafabc06a7eced67d1d3fd60f2c0c88c152/rtc_tools/frame_analyzer/linear_least_squares_unittest.cc
[add] https://crrev.com/1927dfafabc06a7eced67d1d3fd60f2c0c88c152/rtc_tools/frame_analyzer/video_color_aligner.cc
[add] https://crrev.com/1927dfafabc06a7eced67d1d3fd60f2c0c88c152/rtc_tools/frame_analyzer/video_color_aligner.h
[add] https://crrev.com/1927dfafabc06a7eced67d1d3fd60f2c0c88c152/rtc_tools/frame_analyzer/video_color_aligner_unittest.cc

Project Member

Comment 25 by bugdroid1@chromium.org, Oct 16

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/3100fc1f0575433e419fc1fda65a7e2ec26abb32

commit 3100fc1f0575433e419fc1fda65a7e2ec26abb32
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Tue Oct 16 13:49:04 2018

Use color aligning in video quality analysis tool

Bug: webrtc:9642
Change-Id: I217e054c20f26cf788dd97f42e7e4ade1a879fe7
Reviewed-on: https://webrtc-review.googlesource.com/c/98980
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25209}
[modify] https://crrev.com/3100fc1f0575433e419fc1fda65a7e2ec26abb32/rtc_tools/frame_analyzer/video_quality_analysis.cc
[modify] https://crrev.com/3100fc1f0575433e419fc1fda65a7e2ec26abb32/rtc_tools/frame_analyzer/video_temporal_aligner.cc

Project Member

Comment 26 by bugdroid1@chromium.org, Oct 22

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/ff292f30d9a4b7a56aea872fe488d342f47202a3

commit ff292f30d9a4b7a56aea872fe488d342f47202a3
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Mon Oct 22 11:36:40 2018

Remove deprecated barcode scanning functionality

This code is not used anymore, but it's not possible to land this CL
until  issue webrtc:9665  is fixed.

Bug: webrtc:9642, webrtc:9665 
Change-Id: Idb68e9bdf51b4239788cd6869dcb44dae87d7c56
Reviewed-on: https://webrtc-review.googlesource.com/c/95951
Reviewed-by: Paulina Hensman <phensman@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25289}
[modify] https://crrev.com/ff292f30d9a4b7a56aea872fe488d342f47202a3/examples/BUILD.gn
[modify] https://crrev.com/ff292f30d9a4b7a56aea872fe488d342f47202a3/examples/androidtests/video_quality_loopback_test.py
[delete] https://crrev.com/635474e3d5507d1d6cfe22b9483aaa6bd51fbc2f/rtc_tools/barcode_tools/DEPS
[delete] https://crrev.com/635474e3d5507d1d6cfe22b9483aaa6bd51fbc2f/rtc_tools/barcode_tools/README
[delete] https://crrev.com/635474e3d5507d1d6cfe22b9483aaa6bd51fbc2f/rtc_tools/barcode_tools/barcode_decoder.py
[delete] https://crrev.com/635474e3d5507d1d6cfe22b9483aaa6bd51fbc2f/rtc_tools/barcode_tools/barcode_encoder.py
[delete] https://crrev.com/635474e3d5507d1d6cfe22b9483aaa6bd51fbc2f/rtc_tools/barcode_tools/build_zxing.py
[delete] https://crrev.com/635474e3d5507d1d6cfe22b9483aaa6bd51fbc2f/rtc_tools/barcode_tools/helper_functions.py
[delete] https://crrev.com/635474e3d5507d1d6cfe22b9483aaa6bd51fbc2f/rtc_tools/barcode_tools/yuv_cropper.py
[modify] https://crrev.com/ff292f30d9a4b7a56aea872fe488d342f47202a3/rtc_tools/compare_videos.py
[modify] https://crrev.com/ff292f30d9a4b7a56aea872fe488d342f47202a3/rtc_tools/video_analysis.py
[delete] https://crrev.com/635474e3d5507d1d6cfe22b9483aaa6bd51fbc2f/tools_webrtc/video_quality_toolchain/linux/zxing.sha1
[delete] https://crrev.com/635474e3d5507d1d6cfe22b9483aaa6bd51fbc2f/tools_webrtc/video_quality_toolchain/mac/zxing.sha1
[delete] https://crrev.com/635474e3d5507d1d6cfe22b9483aaa6bd51fbc2f/tools_webrtc/video_quality_toolchain/win/zxing.exe.sha1

Project Member

Comment 27 by bugdroid1@chromium.org, Oct 22

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/38a34198a3ba63919afb721a74deef85e9aa56d2

commit 38a34198a3ba63919afb721a74deef85e9aa56d2
Author: Alessio Bazzica <alessiob@webrtc.org>
Date: Mon Oct 22 13:00:44 2018

Revert "Remove deprecated barcode scanning functionality"

This reverts commit ff292f30d9a4b7a56aea872fe488d342f47202a3.

Reason for revert: issues with downstream projects

Original change's description:
> Remove deprecated barcode scanning functionality
> 
> This code is not used anymore, but it's not possible to land this CL
> until  issue webrtc:9665  is fixed.
> 
> Bug: webrtc:9642, webrtc:9665 
> Change-Id: Idb68e9bdf51b4239788cd6869dcb44dae87d7c56
> Reviewed-on: https://webrtc-review.googlesource.com/c/95951
> Reviewed-by: Paulina Hensman <phensman@webrtc.org>
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#25289}

TBR=phoglund@webrtc.org,mbonadei@webrtc.org,magjed@webrtc.org,phensman@webrtc.org

Change-Id: I440025777a17d8580526289d4198da1fc3f7d62e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:9642,  webrtc:9665 
Reviewed-on: https://webrtc-review.googlesource.com/c/107348
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25291}
[modify] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/examples/BUILD.gn
[modify] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/examples/androidtests/video_quality_loopback_test.py
[add] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/rtc_tools/barcode_tools/DEPS
[add] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/rtc_tools/barcode_tools/README
[add] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/rtc_tools/barcode_tools/barcode_decoder.py
[add] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/rtc_tools/barcode_tools/barcode_encoder.py
[add] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/rtc_tools/barcode_tools/build_zxing.py
[add] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/rtc_tools/barcode_tools/helper_functions.py
[add] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/rtc_tools/barcode_tools/yuv_cropper.py
[modify] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/rtc_tools/compare_videos.py
[modify] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/rtc_tools/video_analysis.py
[add] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/tools_webrtc/video_quality_toolchain/linux/zxing.sha1
[add] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/tools_webrtc/video_quality_toolchain/mac/zxing.sha1
[add] https://crrev.com/38a34198a3ba63919afb721a74deef85e9aa56d2/tools_webrtc/video_quality_toolchain/win/zxing.exe.sha1

Project Member

Comment 28 by bugdroid1@chromium.org, Oct 22

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/165148d84d00bc879d955cdefe838289bfaa4143

commit 165148d84d00bc879d955cdefe838289bfaa4143
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Mon Oct 22 21:30:58 2018

Reland "Remove deprecated barcode scanning functionality"

This is a reland of ff292f30d9a4b7a56aea872fe488d342f47202a3

I'm leaving empty .py files in place in order to not break downstream client builds.

Original change's description:
> Remove deprecated barcode scanning functionality
>
> This code is not used anymore, but it's not possible to land this CL
> until  issue webrtc:9665  is fixed.
>
> Bug: webrtc:9642, webrtc:9665 
> Change-Id: Idb68e9bdf51b4239788cd6869dcb44dae87d7c56
> Reviewed-on: https://webrtc-review.googlesource.com/c/95951
> Reviewed-by: Paulina Hensman <phensman@webrtc.org>
> Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
> Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#25289}

TBR=phensman@webrtc.org,phoglund@webrtc.org

Bug: webrtc:9642,  webrtc:9665 
Change-Id: I248f8656b14c89b0b92e777f4408ee6a6dad41f9
Reviewed-on: https://webrtc-review.googlesource.com/c/107360
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25296}
[modify] https://crrev.com/165148d84d00bc879d955cdefe838289bfaa4143/examples/BUILD.gn
[modify] https://crrev.com/165148d84d00bc879d955cdefe838289bfaa4143/examples/androidtests/video_quality_loopback_test.py
[delete] https://crrev.com/39feabe35c5175a2c497b3b4ac79787c1bc14ab1/rtc_tools/barcode_tools/DEPS
[delete] https://crrev.com/39feabe35c5175a2c497b3b4ac79787c1bc14ab1/rtc_tools/barcode_tools/README
[modify] https://crrev.com/165148d84d00bc879d955cdefe838289bfaa4143/rtc_tools/barcode_tools/barcode_decoder.py
[delete] https://crrev.com/39feabe35c5175a2c497b3b4ac79787c1bc14ab1/rtc_tools/barcode_tools/barcode_encoder.py
[delete] https://crrev.com/39feabe35c5175a2c497b3b4ac79787c1bc14ab1/rtc_tools/barcode_tools/build_zxing.py
[modify] https://crrev.com/165148d84d00bc879d955cdefe838289bfaa4143/rtc_tools/barcode_tools/helper_functions.py
[delete] https://crrev.com/39feabe35c5175a2c497b3b4ac79787c1bc14ab1/rtc_tools/barcode_tools/yuv_cropper.py
[modify] https://crrev.com/165148d84d00bc879d955cdefe838289bfaa4143/rtc_tools/compare_videos.py
[modify] https://crrev.com/165148d84d00bc879d955cdefe838289bfaa4143/rtc_tools/video_analysis.py
[delete] https://crrev.com/39feabe35c5175a2c497b3b4ac79787c1bc14ab1/tools_webrtc/video_quality_toolchain/linux/zxing.sha1
[delete] https://crrev.com/39feabe35c5175a2c497b3b4ac79787c1bc14ab1/tools_webrtc/video_quality_toolchain/mac/zxing.sha1
[delete] https://crrev.com/39feabe35c5175a2c497b3b4ac79787c1bc14ab1/tools_webrtc/video_quality_toolchain/win/zxing.exe.sha1

Project Member

Comment 29 by bugdroid1@chromium.org, Nov 22

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/286df00f724843367f24e3d4e32bd380807b1397

commit 286df00f724843367f24e3d4e32bd380807b1397
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Thu Nov 22 15:30:15 2018

Add tool for aligning cropped region of video files

This class adds logic for aligning what part of a test video has been
encoded from a reference video. It does that by cropping and zooming in
on a region of the reference video that most closely matches the test
video. A small cropping does not have much impact on human perception,
but it has a big impact on PSNR and SSIM calculations.

For example, if the test video is cropped with one row in the top and
bottom, adjusting for this improves average PSNR from 27.7146 to
29.3357 and average SSIM from 0.934891 to 0.95318 in an example test
video.

TBR=phoglund

Bug: webrtc:9642
Change-Id: I02cfe0e2261fb58df8cdb1e15ba93285e3dc4538
Reviewed-on: https://webrtc-review.googlesource.com/c/99480
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@google.com>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25755}
[modify] https://crrev.com/286df00f724843367f24e3d4e32bd380807b1397/rtc_tools/BUILD.gn
[add] https://crrev.com/286df00f724843367f24e3d4e32bd380807b1397/rtc_tools/frame_analyzer/video_geometry_aligner.cc
[add] https://crrev.com/286df00f724843367f24e3d4e32bd380807b1397/rtc_tools/frame_analyzer/video_geometry_aligner.h
[add] https://crrev.com/286df00f724843367f24e3d4e32bd380807b1397/rtc_tools/frame_analyzer/video_geometry_aligner_unittest.cc

Project Member

Comment 30 by bugdroid1@chromium.org, Nov 22

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/f259078009238708daa9fbe5027adf1c182ceae2

commit f259078009238708daa9fbe5027adf1c182ceae2
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Thu Nov 22 15:44:14 2018

Use cropping aligning in video quality analysis tool

TBR=phoglund

Bug: webrtc:9642
Change-Id: I32e54473ef6699b862b36c36c7d975b381db6ed2
Reviewed-on: https://webrtc-review.googlesource.com/c/99580
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@google.com>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25757}
[modify] https://crrev.com/f259078009238708daa9fbe5027adf1c182ceae2/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/f259078009238708daa9fbe5027adf1c182ceae2/rtc_tools/frame_analyzer/video_quality_analysis.cc

Sign in to add a comment