https://bugs.chromium.org/p/chromium/issues/detail?id=889768#c5

What I can see is that, the current chromium media-gpu-vp9-decode path always checks the frame header for the resolution and if there is a mismatch with already configured one, it will initiate a full cycle of buffer pool re-allocation and context initiation. Please correct me if I'm wrong.

IMHO, this is a sub-optimal solution. How about keeping the surface pool as long as the new resolution is lower than the currently configured one ?

Second case is when the new frame come up with a higher resolution, in this case  hardware decoder requires a full context reset and a surface pool reallocation.
Well the question is, how could it work if the new frame(with higher resolution) has inter prediction from one of the old frame. AFAIK, it is not possible to keep two vaapi contexts at the same time with two different resolution video frames. Anyways I will double check with a few driver developers.

A reasonable compromise would be to utilize the resolution provided by the storage formats mkv/webm. If there is multi resolution frames with in a key-frame period, mkv/webm usually advertises the higher resolution of the frames (Who is the MKV expert to ask??). If this is true, we can avoid hitting the surface reallocation every now and then with in a key-frame period for each resolution change, rather we pre-allocate the surface pool based on the higher resolution parsed from the mkv/web, so that the elementary stream frame-header resolution will be always less than or equal to the configured one with in the key-frame period.

Adding Dale and Chris to this for suggestions.

What you guys think about it?

+Ted for VP9 hw decoder
+James for VP9 / webm trivia

> How about keeping the surface pool as long as the new resolution is lower than the currently configured one ?

This question goes to Ted. 

> Well the question is, how could it work if the new frame(with higher resolution) has inter prediction from one of the old frame. 

I'm not sure that's possible. I expect resolution would only change at an I-frame boundary. James, can you comment? 

> If there is multi resolution frames with in a key-frame period, mkv/webm usually advertises the higher resolution of the frames

Multi-resolution inside a single webm file is pretty rare. I don't think there's a strong contract around whether the video track would describe the largest resolution (vs the first or last etc). Given the rarity, I would hesitate to assume it is otherwise following best-practices for muxing webm (e.g. its maybe old, or spliced together by tools). 

>> Well the question is, how could it work if the new frame(with higher resolution) has inter prediction from one of the old frame. 
>
> I'm not sure that's possible. I expect resolution would only change at an I-frame boundary. James, can you comment? 

Resolution change only occurs on a keyframe or an intra-only frame.

>Resolution change only occurs on a keyframe or an intra-only frame.

res change can happen only on key frame or intra frame, but a new frame with higher resolution can still depends on an old frame which came before intra frame. Which means we can't deallocate old frames when there is an intra only frame. So we should keep two or more resolution type video frames at the same time.

From what I have learned, the requirement to have surfaces bound to the Context is not true any more. This was mandated before, but ideally it should work with out this bound . It is a good news for us to try enabling multi-resolution videos using dynamically created surfaces. But I myself haven't tested this on other media stacks. In gstreamer, we tried to follow this constraint (surface must be bound to context) and used the tricks I mentioned in comment3 for supporting multi-res videos.