This sounds roughly equivalent to the parent sending a fallback for each surfaceid that it has given to the child since the last surfaceid it saw reported from the child, meaning it's not part of the critical path for the fallback frame anymore?

I'm having trouble understanding your question, the parent referencing the new fallback is off the critical path, yes. Is that your question?

What I want to get out of this is parent surface could optimistically sends SQS for the primary surface, and if we need to fallback during draw time we could modify the SQS to fit the fallback. In a way try to limit the code to make decision about using fallback to one place.

Yea, I think this is a win-win if there are no unforeseen problems, we reduce gutter AND we make parent CompositorFrames smaller.

Like ur suggesting the Display pick a surface id and somehow know how to fit it into the parent's scene which I'm not sure is reasonable. But you're trying to move the parent out of the path from the child to the display, if I understand correctly?

To me it sounds like what you're hoping for here would be the parent sending more than one fallback, so the display can use the best one instead of only a single one, because then the parent can specify how to show each fallback as it knows things.

Well, the primary surface in the parent is what the parent WANTS, but if it's not available at aggregation time, then you can grab the latest available frame from the child (since the child is behind, presumably). You grab the latest surface from the child and clip/gutter it relatively to the desired primary surface from the parent.

Ah the question is who knows about the fallback surfaces' ID? Like if parent doesn't send the fallback surface ID, how does display know which surface to use for fallback?

It looks at the latest surface in the child's FrameSink?

Cool.

So re comment#5, it's not removing parent from child to display all the time, but only when we hit deadline and force to draw. So right now parent does work for primary and fallback, and this is remove parent from fallback path.

For fallback, parent will have a surface it expects (size+dsf) and lastest surface from child will try to fit in with its own size+dsf. The resource managing is on a different path.

How does it know how to gutter or scale or whatever it? That's part of the fallback information. That's what I'm trying to get at.

#10: It's a function of the primary surface. The primary surface's SurfaceInfo is what the parent wants in terms of size, and device scale factor and so on. The gutter size is the diff between primary and fallback.

I spent a bunch of time experimenting with this the last couple of days, and it's tricky (there are subtle timing issues) but we do occasionally fall 1-2 SURFACES (not frames but sizes) behind due to propagation delay of fallback surface IDs at surface aggregation time.

In other words, the display compositor knows about fresher CompositorFrames from children but it's not using them when producing a display frame because 

I think this approach can reduce guttering. I'll poke at it a bit more.

TBH it doesn't matter much which fallback surface we use if we need to fallback. I don't think the user cares if the gutter is 10 pixels instead of 50 - I don't think it's a metric we should spend time optimizing.

This also reduces peak memory potentially, and reduces work during resize and other synchronization operations, I think.

Right now we rely on the parent to tell us what fallback to use, which means we need to inform the parent when a new surface ID becomes available in the display compositor which necessitates a bunch of IPCs during resize.

Instead if the parent just specified the primary and not the fallback, then we don't need to inform the parent of a new fallback. Furthermore, Viz can start garbage collecting earlier fallbacks sooner.

WDYT? Maybe not a high priority, but it seems like we shouldn't dismiss this idea entirely.

You still need to specify how to fallback: e.g. blank (for tab switch) vs previous content (for resize), and how to adapt the fallback content to the new size (e.g. guttering vs scaling vs letterboxing).
I suppose you can define a number of modes and parametrization (e.g fill color). Embedder has very little control (thinking of the needs beyond tabs: canvas, video, maybe pepper at some point). Keep in mind that you certainly don't want to use arbitrary old content (thinking security issues around navigation).

piman@: Yup, agreed, I think the fallback policy would live on the SurfaceDrawQuad.

I've had this implemented in prototype form and recent refactorings made it really easy to implement upstream so I just did (and hey, things look better too and perform better, because I can close more recent surfaces too!).

https://chromium-review.googlesource.com/c/chromium/src/+/782341
https://chromium-review.googlesource.com/c/chromium/src/+/820162

Marking as Fixed.