Is this true in a multi-container context or only when we know that we will only set up a container once?

Keeping the container image around would only be useful if you wanted to make multiple containers, all with our Debian image, in succession. Caching these images locally is less valuable over time since you'd want to download a newer image anyway.

Deleting the base container image after setup frees up around 400MB instantly.

I use multiple containers, but I agree with this proposal.

Most people won't use multiple containers, and the image will change over time anyway.