Right now, the image decoders link the concepts of zeroing pixel data and setting the alpha.

Zeroing the pixel data should only effect values.
Setting the alpha on or off should only effect behavior.

For example, if I have an opaque image and I set the pixel values to 0 my image should appear black. But we currently also enable the alpha in this case which will make my image appear transparent. That forces me to reset the alpha back to disabled after zeroing the pixel data.