I'll take a look.

I think there is no much improvement Umpire can do.
We just need to make Dome directly extract the bundle to somewhere Umpire can access, like /cros_docker/.

I was wondering if there are more nginx integration work can be done, for example asking nginx to directly process the input stream as tar archive and extract somewhere, then let Dome handle.

There's also another approach, to support "import from local", for example asking people to upload and extract at somewhere in /cros_docker/bundles and ask Dome/Umpire to read from there.

I think pihsun has already spent a lot of effort on this?

Dome part is mostly done (tracked in b/65352669 and b/31417203), but there are possible improvements that can be done on Umpire side.

@pihsun can you explain what in umpire can be done, and will this be a good first-bug for nooglers?

Since we utilize cros_payload.sh in umpire now, and we need to speed up cros_payload.sh to speed up umpire.

There are three stages for the bundle processing time for dome + umpire.
1. Upload compressed bundle to server (takes about 8 seconds now).
2. Decompress the bundle
3. Add each component using cros_payload.sh

2. and 3. combined currently takes about 75 seconds.

Some possible improvements may be like streaming between steps, but that would require a major architecture change.

It's still probably worthwhile to check what cause cros_payload to take that long, and see if there's any speed up that can be done.

Are you talking about a full bundle or image, and which compressor was used in that bundle?

I did try to support streaming when creating cros_payload, but that was too difficult because we usually need to mount file system, which can't be easily done without using a temporary decompressed blob.

BTW, if the image was compressed by xz (which is what builders doing today), then we can't speed it up with "pixz -d" in decompression. And it's hard to change builders default environment...

btw, 75 = 
 decompress (bundle) => write to disk from tar => cros_payload to read file => compress by pigz => write to output folder.

on the other side, I wonder if it's possible to enhance cros_payload with python so that 

extract and read tar on the fly -> parse GPT to figure out partition -> stream image file and put contents into right partition blob -> compress and pipe rootfs to disk/ram -> extract necessary files from it.

Well, maybe it is still possible, but would be not easy.