Version: M55-Dev, build 8872.6.0, chrome 55.0.2883.7
Device:  Rikku (chromebox)

What steps will reproduce the problem?
I haven't been able to reproduce this yet, and it has only happened once.

The steps I took:
(1) Fresh device, OOBE, Consumer mode. Enabled Kiosk mode (<ctrl>-<alt>-k)

(2) Logged in as owner and added 4 apps to the Kiosk list (Chrome Sign, Stratos Media Player, AIRSecure Test and an internal testing app (Multi-App Kiosk Test: id=bkledbfligfdnfkmccllbllealecompm).

(3) Set Chrome Sign to be auto-launched, then immediately deleted Chrome Sign from the list.  Next, set Multi-App Kiosk Test as the auto-launched app, Clicked [Done] and signed out.  Confirmed Multi-App to be auto-launched, and...

What is the expected output?
I expected to see Multi-App's icon (a lime green square) and title, with the "Initializing..." statement below it.

What do you see instead?
The icon was Chrome Sign's rainbow donut!  The title was correct, though, and the executing app was indeed Multi-App.

I went to VT2 (<ctrl>-<alt>-f2), logged in as root:
# cd /home/chronos/kiosk/crx
# ls
and it showed 4(!) apps, not the three I expected.  Chrome Sign's crx file was still there...

Rebooted.  Chrome Sign's crx was gone and the Multi-App icon showed up correctly each time it ran.  It looked like the deletion of Chrome Sign from the device was delayed to some point after clicking [Done] in the Manage Kiosk dialog box.