Could you walk through a real world example of behavior showing why the flag exists and a specific bug that your change will fix?

This is what I have been testing:
1) Click Google home app -> account preferences -> Media accounts -> Spotify -> Link account
2) ARC requests a new tab via ChromeShellDelegate to https://accounts.google.com/MergeSession?arg=....
3) New tab is opened in Chrome, it points to https://accounts.spotify.com/en/authorize...
4) Click log-in, this takes us to a login page https://account.apotify.com/en/login?...
5) after login this takes us to https://accounts.spotify.com/en/authorize...

You can either:
6) Hit Okay, this takes you back to Google Home app on ARC side, after redirecting first thru https://clients3.google.com/cast/auth/3p/spotify/...
this redirection gets ignored by AppsNavigationThrottle, this is WAI
7) Then will continue to comgooglecast://chromecast.auth.com/done?... via ArcExternalProtocolDialog, this escape back to Home app on ARC side, provided the flag was preserved.

Hit cancel or just stay in the login screen:
8) Typing a new url or navigating back/forth will delete the flag, so it doesn't matter if the new page contains urls compatible with ArcExternalProtocolDialog, it won't bypass the picker UI because of this flag (testing with pandora and spotify linking pages).

Thanks for the info!

To test whether or not the flag is preserved for cases when you have a direct link to a custom scheme, we can use this command:
adb shell am start -a android.intent.action.VIEW -d "https://paul.kinlan.me/deep-app-linking-on-android-and-chrome/"

I was able to bypass the UI in the loaded tab. 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3136f6737c47c847256ff2832f1e31bf54adb709

commit 3136f6737c47c847256ff2832f1e31bf54adb709
Author: David Jacobo <djacobo@chromium.org>
Date: Sat May 12 03:33:53 2018

Remove ArcWebContentsData via AppsNavigationThrottle

ArcWebContentsData is used to mark tabs that were generated via
ChromeShellDelegate by ARC request. This flag was introduced so we could
offer a smoother authentication process, more specifically so the user
doesn't have to deal with the intent picker UI nor leave tabs opened in
the browser in the process.

However since this flag allows the user to exit Chrome's context we need
to remove the flag when no longer needed, in case the tab itself
survives the authentication process. (e.g. maybe the user didn't
complete the login process, or reused this tab to start a new query via
the omnibox)

In this CL we propose a way to get rid of such flag as soon as the user:
- crosses the same origin boundary (this boils down to changing either
the domain, port or scheme).
- press the forward/back button.
- types a url directly in the omnibox.

TBR=sky@chromium.org

Bug:  838288 
Test: Build
Change-Id: I7ba30d7874d5384d8ce4e3a9317d827198c23acd
Reviewed-on: https://chromium-review.googlesource.com/1034027
Commit-Queue: David Jacobo <djacobo@chromium.org>
Reviewed-by: Yusuke Sato <yusukes@chromium.org>
Reviewed-by: Ben Wells <benwells@chromium.org>
Cr-Commit-Position: refs/heads/master@{#558106}
[modify] https://crrev.com/3136f6737c47c847256ff2832f1e31bf54adb709/chrome/browser/chromeos/BUILD.gn
[modify] https://crrev.com/3136f6737c47c847256ff2832f1e31bf54adb709/chrome/browser/chromeos/apps/intent_helper/apps_navigation_throttle.cc
[add] https://crrev.com/3136f6737c47c847256ff2832f1e31bf54adb709/chrome/browser/chromeos/arc/arc_web_contents_data.cc
[add] https://crrev.com/3136f6737c47c847256ff2832f1e31bf54adb709/chrome/browser/chromeos/arc/arc_web_contents_data.h
[modify] https://crrev.com/3136f6737c47c847256ff2832f1e31bf54adb709/chrome/browser/chromeos/arc/intent_helper/arc_external_protocol_dialog.cc
[modify] https://crrev.com/3136f6737c47c847256ff2832f1e31bf54adb709/chrome/browser/chromeos/arc/intent_helper/arc_external_protocol_dialog.h
[modify] https://crrev.com/3136f6737c47c847256ff2832f1e31bf54adb709/chrome/browser/chromeos/arc/intent_helper/arc_external_protocol_dialog_unittest.cc
[modify] https://crrev.com/3136f6737c47c847256ff2832f1e31bf54adb709/chrome/browser/ui/ash/chrome_shell_delegate.cc

I don't consider this blocking but its a good improvement for user's UX, it only affects Chrome OS users. 

Tested in Cros 10675.0.0

+ kbleicher@ (Chrome OS M67 Release TPM) for merge review

This bug requires manual review: M67 has already been promoted to the beta branch, so this requires manual review
Please contact the milestone owner if you have questions.
Owners: cmasso@(Android), cmasso@(iOS), kbleicher@(ChromeOS), govind@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

This is a large change to merge this late into M67.  I'm rejecting per #6 (not blocking and a UX issue).   Also not clear this was introduced in M67.

I synced with my TL and yeah I think we can live without these on M67 :) thanks!