Showing omnibox suggestions is expected behavior. Having "Home" selected is not and it should not be highlighted. I'll adjust the title of the bug to reflect that.

If the omnibox suggestions become visible, it seems like we should probably switch to the placeholder content right (seen when focusing the omnibox from peeking state)?

Sorry I'm not quite sure I understand. What's the placeholder content?

The blank page that is in place of normal content when focusing the omnibox from the peeking state.

Just synced offline - it sounds like yes, we should swap out the Home content for the suggestions when the suggestions are visible.

Issue 733487 has been merged into this issue.

This is now blocking  issue 735945 : To log some UMA for CTR calculation we rely on the sheet visibility, inferred from whether the home sheet is selected. The current bug makes it trigger when the user simply taps on the omnibox to perform a search or enter a URL.

Can we please have that fixed for M62? Let me know if you want me to take over.

Triage ping. Any updates here?

Triage ping. Should this be moved to M63?

Yes, Fine-Pri-2.0 to 2.9 is M63

@dgn: Which of our metrics are affected by this bug? If we cannot reliably measure our KPIs for the Chrome Home experiment, this would be really bad.

Impressions mostly, and events/user actions related to when the surface is visible.

Quickly chatted with tschumann@ about this issue. For us this is a serious issue which we should get fixed for the stable experiment. 
Rationale: ChromeHome has potentially high impact on how often we have to fetch new content suggestions from our backend. Therefore the experiment is crucial for us to come up with some traffic estimates. Unfortunately, it seems like our metrics are useless as long as this bug exists.

@Kingston: How much effort would it be to fix this? And when are you planning to run your stable experiment? M62?

+Jan

This would also affect the user classification of Zine (which is crucial for traffic and quality control). Can we get a quick-fix merged into M62?
Jan, can you confirm? I'd assume that more users than expected would get moved into the middle-tier bucket (active NTP users). The top tier (active suggestion consumers) should stay unaffected, right?

My gut feeling is that impact on both fetches and user classification is not that huge (say, order of 10%). Apart from that, I agree, we get way clearer insights if this is fixed.

The difficulty of fixing this depends on when the metrics are recorded. Right now, the sheet always tries to load the suggestions content when opened, we intercept this in an observer to replace it with the blank background (the placeholder).

As discussed "offline", will check the selected state of the location bar as workaround, but the place where we want to be is with the omnibox suggestion list being a bottom sheet content. 

We also want to support the use case of bottom sheet contents without toolbar buttons more generally for the new Site Exploration UI and potentially for the site info UI, so this is definitely something worth fixing soon.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/685deb97ac04365e911822a284a8f2424e02f35f

commit 685deb97ac04365e911822a284a8f2424e02f35f
Author: Nicolas Dossou-gbete <dgn@chromium.org>
Date: Thu Sep 07 03:04:16 2017

🏡 Notify Home sheet it's hidden by Omnibox suggestions

Omnibox suggestions hide the home sheet when they are shown, this patch
is a workaround to ensure we don't notify the home sheet that it is
visible in that case, to avoid triggering soft content suggestion
fetches when not needed.

Bug: 760974, 731128 
Change-Id: I81da32ed17dbfd5aaa736aa28b3558363e5d471e
Reviewed-on: https://chromium-review.googlesource.com/651406
Commit-Queue: Nicolas Dossou-Gbété <dgn@chromium.org>
Reviewed-by: Sami Kyöstilä <skyostil@chromium.org>
Reviewed-by: Michael van Ouwerkerk <mvanouwerkerk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#500200}
[modify] https://crrev.com/685deb97ac04365e911822a284a8f2424e02f35f/chrome/android/java/src/org/chromium/chrome/browser/suggestions/SuggestionsSheetVisibilityChangeObserver.java
[modify] https://crrev.com/685deb97ac04365e911822a284a8f2424e02f35f/chrome/android/java_sources.gni
[add] https://crrev.com/685deb97ac04365e911822a284a8f2424e02f35f/chrome/android/javatests/src/org/chromium/chrome/browser/suggestions/SuggestionsSheetVisibilityChangeObserverTest.java
[modify] https://crrev.com/685deb97ac04365e911822a284a8f2424e02f35f/chrome/test/android/javatests/src/org/chromium/chrome/test/BottomSheetTestRule.java

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fbac35ba9363c9cc1315eee835f57eb8819a9b29

commit fbac35ba9363c9cc1315eee835f57eb8819a9b29
Author: Nicolas Dossou-gbete <dgn@chromium.org>
Date: Tue Sep 12 16:20:10 2017

🏡 Notify Home sheet it's hidden by Omnibox suggestions

Omnibox suggestions hide the home sheet when they are shown, this patch
is a workaround to ensure we don't notify the home sheet that it is
visible in that case, to avoid triggering soft content suggestion
fetches when not needed.

Merge required a importing GN change from https://crrev.com/c/649546

TBR=dgn@chromium.org

(cherry picked from commit 685deb97ac04365e911822a284a8f2424e02f35f)

Bug: 760974, 731128 
Change-Id: I81da32ed17dbfd5aaa736aa28b3558363e5d471e
Reviewed-on: https://chromium-review.googlesource.com/651406
Commit-Queue: Nicolas Dossou-Gbété <dgn@chromium.org>
Reviewed-by: Sami Kyöstilä <skyostil@chromium.org>
Reviewed-by: Michael van Ouwerkerk <mvanouwerkerk@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#500200}
Reviewed-on: https://chromium-review.googlesource.com/663720
Reviewed-by: Nicolas Dossou-Gbété <dgn@chromium.org>
Cr-Commit-Position: refs/branch-heads/3202@{#167}
Cr-Branched-From: fa6a5d87adff761bc16afc5498c3f5944c1daa68-refs/heads/master@{#499098}
[modify] https://crrev.com/fbac35ba9363c9cc1315eee835f57eb8819a9b29/chrome/android/java/src/org/chromium/chrome/browser/suggestions/SuggestionsSheetVisibilityChangeObserver.java
[modify] https://crrev.com/fbac35ba9363c9cc1315eee835f57eb8819a9b29/chrome/android/java_sources.gni
[add] https://crrev.com/fbac35ba9363c9cc1315eee835f57eb8819a9b29/chrome/android/javatests/src/org/chromium/chrome/browser/suggestions/SuggestionsSheetVisibilityChangeObserverTest.java
[modify] https://crrev.com/fbac35ba9363c9cc1315eee835f57eb8819a9b29/chrome/test/android/BUILD.gn
[modify] https://crrev.com/fbac35ba9363c9cc1315eee835f57eb8819a9b29/chrome/test/android/javatests/src/org/chromium/chrome/test/BottomSheetTestRule.java

SuggestionsSheetVisibilityChangeObserverTest test failure on the first line of the test trying to click on the URL.

Maybe we should just use our own methods instead of using weird espresso things. I've always found the whole "wait for idle" concept to be cause of loads of flakes...

android.support.test.espresso.PerformException: Error performing 'single click' on view 'with id: org.chromium.chrome:id/url_bar'.
	at android.support.test.espresso.PerformException$Builder.build(PerformException.java:83)
	at android.support.test.espresso.base.DefaultFailureHandler.getUserFriendlyError(DefaultFailureHandler.java:73)
	at android.support.test.espresso.base.DefaultFailureHandler.handle(DefaultFailureHandler.java:53)
	at android.support.test.espresso.ViewInteraction.runSynchronouslyOnUiThread(ViewInteraction.java:184)
	at android.support.test.espresso.ViewInteraction.doPerform(ViewInteraction.java:115)
	at android.support.test.espresso.ViewInteraction.perform(ViewInteraction.java:87)
	at org.chromium.chrome.browser.suggestions.SuggestionsSheetVisibilityChangeObserverTest.testHomeSheetVisibilityOnOmniboxAndSwipeToolbar(SuggestionsSheetVisibilityChangeObserverTest.java:114)
	at java.lang.reflect.Method.invokeNative(Method.java)
	at java.lang.reflect.Method.invoke(Method.java:515)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.chromium.base.test.ScreenshotOnFailureStatement.evaluate(ScreenshotOnFailureStatement.java:37)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.chromium.chrome.test.ChromeActivityTestRule$1.evaluate(ChromeActivityTestRule.java:65)
	at android.support.test.internal.statement.UiThreadStatement.evaluate(UiThreadStatement.java:55)
	at android.support.test.rule.ActivityTestRule$ActivityStatement.evaluate(ActivityTestRule.java:270)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:175)
	at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:40)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.chromium.base.test.BaseJUnit4ClassRunner.run(BaseJUnit4ClassRunner.java:164)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:59)
	at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:262)
	at org.chromium.base.test.BaseChromiumAndroidJUnitRunner.onStart(BaseChromiumAndroidJUnitRunner.java:98)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
Caused by: android.support.test.espresso.AppNotIdleException: Looped for 99 iterations over 60 SECONDS. The following Idle Conditions failed ASYNC_TASKS_HAVE_IDLED.
	at android.support.test.espresso.IdlingPolicy.handleTimeout(IdlingPolicy.java:61)
	at android.support.test.espresso.base.UiControllerImpl.loopUntil(UiControllerImpl.java:477)
	at android.support.test.espresso.base.UiControllerImpl.loopMainThreadUntilIdle(UiControllerImpl.java:362)
	at android.support.test.espresso.base.UiControllerImpl.injectMotionEvent(UiControllerImpl.java:241)
	at android.support.test.espresso.action.MotionEvents.sendUp(MotionEvents.java:138)
	at android.support.test.espresso.action.MotionEvents.sendUp(MotionEvents.java:118)
	at android.support.test.espresso.action.Tap.sendSingleTap(Tap.java:135)
	at android.support.test.espresso.action.Tap.access$100(Tap.java:35)
	at android.support.test.espresso.action.Tap$1.sendTap(Tap.java:40)
	at android.support.test.espresso.action.GeneralClickAction.perform(GeneralClickAction.java:98)
	at android.support.test.espresso.ViewInteraction$1.run(ViewInteraction.java:144)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:422)
	at java.util.concurrent.FutureTask.run(FutureTask.java:237)
	at android.os.Handler.handleCallback(Handler.java:733)
	at android.os.Handler.dispatchMessage(Handler.java:95)
	at android.os.Looper.loop(Looper.java:136)
	at android.app.ActivityThread.main(ActivityThread.java:5001)
	at java.lang.reflect.Method.invokeNative(Method.java)
	at java.lang.reflect.Method.invoke(Method.java:515)
	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:785)
	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:601)
	at dalvik.system.NativeStart.main(NativeStart.java)

Bot link: https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/388220

I think it might be waiting for the blinking cursor or something? The screenshot from the test shows the omnibox has focus..

Hm, in the screenshot there is no blinking cursor though, as the text is selected :)

The error message says that ASYNC_TASKS_HAVE_IDLED is the condition that failed, so it seems there is still an AsyncTask running somewhere... that is of course a lot more likely than the UI not being idle.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/12940865d67dd508968ed1042dc5add3e3d84eff

commit 12940865d67dd508968ed1042dc5add3e3d84eff
Author: Matthew Jones <mdjones@chromium.org>
Date: Wed Sep 20 20:22:28 2017

[Home] Convert omnibox suggestions to be BottomSheetContent

Omnibox suggestions are now a BottomSheetContent. This allows the
suggestions to take advantage of the BottomSheet's frame work
including animation/transitions and scroll handling. Any suggestions
specific code has been removed from the bottom sheet. Special logic
has been added to the BottomSheetController to determine what content
should show after the suggestions are hidden (omnibox unfocused) so
that the home sheet does not flash in during transition.

This change also replaces the content type 'TYPE_PLACEHOLDER' with
'TYPE_AUXILIARY' for non-primary content.

BUG= 764353 ,  731128 

Change-Id: Id9dc714b23cd40103f231cfc3e21b21875210576
Reviewed-on: https://chromium-review.googlesource.com/665166
Commit-Queue: Matthew Jones <mdjones@chromium.org>
Reviewed-by: Nicolas Dossou-Gbété <dgn@chromium.org>
Reviewed-by: Ted Choc <tedchoc@chromium.org>
Reviewed-by: Theresa <twellington@chromium.org>
Cr-Commit-Position: refs/heads/master@{#503232}
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/res/layout/bottom_control_container.xml
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/src/org/chromium/chrome/browser/omnibox/LocationBar.java
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/src/org/chromium/chrome/browser/omnibox/LocationBarLayout.java
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/src/org/chromium/chrome/browser/omnibox/LocationBarPhone.java
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/src/org/chromium/chrome/browser/suggestions/SuggestionsSheetVisibilityChangeObserver.java
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/src/org/chromium/chrome/browser/toolbar/BottomToolbarPhone.java
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/src/org/chromium/chrome/browser/toolbar/CustomTabToolbar.java
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/src/org/chromium/chrome/browser/widget/bottomsheet/BottomSheet.java
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/src/org/chromium/chrome/browser/widget/bottomsheet/BottomSheetContentController.java
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/src/org/chromium/chrome/browser/widget/bottomsheet/BottomSheetMetrics.java
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/java/src/org/chromium/chrome/browser/widget/bottomsheet/PlaceholderSheetContent.java
[modify] https://crrev.com/12940865d67dd508968ed1042dc5add3e3d84eff/chrome/android/javatests/src/org/chromium/chrome/browser/suggestions/SuggestionsSheetVisibilityChangeObserverTest.java