This is interesting to think about in the context of Chrome Home and whether we'll change our updating behavior in M58.

Principle:
We fetch whenever we find appropriate, but we should only update the UI if it's not visible yet.

Key question:
Is Chrome Home surface constructed before it's pulled up?

Outcome: 
Eng would like to experiment in Beta for M57, but we need a plan for avoiding a jarring UI. We can discuss a path forward in the next UX weekly. Once we move to Chrome Home, this may no longer be an issue (depending on when the surface is constructed).

Jan and Michael, let's just move ahead and get the plumbing done. This is meant to be an "eng exploration". Let's try to figure out a way how we can get this working. Then we can try it our internally and get people's opinions. 

Second step is then to bring it into a stage where we could even run a public experiment. For that we then need your input, Rachel :-) Happy to start the discussion well in advance. 

I had an offline discussion with Rachel today:

Updating the snippets in the UI in M57 Beta:
 - Updating the snippets with a fade-out -> fade-in animation (as currently implemented for the no-snippets -> show-some snippets animation) sounds good to Rachel
 - One important thing is to avoid this update when there is a peeking card animation (so that the peeking card it is not a moving and changing mess). This is not an issue for the experiments as the peeking car animation shows only a couple of times.

Updating the snippets in Chrome Home:
 - Not a UX issue no matter whether the UI elements are constructed before sliding Home up or while sliding it up. In both cases we see the new snippets.

Implementation pointers (thanks to dgn@):

IsCurrentPage: https://cs.chromium.org/chromium/src/chrome/android/java/src/org/chromium/chrome/browser/ntp/NewTabPage.java?q=isCurrentPage+f:ntp&sq=package:chromium&l=675&dr=CSs

IsPeeking: (means that the user is above the fold) https://cs.chromium.org/chromium/src/chrome/android/java/src/org/chromium/chrome/browser/ntp/cards/CardViewHolder.java?dr=CSs&q=isPeeking+f:ntp&sq=package:chromium&l=213

Not directly available from the NTPAdapter though. something like mRecyclerView.findViewHolderForAdapterPosition(getFirstCardPosition).isPeeking() should do.

Suggestion notifications from C++: https://cs.chromium.org/chromium/src/chrome/android/java/src/org/chromium/chrome/browser/ntp/cards/SectionList.java?q=f:ntp+onNewSuggestions&sq=package:chromium&l=121&dr=CSs

that comes from calls to ntp_snippets::ContentSuggestionsService::Observer (the snippets bridge) or through jni directly as SuggestionsSource.Observer

A video recording of my current implementation.

Deleted: screen-recording-84B7N15A28010647-20170104T140221.mp4 3.3 MB

	screen-recording-84B7N15A28010647-20170104T140221.mp4 3.3 MB View Download

The important thing happens in 5th second of the video, notice the flickr of the peeking card.

Michael, any ideas how to make it smoother?

Isn't that flicker just a quick rendering of the default animation? You could try to temporarily disable animation but that feels rough. Maybe Nicolas has an idea for this?

Hey guys,
What's the timeline for this piece? It looks like Jan won't be joining the meeting today, but this also looks like a P1. Is it a concern if this waits until Friday (our next meeting)?

I am sorry Rachel for not updating the bug. Actually the current implementation I work on never updates any visible snippet (such as the peeking snippet). So there is no visible animation whatsoever.

When you see the peeking card for articles, only articles 2-10 get updated. Do you see any issue from UX point of view?

Jan and I discussed in person and agreed that the current approach makes sense.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d14a88c13b200244200ad1fe5298f01a9cc3e52f

commit d14a88c13b200244200ad1fe5298f01a9cc3e52f
Author: jkrcal <jkrcal@chromium.org>
Date: Mon Jan 16 16:59:42 2017

[NTP Suggestions] Updating the suggestions before going below the fold.

Currently the NTP UI updates suggestions only when the current list of
suggestions for the given category is empty. After this CL, suggestions
are also updated for non-empty category. The pre-conditions are:
 - the user has not yet scrolled below the fold on the NTP,
 - there is no peeking-card animation on the NTP.

BUG= 674163 

Review-Url: https://codereview.chromium.org/2607333002
Cr-Commit-Position: refs/heads/master@{#443908}

[modify] https://crrev.com/d14a88c13b200244200ad1fe5298f01a9cc3e52f/chrome/android/java/src/org/chromium/chrome/browser/ntp/cards/CardsVariationParameters.java
[modify] https://crrev.com/d14a88c13b200244200ad1fe5298f01a9cc3e52f/chrome/android/java/src/org/chromium/chrome/browser/ntp/cards/NewTabPageRecyclerView.java
[modify] https://crrev.com/d14a88c13b200244200ad1fe5298f01a9cc3e52f/chrome/android/java/src/org/chromium/chrome/browser/ntp/cards/SectionList.java
[modify] https://crrev.com/d14a88c13b200244200ad1fe5298f01a9cc3e52f/chrome/android/java/src/org/chromium/chrome/browser/ntp/cards/SuggestionsSection.java
[modify] https://crrev.com/d14a88c13b200244200ad1fe5298f01a9cc3e52f/chrome/android/java/src/org/chromium/chrome/browser/ntp/snippets/FakeSuggestionsSource.java
[modify] https://crrev.com/d14a88c13b200244200ad1fe5298f01a9cc3e52f/chrome/android/junit/src/org/chromium/chrome/browser/ntp/cards/NewTabPageAdapterTest.java
[modify] https://crrev.com/d14a88c13b200244200ad1fe5298f01a9cc3e52f/chrome/android/junit/src/org/chromium/chrome/browser/ntp/cards/SuggestionsSectionTest.java

Currently (after the soon-to-land CL), the UI makes sure that after the update there are as many suggestions as in the newly provided list.

Example: 
Assume there 
 - are only 5 suggestions (in a section that can have at most 10 suggestions),
 - 3 of them are seen, and 
 - the new list contains 6 suggestions (2 of them duplicates of the seen ones).
The 3 seen suggestions are kept, we remove the 2 duplicates from the new suggestions and finally remove the last suggestion from the new list so that it has 3 suggestions. We append these 3 new suggestions to the 3 seen ones to have exactly 6 suggestions after the update.

dgn@ raised a question whether we should rather extend CategoryInfo by the max number of suggestions in that category and do not remove excess suggestions from the new list unless we exceed the max number.

Example cont.:
In this behaviour, we would only filter out 2 duplicates and append the 4 new suggestions to the 3 seen ones and have 7 suggestions after the update.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/29f7cd135146bf1ac2e4020d04014f69fce261ac

commit 29f7cd135146bf1ac2e4020d04014f69fce261ac
Author: jkrcal <jkrcal@chromium.org>
Date: Thu Jan 19 23:10:27 2017

[NTP suggestions UI] Track precise count of suggestions seen.

Sometimes, more than the first suggestion is marked as seen before the
user scrolls below the fold.

This CL makes the updates more fine-grained and thus more robust. The
section will keep track of the total count of suggestions shown to the
user and update only the remaining ones.

BUG= 674163 

Review-Url: https://codereview.chromium.org/2639933003
Cr-Commit-Position: refs/heads/master@{#444879}

[modify] https://crrev.com/29f7cd135146bf1ac2e4020d04014f69fce261ac/chrome/android/java/src/org/chromium/chrome/browser/ntp/cards/SuggestionsSection.java
[modify] https://crrev.com/29f7cd135146bf1ac2e4020d04014f69fce261ac/chrome/android/junit/src/org/chromium/chrome/browser/ntp/cards/ContentSuggestionsTestUtils.java
[modify] https://crrev.com/29f7cd135146bf1ac2e4020d04014f69fce261ac/chrome/android/junit/src/org/chromium/chrome/browser/ntp/cards/NewTabPageAdapterTest.java
[modify] https://crrev.com/29f7cd135146bf1ac2e4020d04014f69fce261ac/chrome/android/junit/src/org/chromium/chrome/browser/ntp/cards/SectionListTest.java
[modify] https://crrev.com/29f7cd135146bf1ac2e4020d04014f69fce261ac/chrome/android/junit/src/org/chromium/chrome/browser/ntp/cards/SuggestionsSectionTest.java

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ede80b706285aa0052cf154d89c49f081a74b9a5

commit ede80b706285aa0052cf154d89c49f081a74b9a5
Author: jkrcal <jkrcal@chromium.org>
Date: Fri Jan 20 00:49:15 2017

[Content suggestions] Report updates in the UI to UMA.

Content suggestions on the Android New Tab Page can get updated if the
user has not seen them yet.

This CL adds a histogram to reason about how often are these updates
successful.

BUG= 674163 

Review-Url: https://codereview.chromium.org/2639533003
Cr-Commit-Position: refs/heads/master@{#444896}

[modify] https://crrev.com/ede80b706285aa0052cf154d89c49f081a74b9a5/chrome/android/java/src/org/chromium/chrome/browser/ntp/NewTabPageUma.java
[modify] https://crrev.com/ede80b706285aa0052cf154d89c49f081a74b9a5/chrome/android/java/src/org/chromium/chrome/browser/ntp/cards/SuggestionsSection.java
[modify] https://crrev.com/ede80b706285aa0052cf154d89c49f081a74b9a5/chrome/android/junit/src/org/chromium/chrome/browser/ntp/cards/SectionListTest.java
[modify] https://crrev.com/ede80b706285aa0052cf154d89c49f081a74b9a5/tools/metrics/histograms/histograms.xml

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/858ceafc7cf4f11a6549b8c1ace839a45d943d68

commit 858ceafc7cf4f11a6549b8c1ace839a45d943d68
Author: jkrcal <jkrcal@chromium.org>
Date: Sat Jan 21 11:25:58 2017

[Content suggestions] Report updates in the UI to UMA - cleanup

A cleanup of after-lgtm changes from CL 2639533003. This CL separates
the enum histogram from a counter. They were previously mixed together
in one CL.

BUG= 674163 

Review-Url: https://codereview.chromium.org/2643293003
Cr-Commit-Position: refs/heads/master@{#445287}

[modify] https://crrev.com/858ceafc7cf4f11a6549b8c1ace839a45d943d68/chrome/android/java/src/org/chromium/chrome/browser/ntp/NewTabPageUma.java
[modify] https://crrev.com/858ceafc7cf4f11a6549b8c1ace839a45d943d68/chrome/android/java/src/org/chromium/chrome/browser/ntp/cards/SuggestionsSection.java
[modify] https://crrev.com/858ceafc7cf4f11a6549b8c1ace839a45d943d68/tools/metrics/histograms/histograms.xml