Actually the problem is deeper, because our metrics have the same data source as the user classifier.

On Nexus 6P with condensed layout, each NTP triggers an impression of the first suggestion. Together with Personalised Section Ranking, this may pollute our metrics.

Maybe we can start recording only after the user interacts with the section? So wait for a scroll or tap?

Vitalii, I think this works as expected. If the suggestions is directly visible on your NTP, it is impressed.

Sure, this will skew our data so we need to be careful in interpreting the condensed NTP experiment.

The problem with this bug (for UserClassifier) is that I interpreted "impression" as a signal that the user interacts with Zine which does not hold anymore.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f215f20a3eba699ac39d8bc5c0e4cf8e0d9b0610

commit f215f20a3eba699ac39d8bc5c0e4cf8e0d9b0610
Author: jkrcal <jkrcal@chromium.org>
Date: Thu Mar 09 14:00:12 2017

[UserClassifier] Fix the classification for M58 condensed NTP experiment

Previously, UserClassifier used suggestions impressions as activity
signal. This breaks for condensed NTP when impressions are tied to
opening an NTP.

This CL replaces this signal by clicks. Clicks were previously recorded
and not used. After this CL, impressions will stay being recorded. The
reason is that with further changes in the UI, we may need it again. There is a TODO to remove this metric if not used.

This CL also changes default values for activity metrics. The reason is
to make sure that a new user is classified in the middle class.

Lastly, the CL relaxes the thresholds. Currently, we have 67% rare, 33% active users, 1% active consumers. The aim is to move closer towards 55% / 40% / 5%.

BUG= 699021 

Review-Url: https://codereview.chromium.org/2732243002
Cr-Commit-Position: refs/heads/master@{#455734}

[modify] https://crrev.com/f215f20a3eba699ac39d8bc5c0e4cf8e0d9b0610/components/ntp_snippets/user_classifier.cc
[modify] https://crrev.com/f215f20a3eba699ac39d8bc5c0e4cf8e0d9b0610/components/ntp_snippets/user_classifier.h

Vitalii, as regards verifying on Canary, please
 - open some tab in Chrome, 
 - deleting all user data,
 - open Chrome again (without opening any ntp),
 - check that the numbers for the metrics are approx. 24, 48, 96,
 - check that the class is "Active NTP user"

Verified in 59.0.3040.0.

After the reset, the numbers for the metrics are approx. 24, 48, 96 and the class is "Active NTP user".

Requesting to merge the CL from the comment #4 to M58.

Your change meets the bar and is auto-approved for M58. Please go ahead and merge the CL to branch 3029 manually. Please contact milestone owner if you have questions.
Owners: amineer@(clank), cmasso@(bling), bhthompson@(cros), govind@(desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5b1922716b0af3cb32f16d329d0cbc131b25c46a

commit 5b1922716b0af3cb32f16d329d0cbc131b25c46a
Author: Vitalii Iarko <vitaliii@chromium.org>
Date: Tue Mar 14 07:55:05 2017

Merge "[UserClassifier] Fix the classification for M58 condensed NTP..."

This is a merge of https://codereview.chromium.org/2732243002 into M58
(branch 3029).

There were no conflicts.

=== Original description ===

Previously, UserClassifier used suggestions impressions as activity
signal. This breaks for condensed NTP when impressions are tied to
opening an NTP.

This CL replaces this signal by clicks. Clicks were previously recorded
and not used. After this CL, impressions will stay being recorded. The
reason is that with further changes in the UI, we may need it again. There is a TODO to remove this metric if not used.

This CL also changes default values for activity metrics. The reason is
to make sure that a new user is classified in the middle class.

Lastly, the CL relaxes the thresholds. Currently, we have 67% rare, 33% active users, 1% active consumers. The aim is to move closer towards 55% / 40% / 5%.

BUG= 699021 

Review-Url: https://codereview.chromium.org/2732243002
Cr-Commit-Position: refs/heads/master@{#455734}
(cherry picked from commit f215f20a3eba699ac39d8bc5c0e4cf8e0d9b0610)

Review-Url: https://codereview.chromium.org/2748053002 .
Cr-Commit-Position: refs/branch-heads/3029@{#183}
Cr-Branched-From: 939b32ee5ba05c396eef3fd992822fcca9a2e262-refs/heads/master@{#454471}

[modify] https://crrev.com/5b1922716b0af3cb32f16d329d0cbc131b25c46a/components/ntp_snippets/user_classifier.cc
[modify] https://crrev.com/5b1922716b0af3cb32f16d329d0cbc131b25c46a/components/ntp_snippets/user_classifier.h