New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 783989 link

Starred by 0 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 720131
issue 784633



Sign in to add a comment

Consolidate tab stats trackers and divorce tracking from logging

Project Member Reported by michae...@chromium.org, Nov 10 2017

Issue description

We have a large number of classes that track tab state and tab interactions.
Together these report a large number of metrics via UMA and UKM.

So we have multiple classes keeping track of tab stats, each recording different
stats in different ways. There's a resource_coordinator::TabStats struct that
holds per-tab stats, and a metrics::TabsStats struct that holds global
cumulative tab counts. Different tab metrics are uploaded from classes in:
  * chrome/browser/metrics
  * chrome/browser/resource_coordinator
  * chrome/browser/ui
  * chrome/browser/ui/tabs

These trackers are all useful, but there are too many of them to keep track of.
I'm guessing some code for gathering these metrics is redundant, and that some
metrics have become forgotten. It's difficult to figure out where new metrics
should go, or which tracker class to add new tab state info to.

The motivation for this bug is that we want to log considerably more tab metrics
than we currently do, so a sustainable and maintainable model is critical. I'm
happy to take point on this and ensure our existing code is integrated in a way
that we can all agree on.

As part of this, I'll create a list of metrics we currently record to ensure
they are all still relevant and useful.

I'm NOT intending to change or remove every class mentioned below, just listing
them here. We'll consolidate the ones that make sense to be together.



Generally:

* UI and Views code should only be recording tab metrics relating to UI
  affordances, e.g. how often a new tab is created via the context menu or
  tabstrip button. This should happen naturally when the UI is invoked.

* Changes modeled at the UI layer should be reported when the model changes,
  like how BrowserList reports the "ActiveBrowserChanged" action in
  BrowserList::SetLastActive, or how chrome::RestoreTab() reports the
  "RestoreTab" action.

* Trackers in chrome/browser/metrics and chrome/browser/resource_coordinator
  should be merged into a single tab tracker, instead of using purpose-built
  trackers for memory management and metrics; this should be exposed to
  chrome/browser/metrics and chrome/browser/resource_coordinator.

* The tab tracker should be separate from the data store implementation and the
  metrics uploader. We should be able to easily change how metrics are reported
  or how tab stats are persisted without modifying the code that tracks tab
  activity.



Summary of existing classes that I've found so far:

resource_coordinator::TabManager [1]
  * Monitors memory pressure, discards tabs, delays background tab navigations
  * Observes TabStripModel using a BrowserTabStripTracker
  * Observes tab discards via TabLifetimeObserver

resource_coordinator::TabManagerStatsCollector: "records UMAs during session
restore or background tab opening session" [2]
  * Logs histograms and UKMs during session restore and background tab opening
  * Observes SessionRestore
  * Owned by resource_coordinator::TabManager
  * Uses resource_coordinator::TabStats to store per-tab stats

metrics::TabReactivationTracker: "track deactivation/reactivation cycle" [3]
  * Provides a delegate for tab reactivation/deactivation
  * Observes TabStripModel using a BrowserTabStripTracker
  * Used and owned by metrics::TabUsageRecorder
  * Subclass WebContentsHelper stores closing and deactivation status

metrics::TabUsageRecorder: "record metrics about tab reactivation" [4]
  * Logs histograms for tab state (pinned, important) on tab
    deactivation/reactivation events
  * Observes TabStripModel using a BrowserTabStripTracker
  * Observes tab de/reactivation using a metrics::TabReactivationTracker
  * Global instance created by ChromeBrowserMainExtraPartsMetrics after startup,
    never deleted
  * Subclass WebContentsData stores pinned state and inactivity time

metrics::TabStatsTracker: "tracking and recording the tabs and browser windows
usage" [5]
  * Logs daily cumulative tab/windows counts
  * Global instance created by ChromeBrowserMainExtraPartsMetrics after startup,
    never deleted
  * Observes TabStripModel of each Browser
  * Observes BrowserList using a chrome::BrowserListObserver
  * Subclass UmaStatsReportingDelegate reports cumulative tab/windows counts

metrics::TabStatsDataStore: "keeps track of all the information that the
TabStatsTracker wants to track" [6]
  * Owns and exposes a metrics::TabStats to store global cumulative tab counts
  * Updates metrics::TabStats info on demand
  * Persists metrics::TabStats stats in the PrefStore for daily uploading

TabStripModelStatsRecorder: "records user tab interaction stats" [7]
  * Logs UMA histograms when tabs are activated/deactivated/closed
  * Owned by UMABrowsingActivityObserver
  * Observes TabStripModel using a BrowserTabStripTracker
  * Subclass TabInfo stores active/inactive state and creation time
  * Supposed to match the Android implementation in TabUma.java.

UMABrowsingActivityObserver: "watches for loads and creates histograms of some
global object counts" [8]
  * Logs global counts of RenderProcessHosts and per-browser tab counts upon
    every page load
  * Global instance created by ChromeBrowserMainParts at startup, deleted at
    shutdown
  * Observes navigation events and Chrome shutdown



Please let me know if I missed any...

[1] chrome/browser/resource_coordinator/tab_manager.h
[2] chrome/browser/resource_coordinator/tab_manager_stats_collector.h
[3] chrome/browser/metrics/tab_reactivation_tracker.h
[4] chrome/browser/metrics/tab_usage_recorder.h
[5] chrome/browser/metrics/tab_stats_tracker.h
[6] chrome/browser/metrics/tab_stats_data_store.h
[7] chrome/browser/ui/tabs/tab_strip_model_stats_recorder.h
[8] chrome/browser/ui/uma_browsing_activity_observer.h
 
Blocking: 720131
adding relevant issue as blocking, though this isn't strictly "blocking" that work (feel free to keep adding metrics to existing places)
Cc: fdoray@chromium.org
+fdoray

I've just found the design doc for issue 775644 which includes consolidating
existing tab lifecycle management code. I don't want to supersede existing plans
for tab stats collection, just want to be aware of what exists and what to plan
for. And figure out whether the best place for Browser- and Tab-specific stat
tracking is in chrome/browser/{resource_coordinator,metrics,} or a new directory
for that purpose.

I've dropped a quick meeting on some folks' calendar to help me get up to speed
and see what's possible.
Blocking: 784633
Project Member

Comment 4 by bugdroid1@chromium.org, Jan 11 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/dde643a557b406bb291ce03f5a2e08bca341aabd

commit dde643a557b406bb291ce03f5a2e08bca341aabd
Author: Michael Giuffrida <michaelpg@chromium.org>
Date: Thu Jan 11 21:52:02 2018

Move TabActivityWatcher into resource_coordinator

This CL just moves TabActivityWatcher wholesale into
chrome/browser/resource_coordinator.

TabActivityWatcher will become the central owner of tab state
monitoring, a specialization of lifecycle unit monitoring. It will feed
directly into TabManager, replacing TabManager's tab tracking behavior.
(Possibly in parallel, TabManager will itself become
LifecycleUnitManager.)

TabActivityWatcher currently calls into TabMetricsLogger; this will
later use an observer pattern.

Bug: 783989
Change-Id: I2d0d3188320d872c61a397f2d79f280cf33a35e7
Reviewed-on: https://chromium-review.googlesource.com/837515
Commit-Queue: Michael Giuffrida <michaelpg@chromium.org>
Reviewed-by: Scott Violet <sky@chromium.org>
Reviewed-by: Chris Hamilton <chrisha@chromium.org>
Cr-Commit-Position: refs/heads/master@{#528766}
[modify] https://crrev.com/dde643a557b406bb291ce03f5a2e08bca341aabd/chrome/browser/BUILD.gn
[modify] https://crrev.com/dde643a557b406bb291ce03f5a2e08bca341aabd/chrome/browser/chrome_browser_main.cc
[rename] https://crrev.com/dde643a557b406bb291ce03f5a2e08bca341aabd/chrome/browser/resource_coordinator/tab_activity_watcher.cc
[rename] https://crrev.com/dde643a557b406bb291ce03f5a2e08bca341aabd/chrome/browser/resource_coordinator/tab_activity_watcher.h
[rename] https://crrev.com/dde643a557b406bb291ce03f5a2e08bca341aabd/chrome/browser/resource_coordinator/tab_activity_watcher_unittest.cc
[modify] https://crrev.com/dde643a557b406bb291ce03f5a2e08bca341aabd/chrome/browser/resource_coordinator/tab_manager_web_contents_data.h
[modify] https://crrev.com/dde643a557b406bb291ce03f5a2e08bca341aabd/chrome/browser/ui/BUILD.gn
[modify] https://crrev.com/dde643a557b406bb291ce03f5a2e08bca341aabd/chrome/browser/ui/tab_helpers.cc
[modify] https://crrev.com/dde643a557b406bb291ce03f5a2e08bca341aabd/chrome/test/BUILD.gn

Project Member

Comment 5 by bugdroid1@chromium.org, Jan 30 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed

commit ff7fac5e212adb0b1f6914c80f7640d2e2a252ed
Author: Michael Giuffrida <michaelpg@chromium.org>
Date: Tue Jan 30 05:01:25 2018

Move TabMetricsLogger to c/b/resource_coordinator

With browser/tab activity being consolidated in resource_coordinator,
TabMetricsLogger belongs here. This CL just moves files without changing
functionality.

TabMetricsLogger will later take over responsibility for logging other
logging other tab UKMs from tab_manager_web_contents_data.cc. It will
also be generalized to handle other "tab" types like Chrome apps.

Bug: 783989
Change-Id: I55425b027a4860f52eb49b9169609954fe2f76a9
Reviewed-on: https://chromium-review.googlesource.com/890861
Commit-Queue: Michael Giuffrida <michaelpg@chromium.org>
Reviewed-by: François Doray <fdoray@chromium.org>
Reviewed-by: Scott Violet <sky@chromium.org>
Cr-Commit-Position: refs/heads/master@{#532763}
[modify] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/BUILD.gn
[rename] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/resource_coordinator/BUILD.gn
[modify] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/resource_coordinator/tab_activity_watcher.cc
[modify] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/resource_coordinator/tab_activity_watcher_unittest.cc
[rename] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/resource_coordinator/tab_metrics_event.proto
[rename] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/resource_coordinator/tab_metrics_logger.h
[rename] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/resource_coordinator/tab_metrics_logger_impl.cc
[rename] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/resource_coordinator/tab_metrics_logger_impl.h
[rename] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/resource_coordinator/tab_metrics_logger_impl_unittest.cc
[modify] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/ui/BUILD.gn
[modify] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/ui/tabs/window_activity_watcher.cc
[modify] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/browser/ui/tabs/window_activity_watcher_unittest.cc
[modify] https://crrev.com/ff7fac5e212adb0b1f6914c80f7640d2e2a252ed/chrome/test/BUILD.gn

Project Member

Comment 6 by bugdroid1@chromium.org, Feb 28 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/97d530585734b72cf709485a286b4d7ebe0595a0

commit 97d530585734b72cf709485a286b4d7ebe0595a0
Author: Michael Giuffrida <michaelpg@chromium.org>
Date: Wed Feb 28 04:17:34 2018

Move TabManager UKMs into TabActivityWatcher

This consolidates logging for TabManager.Background.ForegroundedOrClosed
and TabManager.TabLifetime into TabActivityWatcher, which already
handles TabManager.TabMetrics logging.

This corrects several cases where ForegroundedOrClosed is logged
inappropriately and adds some tests for these UKMs.

Bug: 791362,783989
Change-Id: I1f1b15a300df5d1c7cdf54267844f8cfbd31c7e3
Reviewed-on: https://chromium-review.googlesource.com/929009
Commit-Queue: Michael Giuffrida <michaelpg@chromium.org>
Reviewed-by: Scott Violet <sky@chromium.org>
Reviewed-by: Chris Hamilton <chrisha@chromium.org>
Cr-Commit-Position: refs/heads/master@{#539706}
[modify] https://crrev.com/97d530585734b72cf709485a286b4d7ebe0595a0/chrome/browser/resource_coordinator/tab_activity_watcher.cc
[modify] https://crrev.com/97d530585734b72cf709485a286b4d7ebe0595a0/chrome/browser/resource_coordinator/tab_activity_watcher.h
[modify] https://crrev.com/97d530585734b72cf709485a286b4d7ebe0595a0/chrome/browser/resource_coordinator/tab_activity_watcher_browsertest.cc
[modify] https://crrev.com/97d530585734b72cf709485a286b4d7ebe0595a0/chrome/browser/resource_coordinator/tab_activity_watcher_unittest.cc
[modify] https://crrev.com/97d530585734b72cf709485a286b4d7ebe0595a0/chrome/browser/resource_coordinator/tab_manager_web_contents_data.cc
[modify] https://crrev.com/97d530585734b72cf709485a286b4d7ebe0595a0/chrome/browser/resource_coordinator/tab_manager_web_contents_data.h
[modify] https://crrev.com/97d530585734b72cf709485a286b4d7ebe0595a0/chrome/browser/resource_coordinator/tab_metrics_logger.cc
[modify] https://crrev.com/97d530585734b72cf709485a286b4d7ebe0595a0/chrome/browser/resource_coordinator/tab_metrics_logger.h
[modify] https://crrev.com/97d530585734b72cf709485a286b4d7ebe0595a0/chrome/browser/ui/tabs/tab_activity_simulator.cc
[modify] https://crrev.com/97d530585734b72cf709485a286b4d7ebe0595a0/chrome/browser/ui/tabs/tab_activity_simulator.h

Project Member

Comment 7 by bugdroid1@chromium.org, Apr 30 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f02f23449a6ba0e21bfc9802770b387b8ed79114

commit f02f23449a6ba0e21bfc9802770b387b8ed79114
Author: Michael Giuffrida <michaelpg@chromium.org>
Date: Mon Apr 30 04:37:14 2018

Separate TabMetrics data from logging

TabMetricsLogger logs the TabMetrics UKM event for a background tab.
This event includes the state of the tab, as well as some slightly
tricky or nullable fields.

Add a new TabFeatures struct that contains all this information, and
generate the struct in a separate function. The UKM logging function
now simply forwards info from the TabFeatures struct to the UKM event.

Later we will use the same TabFeatures to calculate tab scores with an
ML model. Encapsulating the logic for these features in a single
function ensures that we use the same logic for logging as we will for
inference.

Bug: 784639, 783989
Change-Id: I11e64119ce30465f98409f54f7e97af0251ce4b3
Reviewed-on: https://chromium-review.googlesource.com/1028562
Commit-Queue: Michael Giuffrida <michaelpg@chromium.org>
Reviewed-by: François Doray <fdoray@chromium.org>
Cr-Commit-Position: refs/heads/master@{#554694}
[modify] https://crrev.com/f02f23449a6ba0e21bfc9802770b387b8ed79114/chrome/browser/BUILD.gn
[add] https://crrev.com/f02f23449a6ba0e21bfc9802770b387b8ed79114/chrome/browser/resource_coordinator/tab_features.cc
[add] https://crrev.com/f02f23449a6ba0e21bfc9802770b387b8ed79114/chrome/browser/resource_coordinator/tab_features.h
[modify] https://crrev.com/f02f23449a6ba0e21bfc9802770b387b8ed79114/chrome/browser/resource_coordinator/tab_metrics_logger.cc
[modify] https://crrev.com/f02f23449a6ba0e21bfc9802770b387b8ed79114/chrome/browser/resource_coordinator/tab_metrics_logger.h
[modify] https://crrev.com/f02f23449a6ba0e21bfc9802770b387b8ed79114/chrome/browser/resource_coordinator/tab_metrics_logger_unittest.cc

Labels: Hotlist-DesktopUIValid Hotlist-DesktopUIChecked
**UI Mass Triage**

@michaelpg: could you please help in verifying the issue?
Owner: fdoray@chromium.org
Status: Assigned (was: Started)
I think our tab tracking is more consolidated and we have more purpose-built logging now. We could do more refactoring, but it's probably not worth the time and risk tradeoff.

fdoray: since I'm no longer actively working in this area, I'll leave it up to you whether you want to keep this bug open.

Sign in to add a comment