New issue
Advanced search Search tips

Issue 874728 link

Starred by 2 users

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature



Sign in to add a comment

Evaluation Metrics for Roselle.

Project Member Reported by charleszhao@chromium.org, Aug 16

Issue description


Add more evaluation metrics for Roselle including:

(1) Clicks@k (how many times are apps launched at each suggested app tiles.)

(2) looking_for_a_app_time. How long a user has spent to find the app he needs.
 
Components: -Blink UI>ML
looking_for_a_app_time can be calculated in AppListClientImpl by:

(1) record start_time in AppListClientImpl::StartSearch(trimmed_query) if the trimmed_query is empty (query for recommendations).

(2) record end_time in AppListClientImpl::ActivateItem or AppListClientImpl::OpenSearchResult() with a suggestedapptile.

One major issue with this approach is that: a user can search a blank query in search box which will also trigger AppListClientImpl::StartSearch and update the start_time. And that is not what we want.
 

Cc: xiy...@chromium.org
Re #2:
|start_time| initialized by searching an empty query is interesting, is this guaranteed when the launcher is opened? This could be broken easily. From what I understand the |start_time| is when the launcher is opened, why not just measure this directly?

|end_time| should also record search results that are not suggested apps being opened. How about splitting these metrics by Search Result/app types? So this way |looking_for_a_app_time| would have the buckets: 
 - Suggested App
 - App in the Apps grid (manually found)
 - App Search Result (installed)
 - App Search Result (uninstalled playstore suggestion, webstore suggestion)
 - Answer Card
 - Generic search result
 - Drive Result
... etc

Project Member

Comment 4 by bugdroid1@chromium.org, Aug 23

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fe1a9a1fe82a05c3526ee218d2a89d769c8b2107

commit fe1a9a1fe82a05c3526ee218d2a89d769c8b2107
Author: Charles Zhao <charleszhao@chromium.org>
Date: Thu Aug 23 02:48:21 2018

cros: Add histogram to record Apps.AppListSuggestedChipLaunched.

This records the number of times a suggested chip at a specific index
is launched.

(1) The index of each suggestion_chip_view is assigned in
    suggestion_chip_container_view when the container updates.
(2) The index is recorded when the ButtonPressed event is triggered for
    a search_result_suggestion_chip_view.

Change-Id: I124a85997f4104b1cf70a93d0e76a40c8c8c9864

Bug: 874728
Change-Id: I124a85997f4104b1cf70a93d0e76a40c8c8c9864
Reviewed-on: https://chromium-review.googlesource.com/1177087
Commit-Queue: Charles . <charleszhao@chromium.org>
Reviewed-by: Ilya Sherman <isherman@chromium.org>
Reviewed-by: Xiyuan Xia <xiyuan@chromium.org>
Reviewed-by: Alex Newcomer <newcomer@chromium.org>
Cr-Commit-Position: refs/heads/master@{#585376}
[modify] https://crrev.com/fe1a9a1fe82a05c3526ee218d2a89d769c8b2107/ash/app_list/views/search_result_suggestion_chip_view.cc
[modify] https://crrev.com/fe1a9a1fe82a05c3526ee218d2a89d769c8b2107/ash/app_list/views/search_result_suggestion_chip_view.h
[modify] https://crrev.com/fe1a9a1fe82a05c3526ee218d2a89d769c8b2107/ash/app_list/views/suggestion_chip_container_view.cc
[modify] https://crrev.com/fe1a9a1fe82a05c3526ee218d2a89d769c8b2107/tools/metrics/histograms/histograms.xml

Labels: -Hotlist-Partner-GSuite
Labels: Pri-2
Setting defect without priority to Pri-2.

Sign in to add a comment