Evaluation Metrics for Roselle. |
||||
Issue descriptionAdd more evaluation metrics for Roselle including: (1) Clicks@k (how many times are apps launched at each suggested app tiles.) (2) looking_for_a_app_time. How long a user has spent to find the app he needs.
,
Aug 16
looking_for_a_app_time can be calculated in AppListClientImpl by: (1) record start_time in AppListClientImpl::StartSearch(trimmed_query) if the trimmed_query is empty (query for recommendations). (2) record end_time in AppListClientImpl::ActivateItem or AppListClientImpl::OpenSearchResult() with a suggestedapptile. One major issue with this approach is that: a user can search a blank query in search box which will also trigger AppListClientImpl::StartSearch and update the start_time. And that is not what we want.
,
Aug 16
Re #2: |start_time| initialized by searching an empty query is interesting, is this guaranteed when the launcher is opened? This could be broken easily. From what I understand the |start_time| is when the launcher is opened, why not just measure this directly? |end_time| should also record search results that are not suggested apps being opened. How about splitting these metrics by Search Result/app types? So this way |looking_for_a_app_time| would have the buckets: - Suggested App - App in the Apps grid (manually found) - App Search Result (installed) - App Search Result (uninstalled playstore suggestion, webstore suggestion) - Answer Card - Generic search result - Drive Result ... etc
,
Aug 23
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fe1a9a1fe82a05c3526ee218d2a89d769c8b2107 commit fe1a9a1fe82a05c3526ee218d2a89d769c8b2107 Author: Charles Zhao <charleszhao@chromium.org> Date: Thu Aug 23 02:48:21 2018 cros: Add histogram to record Apps.AppListSuggestedChipLaunched. This records the number of times a suggested chip at a specific index is launched. (1) The index of each suggestion_chip_view is assigned in suggestion_chip_container_view when the container updates. (2) The index is recorded when the ButtonPressed event is triggered for a search_result_suggestion_chip_view. Change-Id: I124a85997f4104b1cf70a93d0e76a40c8c8c9864 Bug: 874728 Change-Id: I124a85997f4104b1cf70a93d0e76a40c8c8c9864 Reviewed-on: https://chromium-review.googlesource.com/1177087 Commit-Queue: Charles . <charleszhao@chromium.org> Reviewed-by: Ilya Sherman <isherman@chromium.org> Reviewed-by: Xiyuan Xia <xiyuan@chromium.org> Reviewed-by: Alex Newcomer <newcomer@chromium.org> Cr-Commit-Position: refs/heads/master@{#585376} [modify] https://crrev.com/fe1a9a1fe82a05c3526ee218d2a89d769c8b2107/ash/app_list/views/search_result_suggestion_chip_view.cc [modify] https://crrev.com/fe1a9a1fe82a05c3526ee218d2a89d769c8b2107/ash/app_list/views/search_result_suggestion_chip_view.h [modify] https://crrev.com/fe1a9a1fe82a05c3526ee218d2a89d769c8b2107/ash/app_list/views/suggestion_chip_container_view.cc [modify] https://crrev.com/fe1a9a1fe82a05c3526ee218d2a89d769c8b2107/tools/metrics/histograms/histograms.xml
,
Sep 7
,
Jan 11
Setting defect without priority to Pri-2. |
||||
►
Sign in to add a comment |
||||
Comment 1 by charleszhao@chromium.org
, Aug 16