[autofill] UMA Autofill.Quality.* does not capture false positives and true negatives. |
|||||
Issue descriptionThe Autofill.Quality.* UMA logging does not capture enough cases to allow correct analysis of precision and recall as described at https://en.wikipedia.org/wiki/Precision_and_recall The cases missing are: - the field was not autofillable but it was predicted as such - the field was not autofillable and no type was predicted
,
Apr 12 2017
One way to look at this is that the histograms are currently recorded by the actual type. Meaning, if the "actual" type was NAME then it records NAME_[MATCH, MISMATCH, UNKNOWN]. You also need the reverse: if the "predicted" type was NAME then record NAME_[MATCH, MISMATCH, UNKNOWN]. If you had that then you'd have the full logic table for all possible combinations.
,
Apr 12 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8f71139f8c5982d83fcdcc840a5275698b71438c commit 8f71139f8c5982d83fcdcc840a5275698b71438c Author: rogerm <rogerm@chromium.org> Date: Wed Apr 12 20:09:37 2017 [autofill] Expand Autofill.Quality.* to log false positives/true negatives. This CL adds Autofill.Quality.* logging for the following cases: * the field was not confirmed as autofillable (either the user left the field empty or filled the field with data that was not recognized); and, - A: the field was predicated as autofillable (false positive) - B: the field was not predicted as autofillable (true negative) These round out the logging required to compute precision, recall and other performance metrics as defined at https://en.wikipedia.org/wiki/Precision_and_recall. BUG= 710474 Review-Url: https://codereview.chromium.org/2809213002 Cr-Commit-Position: refs/heads/master@{#464127} [modify] https://crrev.com/8f71139f8c5982d83fcdcc840a5275698b71438c/components/autofill/core/browser/autofill_metrics.h [modify] https://crrev.com/8f71139f8c5982d83fcdcc840a5275698b71438c/components/autofill/core/browser/autofill_metrics_unittest.cc [modify] https://crrev.com/8f71139f8c5982d83fcdcc840a5275698b71438c/components/autofill/core/browser/form_structure.cc [modify] https://crrev.com/8f71139f8c5982d83fcdcc840a5275698b71438c/tools/metrics/histograms/histograms.xml
,
Jun 2 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3c6caa287dacd94ee5e64eedaf44ea987f287bf8 commit 3c6caa287dacd94ee5e64eedaf44ea987f287bf8 Author: rogerm <rogerm@chromium.org> Date: Fri Jun 02 19:58:48 2017 Log Autofill.Quality.*.ByFieldType by predicted type for unknown data. When the actual data type of an field cannot be determined (because it was left empty or was populated with unknown data) log the ByFieldType version of the metrics by the type predicted by autofill. This will allow analysis of the false-positive rate broken down by type. The previous version of the code was logging all of these under Empty/Unknown => AMBIGUOUS. The CL also fleshes out the ByFieldType enumeration in histograms.xml BUG= 710474 R=sebsg Review-Url: https://codereview.chromium.org/2870163003 Cr-Commit-Position: refs/heads/master@{#476766} [modify] https://crrev.com/3c6caa287dacd94ee5e64eedaf44ea987f287bf8/components/autofill/core/browser/autofill_metrics.cc [modify] https://crrev.com/3c6caa287dacd94ee5e64eedaf44ea987f287bf8/components/autofill/core/browser/autofill_metrics.h [modify] https://crrev.com/3c6caa287dacd94ee5e64eedaf44ea987f287bf8/components/autofill/core/browser/autofill_metrics_unittest.cc [modify] https://crrev.com/3c6caa287dacd94ee5e64eedaf44ea987f287bf8/components/autofill/core/browser/autofill_type.cc [modify] https://crrev.com/3c6caa287dacd94ee5e64eedaf44ea987f287bf8/components/autofill/core/browser/field_types.h [modify] https://crrev.com/3c6caa287dacd94ee5e64eedaf44ea987f287bf8/components/autofill/core/browser/form_structure.cc [modify] https://crrev.com/3c6caa287dacd94ee5e64eedaf44ea987f287bf8/tools/metrics/histograms/enums.xml [modify] https://crrev.com/3c6caa287dacd94ee5e64eedaf44ea987f287bf8/tools/metrics/histograms/histograms.xml
,
Jun 6 2017
,
Jun 6 2017
Your change meets the bar and is auto-approved for M60. Please go ahead and merge the CL to branch 3112 manually. Please contact milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), josafat@(ChromeOS), bustamante@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jun 6 2017
,
Jun 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a7bf9ee09dc8641a1385ac2c48edfbd6400555f8 commit a7bf9ee09dc8641a1385ac2c48edfbd6400555f8 Author: Roger McFarlane <rogerm@chromium.org> Date: Tue Jun 06 19:15:28 2017 Log Autofill.Quality.*.ByFieldType by predicted type for unknown data. When the actual data type of an field cannot be determined (because it was left empty or was populated with unknown data) log the ByFieldType version of the metrics by the type predicted by autofill. This will allow analysis of the false-positive rate broken down by type. The previous version of the code was logging all of these under Empty/Unknown => AMBIGUOUS. The CL also fleshes out the ByFieldType enumeration in histograms.xml BUG= 710474 R=sebsg Review-Url: https://codereview.chromium.org/2870163003 Cr-Original-Commit-Position: refs/heads/master@{#476766} Review-Url: https://codereview.chromium.org/2923123005 . Cr-Commit-Position: refs/branch-heads/3112@{#198} Cr-Branched-From: b6460e24cf59f429d69de255538d0fc7a425ccf9-refs/heads/master@{#474897} [modify] https://crrev.com/a7bf9ee09dc8641a1385ac2c48edfbd6400555f8/components/autofill/core/browser/autofill_metrics.cc [modify] https://crrev.com/a7bf9ee09dc8641a1385ac2c48edfbd6400555f8/components/autofill/core/browser/autofill_metrics.h [modify] https://crrev.com/a7bf9ee09dc8641a1385ac2c48edfbd6400555f8/components/autofill/core/browser/autofill_metrics_unittest.cc [modify] https://crrev.com/a7bf9ee09dc8641a1385ac2c48edfbd6400555f8/components/autofill/core/browser/autofill_type.cc [modify] https://crrev.com/a7bf9ee09dc8641a1385ac2c48edfbd6400555f8/components/autofill/core/browser/field_types.h [modify] https://crrev.com/a7bf9ee09dc8641a1385ac2c48edfbd6400555f8/components/autofill/core/browser/form_structure.cc [modify] https://crrev.com/a7bf9ee09dc8641a1385ac2c48edfbd6400555f8/tools/metrics/histograms/enums.xml [modify] https://crrev.com/a7bf9ee09dc8641a1385ac2c48edfbd6400555f8/tools/metrics/histograms/histograms.xml |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by f...@chromium.org
, Apr 12 2017