New issue
Advanced search Search tips

Issue 755337 link

Starred by 1 user
Status: Fixed
Owner:
dlkumar@google.com
Closed: Jan 2018
Cc:
chrome-payments-team@google.com
rogerm@chromium.org
jsaul@google.com
ma...@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Feature



Sign in to add a comment

Credit card autofill ablation : Introduce new flag to ablate when cc autofill form detected & cards are on file

Project Member Reported by dlkumar@google.com, Aug 14 2017 
We should compare all above metrics in conditional slice of - [Credit card form detected * User has cards on file (local or payments)]. At this time ablation happens without knowing whether user has cards on file and that would dilute metrics. So current ablation experiment is finding metrics for the slice of “credit card form detected” and excludes “user has cards on file” condition. Based on current metrics (shown suggestion/interacted with form) rate is 29%, so metrics are diluted by 3X. Counterfactual logging (User should have shown suggestions) should be implemented to compare conditional metrics properly



Refer Ablation experiment setup for autofill proposal - https://docs.google.com/document/d/1vMGO7sfzYDXpev_5ftyLH-WoKpCHA4I9lVLYuesVT9c/edit#heading=h.k6sbad93o9zk
Project Member

Comment 1 by bugdroid1@chromium.org, Oct 12 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3d541cac26d562d9a6f621362faef99d807056f6

commit 3d541cac26d562d9a6f621362faef99d807056f6
Author: Shanfeng Zhang <szhangcs@google.com>
Date: Thu Oct 12 20:23:05 2017

Credit card autofill ablation experiment setup

This CL is patched from  https://chromium-review.googlesource.com/c/chromium/src/+/615603
Handling this since Kumar doesn't have enough time to finish this in this week.

Bug:755337

Abalation doc - https://docs.google.com/document/d/1vMGO7sfzYDXpev_5ftyLH-WoKpCHA4I9lVLYuesVT9c/edit?ts=59826c12#heading=h.k6sbad93o9zk

Conditional Metrics to log in control & ablation arm
>>
Submitted with suggestion shown (once)
>>

Change-Id: Ie1be217bc9dfcca36419e6635f9f8459be1ec0b5
Reviewed-on: https://chromium-review.googlesource.com/711021
Commit-Queue: Shanfeng Zhang <szhangcs@google.com>
Reviewed-by: Roger McFarlane <rogerm@chromium.org>
Cr-Commit-Position: refs/heads/master@{#508441}
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/chrome/browser/about_flags.cc
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/chrome/browser/flag_descriptions.cc
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/chrome/browser/flag_descriptions.h
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/components/autofill/core/browser/autofill_experiments.cc
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/components/autofill/core/browser/autofill_experiments.h
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/components/autofill/core/browser/autofill_manager.cc
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/components/autofill/core/browser/autofill_manager.h
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/components/autofill/core/browser/autofill_manager_unittest.cc
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/components/autofill/core/browser/autofill_metrics.cc
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/components/autofill/core/browser/autofill_metrics.h
[modify] https://crrev.com/3d541cac26d562d9a6f621362faef99d807056f6/tools/metrics/histograms/enums.xml
Project Member

Comment 2 by bugdroid1@chromium.org, Oct 13 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd

commit 5bfb0010fe3b70d0fe39beac1f233074b9eba3bd
Author: Balazs Engedy <engedy@chromium.org>
Date: Fri Oct 13 08:39:46 2017

Revert "Credit card autofill ablation experiment setup"

This reverts commit 3d541cac26d562d9a6f621362faef99d807056f6.

Reason for revert: 

MemorySanitizer: use-of-uninitialized-value in:
autofill::AutofillMetrics::FormEventLogger::OnFormSubmitted

Original change's description:
> Credit card autofill ablation experiment setup
> 
> This CL is patched from  https://chromium-review.googlesource.com/c/chromium/src/+/615603
> Handling this since Kumar doesn't have enough time to finish this in this week.
> 
> Bug:755337
> 
> Abalation doc - https://docs.google.com/document/d/1vMGO7sfzYDXpev_5ftyLH-WoKpCHA4I9lVLYuesVT9c/edit?ts=59826c12#heading=h.k6sbad93o9zk
> 
> Conditional Metrics to log in control & ablation arm
> >>
> Submitted with suggestion shown (once)
> >>
> 
> Change-Id: Ie1be217bc9dfcca36419e6635f9f8459be1ec0b5
> Reviewed-on: https://chromium-review.googlesource.com/711021
> Commit-Queue: Shanfeng Zhang <szhangcs@google.com>
> Reviewed-by: Roger McFarlane <rogerm@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#508441}

TBR=rogerm@chromium.org,mathp@chromium.org,rkaplow@chromium.org,szhangcs@google.com,dlkumar@google.com

Change-Id: I6dbc69ba0efdbcd2be380db9a67a1843fa54b4ae
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  755337 
Reviewed-on: https://chromium-review.googlesource.com/718356
Reviewed-by: Balazs Engedy <engedy@chromium.org>
Commit-Queue: Balazs Engedy <engedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#508647}
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/chrome/browser/about_flags.cc
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/chrome/browser/flag_descriptions.cc
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/chrome/browser/flag_descriptions.h
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/components/autofill/core/browser/autofill_experiments.cc
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/components/autofill/core/browser/autofill_experiments.h
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/components/autofill/core/browser/autofill_manager.cc
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/components/autofill/core/browser/autofill_manager.h
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/components/autofill/core/browser/autofill_manager_unittest.cc
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/components/autofill/core/browser/autofill_metrics.cc
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/components/autofill/core/browser/autofill_metrics.h
[modify] https://crrev.com/5bfb0010fe3b70d0fe39beac1f233074b9eba3bd/tools/metrics/histograms/enums.xml
Project Member

Comment 3 by bugdroid1@chromium.org, Oct 17 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/19c165680ac7d726a78b79ef2e1db36c43cbc0af

commit 19c165680ac7d726a78b79ef2e1db36c43cbc0af
Author: Shanfeng Zhang <szhangcs@google.com>
Date: Tue Oct 17 02:30:40 2017

Reland "Credit card autofill ablation experiment setup"

This is a reland of 3d541cac26d562d9a6f621362faef99d807056f6
Original change's description:
> Credit card autofill ablation experiment setup
>
> This CL is patched from  https://chromium-review.googlesource.com/c/chromium/src/+/615603
> Handling this since Kumar doesn't have enough time to finish this in this week.
>
> Bug:755337
>
> Abalation doc - https://docs.google.com/document/d/1vMGO7sfzYDXpev_5ftyLH-WoKpCHA4I9lVLYuesVT9c/edit?ts=59826c12#heading=h.k6sbad93o9zk
>
> Conditional Metrics to log in control & ablation arm
> >>
> Submitted with suggestion shown (once)
> >>
>
> Change-Id: Ie1be217bc9dfcca36419e6635f9f8459be1ec0b5
> Reviewed-on: https://chromium-review.googlesource.com/711021
> Commit-Queue: Shanfeng Zhang <szhangcs@google.com>
> Reviewed-by: Roger McFarlane <rogerm@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#508441}

Bug:  755337 
Change-Id: Ife233986e1f3877cea7cdcc128398c64bb1a8818
Reviewed-on: https://chromium-review.googlesource.com/719411
Reviewed-by: Roger McFarlane <rogerm@chromium.org>
Commit-Queue: Shanfeng Zhang <szhangcs@google.com>
Cr-Commit-Position: refs/heads/master@{#509220}
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/chrome/browser/about_flags.cc
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/chrome/browser/flag_descriptions.cc
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/chrome/browser/flag_descriptions.h
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/components/autofill/core/browser/autofill_experiments.cc
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/components/autofill/core/browser/autofill_experiments.h
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/components/autofill/core/browser/autofill_manager.cc
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/components/autofill/core/browser/autofill_manager.h
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/components/autofill/core/browser/autofill_manager_unittest.cc
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/components/autofill/core/browser/autofill_metrics.cc
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/components/autofill/core/browser/autofill_metrics.h
[modify] https://crrev.com/19c165680ac7d726a78b79ef2e1db36c43cbc0af/tools/metrics/histograms/enums.xml

Comment 4 by dlkumar@google.com, Jan 19 2018

Status: Fixed (was: Untriaged)
Closing this. as changes are made and experiment is done already.
►

Sign in to add a comment