New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 857454 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Android , Windows , Chrome , Mac
Pri: 1
Type: Bug

Blocking:
issue 862166



Sign in to add a comment

Some forms are not crwodsourced because of too small fields limit in FormStructure::IsMalformed

Project Member Reported by kolos@chromium.org, Jun 28 2018

Issue description

The following signup forms are not crowdsourced because of this issue (the number after link is the number if <input> elements)
https://email.freenet.de/registrierung/frn?epid=e9900000053&tid=2001004  57
https://secure3.hilton.com/en/hh/customer/join/joinHHonors.htm 67
https://sso.ebs.co.kr/idp/join/form.jsp   179
https://www.studio.co.uk/shop/en/studio/request-catalogue 81
https://www.applytexas.org/adappc/gen/profile.WBX 62
https://www.peixeurbano.com.br/cadastre-se 66
https://ebok.energa.pl/register.php 80 
https://www.sisal.it/homepage/registrazione?codicePromozione=SPORT-120&prov=SBAN1_SCOMMESSE_ACQUISIZIONE 73
https://payouts.payoneer.com/partners/or.aspx?pid=YOYIZC74IO2s4KZQp7tgsw%3d%3d 48
http://my.monsterindia.com/create_account.html    222 (!)
https://www.laredoute.fr/webaccount/register.aspx  76

we should increase |kMaxFieldsOnTheForm| at least for some cases (e.g. on submission and password save, but not for every keystroke or focus change)

The server-side cannot support such forms if no info is crowdsourced. It would be good to fix it asap because data collection would take some time
 

Comment 1 by kolos@chromium.org, Jun 28 2018

|IsMalformed| was introduced because of consuming network bandwidth while uploading "malformed" forms.
https://bugs.chromium.org/p/chromium/issues/detail?id=60422
https://bugs.chromium.org/p/chromium/issues/detail?id=57527 

Comment 2 by kolos@chromium.org, Jun 28 2018

|FormStructure::IsMalformed| is called from two functions: 
* FormStructure::EncodeQueryRequest we can allow bigger |kMaxFieldsOnTheForm| for this case as queries are sent only when a page is rendered or a dynamic form appeared (see the call hierarchy of AutofillAgent::ProcessForms).
* FormStructure::EncodeQueryRequest we can allow bigger |kMaxFieldsOnTheForm| only if |observed_submission| is true. 

So, we can limit the number of uploads of a big form.

Comment 3 by kolos@chromium.org, Jun 28 2018

Roger, Sebastien: WDYT?

Comment 4 by kolos@chromium.org, Jun 28 2018

in #2: the second callsite is FormStructure::EncodeUploadRequest

Comment 5 by rogerm@chromium.org, Jun 28 2018

what value whould you like it to be? Or are you proposing we scrap it entirely?

With query response caching and upload throttling in place, the bandwidth concerns would be mitigated (up to a certain query size) except for forms which grow dynamically.

Comment 6 by kolos@chromium.org, Jun 29 2018

I would suggest 250 (as I've found a sites that has 222 <input>s, after filtering out some fields is is 179).

I also hope that protos shouldn't consume so much bandwidth as xml did (the issue happened in 2010)

Comment 7 by kolos@chromium.org, Jun 29 2018

This problem have significant impact on password generation classification. I would consider to merge it back to Beta as the code change would be small. BUT I would merge back with the value of 100 (then it would cover 95% of problem forms)

Comment 8 by battre@chromium.org, Jun 29 2018

I have discussed with Maxim that it might make sense to make this parameter finchable. WDYT?

Comment 9 by rogerm@chromium.org, Jun 29 2018

SGTM. Finchable with a default of 250?

Why do these sites have so many fields? I don't want us to be engaging in escalation. I.e., Sites try to not be autofillable by sending more fields, we keep raising the limit. I'd rather just remove the limit in that case.
Their intention was not to have a race but they had maaaany radio buttons:
e.g.
http://my.monsterindia.com/create_account.html
https://sso.ebs.co.kr/idp/join/form.jsp (you may get redirected and select 3 checkboxes before you get to that site)
Blocking: 862166
Project Member

Comment 12 by bugdroid1@chromium.org, Jul 11

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ca00af373d1cedda85aa03a039fff991f03ccb5d

commit ca00af373d1cedda85aa03a039fff991f03ccb5d
Author: Maxim Kolosovskiy <kolos@chromium.org>
Date: Wed Jul 11 14:16:07 2018

[Autofill] Allow crowdsourcing of big forms

This CL increases |kMaxFieldsOnTheForm| to 100 in |FormStructure::IsMalformed|.
It allows to crowdsource more forms. See the bug for details. Later the
parameter will be increased even more and will be finchable.

UMA histogram is introduced to monitor the size of forms Autofill has to process.

TODO: Finch-controlled threshold and UKM metric.

Bug: 857454
Change-Id: I50c1382dd36e2c707cd34f1a4f139413d9ea6026
Reviewed-on: https://chromium-review.googlesource.com/1131457
Reviewed-by: Roger McFarlane <rogerm@chromium.org>
Reviewed-by: Dominic Battré <battre@chromium.org>
Reviewed-by: Ilya Sherman <isherman@chromium.org>
Commit-Queue: Maxim Kolosovskiy <kolos@chromium.org>
Cr-Commit-Position: refs/heads/master@{#574150}
[modify] https://crrev.com/ca00af373d1cedda85aa03a039fff991f03ccb5d/components/autofill/core/browser/form_structure.cc
[modify] https://crrev.com/ca00af373d1cedda85aa03a039fff991f03ccb5d/components/autofill/core/browser/form_structure_unittest.cc
[modify] https://crrev.com/ca00af373d1cedda85aa03a039fff991f03ccb5d/tools/metrics/histograms/histograms.xml

Cc: nepper@chromium.org battre@chromium.org
Labels: Merge-Request-68
Requesting merge of doubling the maximal number of fields in form that Autofill sends queries for. The change is just increasing the parameter from 48 to 100 fields + adding UMA metric for fields count. 

The change doubles the size of a query Chrome sends to Autofill server. Based on testing, queries will be still less than 1000 bytes. 

Chrome didn't crowdsource the data for some because of this restriction. It is why we don't have data for some forms. 
Project Member

Comment 14 by sheriffbot@chromium.org, Jul 12

Labels: -Merge-Request-68 Hotlist-Merge-Review Merge-Review-68
This bug requires manual review: We are only 11 days from stable.
Please contact the milestone owner if you have questions.
Owners: cmasso@(Android), kariahda@(iOS), bhthompson@(ChromeOS), abdulsyed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Merge-Review-68 Merge-Approved-68
Approving merge for M68. Branch:3440
Project Member

Comment 16 by bugdroid1@chromium.org, Jul 16

Labels: -merge-approved-68 merge-merged-3440
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f24e72942eb44a835f2298d02d75199706018391

commit f24e72942eb44a835f2298d02d75199706018391
Author: Maxim Kolosovskiy <kolos@chromium.org>
Date: Mon Jul 16 09:03:19 2018

[Merge-M68] [Autofill] Allow crowdsourcing of big forms

This CL increases |kMaxFieldsOnTheForm| to 100 in |FormStructure::IsMalformed|.
It allows to crowdsource more forms. See the bug for details. Later the
parameter will be increased even more and will be finchable.

UMA histogram is introduced to monitor the size of forms Autofill has to process.

TODO: Finch-controlled threshold and UKM metric.

Bug: 857454
Change-Id: I50c1382dd36e2c707cd34f1a4f139413d9ea6026
Reviewed-on: https://chromium-review.googlesource.com/1131457
Reviewed-by: Roger McFarlane <rogerm@chromium.org>
Reviewed-by: Dominic Battré <battre@chromium.org>
Reviewed-by: Ilya Sherman <isherman@chromium.org>
Commit-Queue: Maxim Kolosovskiy <kolos@chromium.org>
Cr-Commit-Position: refs/heads/master@{#574150}

Change-Id: Ic53976ddf92e963119909339ec495938820e4abd
Reviewed-on: https://chromium-review.googlesource.com/1137815
Reviewed-by: Maxim Kolosovskiy <kolos@chromium.org>
Cr-Commit-Position: refs/branch-heads/3440@{#676}
Cr-Branched-From: 010ddcfda246975d194964ccf20038ebbdec6084-refs/heads/master@{#561733}
[modify] https://crrev.com/f24e72942eb44a835f2298d02d75199706018391/components/autofill/core/browser/form_structure.cc
[modify] https://crrev.com/f24e72942eb44a835f2298d02d75199706018391/components/autofill/core/browser/form_structure_unittest.cc
[modify] https://crrev.com/f24e72942eb44a835f2298d02d75199706018391/tools/metrics/histograms/histograms.xml

Maxim, can this bug be marked fixed?
97% fixed.

The threshold is increased to 100 (which covers 97% of forms). Perhaps we want to increase it further with Finch. So, there may be more changes here. 
Project Member

Comment 19 by bugdroid1@chromium.org, Dec 11

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ba5fd2ae8690c6e46f78defdf5401661185f06e7

commit ba5fd2ae8690c6e46f78defdf5401661185f06e7
Author: Maxim Kolosovskiy <kolos@chromium.org>
Date: Tue Dec 11 14:56:19 2018

[Autofill] Increase the limit for the number of fields to 250

According to metrics, 0.3% of forms have >100 fields (the current limit), increase the limit to 250 in order to cover all forms.

Autofill throttling limits the number of queries/uploads are sent for a given form by a given user. Therefore, uploading huge forms wouldn't flood network.

Bug: 857454
Change-Id: Ia92a3d6f2033159df121d8728a263c70ee7914f7
Reviewed-on: https://chromium-review.googlesource.com/c/1369772
Reviewed-by: Vadym Doroshenko <dvadym@chromium.org>
Commit-Queue: Maxim Kolosovskiy <kolos@chromium.org>
Cr-Commit-Position: refs/heads/master@{#615523}
[modify] https://crrev.com/ba5fd2ae8690c6e46f78defdf5401661185f06e7/components/autofill/core/browser/form_structure.cc
[modify] https://crrev.com/ba5fd2ae8690c6e46f78defdf5401661185f06e7/components/autofill/core/browser/form_structure_unittest.cc

Labels: -Hotlist-Merge-Review -merge-merged-3440 Merge-Request-72
Labels: -Merge-Request-72 Merge-Approved-72
Pls merge your change to M72 branch 3626 ASAP. Thank you.
Project Member

Comment 23 by bugdroid1@chromium.org, Dec 13

Labels: -merge-approved-72 merge-merged-3626
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6b7e8e1591be22dad241574de2cd77c3b7103cd3

commit 6b7e8e1591be22dad241574de2cd77c3b7103cd3
Author: Maxim Kolosovskiy <kolos@chromium.org>
Date: Thu Dec 13 06:59:24 2018

[Merge-M72][Autofill] Increase the limit for the number of fields to 250

According to metrics, 0.3% of forms have >100 fields (the current limit), increase the limit to 250 in order to cover all forms.

Autofill throttling limits the number of queries/uploads are sent for a given form by a given user. Therefore, uploading huge forms wouldn't flood network.

Bug: 857454
Change-Id: Ia92a3d6f2033159df121d8728a263c70ee7914f7
Reviewed-on: https://chromium-review.googlesource.com/c/1369772
Reviewed-by: Vadym Doroshenko <dvadym@chromium.org>
Commit-Queue: Maxim Kolosovskiy <kolos@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#615523}(cherry picked from commit ba5fd2ae8690c6e46f78defdf5401661185f06e7)
Reviewed-on: https://chromium-review.googlesource.com/c/1375114
Reviewed-by: Maxim Kolosovskiy <kolos@chromium.org>
Cr-Commit-Position: refs/branch-heads/3626@{#320}
Cr-Branched-From: d897fb137fbaaa9355c0c93124cc048824eb1e65-refs/heads/master@{#612437}
[modify] https://crrev.com/6b7e8e1591be22dad241574de2cd77c3b7103cd3/components/autofill/core/browser/form_structure.cc
[modify] https://crrev.com/6b7e8e1591be22dad241574de2cd77c3b7103cd3/components/autofill/core/browser/form_structure_unittest.cc

Labels: Merge-Merged-72-3626
The following revision refers to this bug: 
https://chromium.googlesource.com/chromium/src.git/+/6b7e8e1591be22dad241574de2cd77c3b7103cd3

Commit: 6b7e8e1591be22dad241574de2cd77c3b7103cd3
Author: kolos@chromium.org
Commiter: kolos@chromium.org
Date: 2018-12-13 06:59:24 +0000 UTC

[Merge-M72][Autofill] Increase the limit for the number of fields to 250

According to metrics, 0.3% of forms have >100 fields (the current limit), increase the limit to 250 in order to cover all forms.

Autofill throttling limits the number of queries/uploads are sent for a given form by a given user. Therefore, uploading huge forms wouldn't flood network.

Bug: 857454
Change-Id: Ia92a3d6f2033159df121d8728a263c70ee7914f7
Reviewed-on: https://chromium-review.googlesource.com/c/1369772
Reviewed-by: Vadym Doroshenko <dvadym@chromium.org>
Commit-Queue: Maxim Kolosovskiy <kolos@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#615523}(cherry picked from commit ba5fd2ae8690c6e46f78defdf5401661185f06e7)
Reviewed-on: https://chromium-review.googlesource.com/c/1375114
Reviewed-by: Maxim Kolosovskiy <kolos@chromium.org>
Cr-Commit-Position: refs/branch-heads/3626@{#320}
Cr-Branched-From: d897fb137fbaaa9355c0c93124cc048824eb1e65-refs/heads/master@{#612437}

Sign in to add a comment