Re-assigning to new sync owners

600k bookmarks does indeed seem excessive, but I'm not sure what a good solution would be - simply dropping most of them clearly isn't great.

Do we have any idea how users end up in this state? Clearly nobody will add half a million bookmarks manually. Is it an import gone wrong? A bug in Sync that duplicates bookmarks? A rogue extension? Maybe we can fix the problem at the root, instead of mitigating symptoms.
Also, what percentage of users is affected? That number would help prioritizing this.

FWIW, mamir@ is working on a refactoring that will significantly reduce the memory overhead of Sync for bookmarks, but I imagine that the bookmarks UI will anyway be unusable with so many items.

Assigning to myself for now for the investigation/prioritization.

FYI, I remember Yaron telling me they did a server side script that removes duplicate bookmarks from the sync storage in google and once the user syncs again the number of bookmarks have reduced. I didn't put effort into this since this is observed on too less number of users. 

You can look at this uma to see how many people are affected
https://uma.googleplex.com/p/chrome/histograms/?endDate=20180324&dayCount=1&histograms=Sync.ModelTypeCount.BOOKMARK&fixupData=true&showMax=true&analysis=0.25%200.5%200.75%200.95%200.99%200.995&filters=isofficial%2Ceq%2CTrue&implicitFilters=isofficial

It looked like few 1000 hits more than 100K bookmarks. But, another thing to note is it turns out people with larger bookmarks tend to use chrome more. Since we measure memory usage every X minutes, and bookmarks always stays in memory, it shows up more often on memory reports than the number of users.

I looked to UMA metric : Sync.ModelTypeCount.BOOKMARK

For windows, we can estimate that a little bit less than 1% of the reported population has above 10K bookmarks or more.

I assume 10K bookmarks is an reasonable upper limit for amount of valid bookmarks that could be enforced by Chrome.

I double checked with the received reports and the size of the memlog population and the UMA metrics seems about the right magnitude. We can't be exact with memlog metrics since a given user can sent multiple reports.

Looking at Sync.ModelTypeCount.BOOKMARK for Windows Stable [1], it looks like maybe 0.15% of users have > 10k bookmarks. The value at the 99th %ile is 2316. So the number of users affected by a hypothetical 10k limit on the number of bookmarks would be pretty small.

[1] https://uma.googleplex.com/p/chrome/histograms/?endDate=20180325&dayCount=7&histograms=Sync.ModelTypeCount.BOOKMARK&fixupData=true&showMax=true&analysis=0.5%200.75%200.95%200.99&filters=platform%2Ceq%2CW%2Cchannel%2Ceq%2C4%2Cisofficial%2Ceq%2CTrue&implicitFilters=isofficial

As discussed out-of-band: The relative overhead introduced by Sync is clearly too large. This is actively being worked on.
If we want to add any hard limit on the number of bookmarks, that would need to happen within bookmarks, not sync. As such, assigning to sky@ as owner of components/bookmarks/ for triage/prioritization.

In principle, I think it makes sense to put hard limits on capacities of any storage component in Chrome, as it's another vector of abuse for misbehaving components (extensions, third party software). We enforce these kinds of limits for content storage (local storage and the like), so why not for our own components?

In terms of picking a value, catering to the 99.5 or 99.9th percentile is probably reasonable. We know that our bookmark UX breaks down terribly for large numbers of bookmarks, so the proposed 10K seems reasonable on the surface.

If we have a cap we need to logic to enforce the cap, and this is where things get murky. There a couple ways to go:

- Enforce a hard cap and disallow adding new bookmarks. This can fail silently (1) or message the user (2).
- Enforce the cap, but always allow adding new bookmarks. This can silently expire a bookmark (3) or message the user (4).

Obviously, the easiest to implement is (1), but that's the absolute worst user experience. I'd say that options (3) and (4) (new bookmarks valued more than old bookmarks) are both better than (1) or (2).

Comparing (3) and (4), (3) is easiest to implement. (4) might be too heavy handed. Potentially something that involves an LRU expiry, with messaging only when the bookmark to be expired was actually used within a given time frame?

The UX is also somewhat problematic. Should the UX be blocking and require a user decision (a), or is it purely to inform the user of the action that was taken on their behalf (b)? We tend to shy away from UX of the form of (b).

Are there other ways to address this? Do we know if this looks like real users, or malicious extensions / automation / testing / fuzzing? Another potential way to address this is via rate limiting on the actual addition of bookmarks?

Then there's the question of how to migrate existing users that are already over the cap. Simply trim? Warn before trimming? Let them stay over the cap until they fall below it again?

Seems like a fair amount of work for a problem that addresses a small part of the user population. sky@, your thoughts?

(FWIW, I think all new storage components should be build with plans to prevent unbounded growth. For existing components, each case has to be considered individually.)

Triage ping: Any updates on this?
Given how rare it is, should we demote it to P3?

Reducing prio on this, since a) it's very rare, and b) there's an ongoing refactoring that will remove most of the Sync memory overhead (bug 516866).

***Mass UI Triage***