New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 710423 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Nov 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Android
Pri: 2
Type: Bug



Sign in to add a comment

Minimize cellular traffic for ntp_snippets

Project Member Reported by jkrcal@chromium.org, Apr 11 2017

Issue description

We should minimize data we consume over cellular connection. 
Especially downlink in India / EM.


Data from M57 Stable (only cellular traffic, from last 7 days):
(with only a partial roll-out of soft fetches)

Worldwide data:
***************

NTPSnippets 216 GB

UMA 13.1 TB
Variation 640 GB
Sync 4.1 TB
Omnibox 565 GB
Domain Reliability 960 GB

We consume around 1% of the data compared to the above mentioned services.


In India
********
 - upstream:
NTPSnippets 2GB
all tagged traffic 1,525TB

 - downstream:
NTPSnippets 34GB
all tagged traffic 320GB

We consume 0.3% of tagged data. 
We consume more than 10% of downstream tagged data!



Based on following histograms:
DataUse.MessageSize.AllServices.Downstream.Background.Cellular
DataUse.MessageSize.AllServices.Downstream.Background.NotCellular
DataUse.MessageSize.AllServices.Downstream.Foreground.Cellular
DataUse.MessageSize.AllServices.Downstream.Foreground.NotCellular
DataUse.MessageSize.AllServices.Upstream.Background.Cellular
DataUse.MessageSize.AllServices.Upstream.Background.NotCellular
DataUse.MessageSize.AllServices.Upstream.Foreground.Cellular
DataUse.MessageSize.AllServices.Upstream.Foreground.NotCellular
 

Comment 1 by jkrcal@chromium.org, Apr 11 2017

The numbers might get worse noticably worse by the introduction of soft fetches.

For M59, this is addressed by having a separate knob for frequency of soft background fetches over wifi. We should:
 - get the user classifier classify all our engaged users in the top class (top 5% instead of the current ~1%)
 - set-up a reasonably high fetching frequency for the top class
 - set a considerably lower fetching frequency for other users, when not on wi-fi.
 - be even more radical with fetching frequencies for rare NTP users?
Thanks for collecting this data, Jan!

We need to be careful though to not come to the wrong conclusions too quickly.
Is this only looking at data used in the background or are we also counting data that's being used interatively (e.g. loading the thumbnails, loading more suggestions via fetch-more, etc.)?

We should also compare it with other "assistive" traffic -- I guess traffic for omnibox suggestions sounds interesting.

I'd be careful to blindly optimize this metric and not fetching suggestions for not-yet engaged users -- if a user only occasionally goes to the NTP, we only have few chances to convince him of a great offering.

That said, we should of course be mindful about data usage and avoid burning data with no value. This means we need to understand the value of the feature having fresh data (PM topic) and when reducing data usage, make sure to save at places that matter.

Comment 3 by jkrcal@chromium.org, Apr 11 2017

This is all traffic together. For M58 on we will have separate measurements for the thumbnails.

We cannot distinguish fetch-more from background fetches here but we have other metrics that will allow us to estimate the respective traffic.

Let's wait with this issue for a while until we get better measurements in M58 Stable. I would still like to give it a thought for the params in M59 Stable.

Comment 4 by mastiz@chromium.org, Jul 25 2017

Owner: jkrcal@chromium.org
Status: Assigned (was: Available)
Assigning to jkrcal@ to avoid P1s without owner. Please provide an update and/or lower priority.

Comment 5 by jkrcal@chromium.org, Jul 31 2017

Cc: sfiera@chromium.org
Labels: -Pri-1 Pri-2
Decreasing priority.

Current data from Stable (last week):
 - the background cellular data consumption is negligible (0.5% of all traffic)
 - the foreground upstream cellular data consumption is super negligible (0.1%)
 - the only (slightly) concerning type is foreground downstream cellular data:
    - suggestions: 0.5% - still okay
    - thumbnails: 3.5% - worth looking at, is there a low-hanging fruit?

sfiera@, do you have any insights into potential data savings for the thumbnails from the hackathon, earlier this year?

Comment 7 by sfiera@chromium.org, Jul 31 2017

You mean re: cropping? I was looking at purely client-side improvements, which means we couldn't improve data usage on our own. I can still summarize some points from the thumbnail set I used:

* Most of our thumbnails are ~300px wide (~75%), often in aspect ratios of 16:9 (~50% overall)
* Only 1 of 50 was already square.
* On average, about 40% of our thumbnails are never seen if we crop to square.
* All JPEG

Keep in mind, this thumbnail set is several months old, but I'd guess it's still representative.

Comment 8 by jkrcal@chromium.org, Jul 31 2017

Cc: tschumann@chromium.org
Thanks for the data points, Chris!

I see two possibilities:
 A) server-side cropping to square (assuming 16:9 and width of 300px, this is something like 170x170px),
 B) server-side downscaling for Wi-Fi / low-end devices (maybe to size of 100x100px)

Tim, two quick qq:
 - do you think the thumbnail traffic is worth looking at?
 - are we important enough customer to FCS/Feed so that they might care about A) or B) or both?
+1 to trying A and B. Also we could try C: use lossy webp encoding instead of jpg: go/preparing-image-assets
Cc: bengr@chromium.org
[adding Ben to assess whether this is something worth improving]

This is useful data.  Do I understand it correctly, that this is 5% of foreground cellular data which chrome services use (i.e. "background" stuff but no actual web pages)?

I don't think it's worth to do anything with the existing pre-Jardin product.
However, it's good data for the feed integration (client protocol).

So unless Ben objects, I'd mark this as won't fix.



Cc: rajendrant@chromium.org
I don't think this is worth further improving. However, it would be good also to look at per-user data use metrics to asses whether this is a more significant fraction of data use for some users. E.g., if we learned that this is 50% of data use for a large set of users, then we should do something about it. 

rajendrant@ can help you with that analysis.
Status: WontFix (was: Assigned)
Closing as WontFix based on insights from go/chrome-datause.

Sign in to add a comment