According to https://developer.android.com/reference/java/util/concurrent/ThreadPoolExecutor.html, there is a way to define a RejectedExecutionHandler that would tell us some information about tasks that fail to get scheduled. Maybe we should add one of these and collect data.

Hmm... it seems that AsyncTask.THREAD_POOL_EXECUTOR uses the default RejectedExecutionHandler, which would raise an RejectedExecutionException in case the task fails to be scheduled, which would result in a java unhandled exception crash. 50k+ such crashes would have definitely showed up.

This likely means that the other 50k+ async tasks just weren't scheduled, or were scheduled but never completed. I'll add a UMA stat to record how many tasks were scheduled.

It seems another UMA stat already does that. UMA.Debug.EnableCrashUpload.DeferredStartUptime2 is recorded when initDeferredStartupForApp is called, which then queues the AsyncTask. That is recorded 121,656 times. So almost all the tasks do complete until duration is recorded.

Now to figure out why there are 52k extra records...

https://uma.googleplex.com/p/chrome/histograms/?endDate=20170119&dayCount=1&histograms=UMA.Debug.EnableCrashUpload.DeferredStartUpDuration%2CUMA.Debug.EnableCrashUpload.DeferredStartUpDurationAsync%2CUMA.Debug.EnableCrashUpload.DeferredStartUptime2&fixupData=true&showMax=true&filters=channel%2Ceq%2C1%2Cisofficial%2Ceq%2CTrue&implicitFilters=isofficial

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/467d339230d5db21352a3568e906ab96818062d5

commit 467d339230d5db21352a3568e906ab96818062d5
Author: wnwen <wnwen@chromium.org>
Date: Tue Jan 24 20:03:51 2017

Android: Re-order deferred async task UMA record

Ensure that when the UMA record for duration is written the entire async
task is done.

BUG= 683944 

Review-Url: https://codereview.chromium.org/2653733002
Cr-Commit-Position: refs/heads/master@{#445789}

[modify] https://crrev.com/467d339230d5db21352a3568e906ab96818062d5/chrome/android/java/src/org/chromium/chrome/browser/DeferredStartupHandler.java
[modify] https://crrev.com/467d339230d5db21352a3568e906ab96818062d5/tools/metrics/histograms/histograms.xml

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1b94b8d69f748e9c8e686220c26bb87c1a461602

commit 1b94b8d69f748e9c8e686220c26bb87c1a461602
Author: wnwen <wnwen@chromium.org>
Date: Wed Jan 25 01:56:20 2017

Android: Guard against multiple stat uploads

Deferred startup handler should only record UMA at most once. Clients
should add their own tasks for recording startup metrics.

BUG= 683944 

Review-Url: https://codereview.chromium.org/2657673002
Cr-Commit-Position: refs/heads/master@{#445909}

[modify] https://crrev.com/1b94b8d69f748e9c8e686220c26bb87c1a461602/chrome/android/java/src/org/chromium/chrome/browser/DeferredStartupHandler.java

New stats show that CL in #5 fixed the discrepancy. We were over counting completes.

https://uma.googleplex.com/p/chrome/histograms/?endDate=20170129&dayCount=1&histograms=UMA.Debug.EnableCrashUpload.DeferredStartUpAsyncTaskDuration%2CUMA.Debug.EnableCrashUpload.DeferredStartUpCompleteTime%2CUMA.Debug.EnableCrashUpload.DeferredStartUptime2%2CUMA.Debug.EnableCrashUpload.Uptime3&fixupData=true&showMax=true&filters=channel%2Ceq%2C1%2Cversion_tags%2Ccnt%2CL%2Cisofficial%2Ceq%2CTrue&implicitFilters=isofficial