New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 753485 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: May 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature

Blocking:
issue 756948



Sign in to add a comment

Prevent uploading expired histograms.

Project Member Reported by iburak@google.com, Aug 8 2017

Issue description

Get expired histograms from histograms.xml and do not upload them.
 
Are we sure that we want to prevent upload, rather than filing bugs as we do with histograms that haven't been viewed in a while?  It's easy to for humans to misjudge when an expiration should be set to, and I expect that we'll see many CLs bumping expiration dates.  Do we really want to have mistakes or delays in this process cause data loss?  It's not uncommon for people to want to perform multi-day analysis, so a gap in a single day can actually be quite problematic.

(On the other hand, it's already the case that occasionally, individual metrics  will have noisy or corrupted data on some days, resulting in days that need to be excluded from multi-day analysis.  So, this wouldn't be strictly a new issue; it would just make an existing issue become much more common.)

Perhaps we can strike a balance by filing a bug as metrics are about to expire, and preventing their upload after N days -- say N=30?  This still gives us the benefit of data savings from metrics that are unused and unowned, but it gives the owners a grace period to avoid data loss.

Comment 2 by iburak@google.com, Aug 17 2017

We are thinking about filing a bug seven days before the expiry date and stopping uploading a histogram when the expiry date comes. Also, the bug will be updated on this date to notify the owners that Chrome will no longer record this histogram. 
I think seven days is probably too short of a notice. I would suggest 30 days and then pinging it again 14 days and 7 days before.

Otherwise, 7 days could be easily missed if someone's just on vacation.
I agree, we'll want a duration that's vacation-friendly.

I think in another thread someone else suggested having histograms expire by milestone rather than by date.  I thought that was actually a really great idea!  Is that the current plan, or is the plan to use dates?  FWIW, one big advantage of using milestones is that there wouldn't ever be a need to merge expiry changes to branches.
Right now we're exploring whether we can get the data corresponding to a milestone (we're seeing it this date can be checked in by the script that bumps the milestone in Chrome repo).

So then we can have the best of both worlds - dates that are easier to reason about for humans (i.e. it's easier to set expiry date 1 year later when introducing a metric than figuring out milestone offsets) but still have things only expire at new milestone transitions. 
Cool, that sounds like a nice implementation strategy!
Project Member

Comment 8 by bugdroid1@chromium.org, Sep 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5685499bb10100030d43821f098ebd7437286588

commit 5685499bb10100030d43821f098ebd7437286588
Author: Ira Burak <iburak@google.com>
Date: Tue Sep 12 20:04:19 2017

Fix compilation bug for Windows.

Windows doesn't allow defining empty arrays, so for the case when there are no expired histograms, the dummy value is added.

Bug:  753485 
Change-Id: I69efcaa7ea63939d1e1b75497f6a1a252de010f2
Reviewed-on: https://chromium-review.googlesource.com/650468
Reviewed-by: Alexei Svitkine <asvitkine@chromium.org>
Reviewed-by: Jesse Doherty <jwd@chromium.org>
Commit-Queue: Ira Burak <iburak@google.com>
Cr-Commit-Position: refs/heads/master@{#501371}
[modify] https://crrev.com/5685499bb10100030d43821f098ebd7437286588/tools/metrics/histograms/generate_expired_histograms_array.py
[modify] https://crrev.com/5685499bb10100030d43821f098ebd7437286588/tools/metrics/histograms/generate_expired_histograms_array_unittest.py

Project Member

Comment 9 by bugdroid1@chromium.org, Sep 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/411c1a6e3ff0586a5f6c931480de270b35a3adf3

commit 411c1a6e3ff0586a5f6c931480de270b35a3adf3
Author: Ira Burak <iburak@google.com>
Date: Wed Sep 13 13:43:28 2017

Add functionality to base::StatisticsRecorder to filter expired metrics.

Bug:  753485 
Change-Id: I157bdfbdfeb29ff00df39cc220f1cbf4f10475a0
Reviewed-on: https://chromium-review.googlesource.com/615664
Commit-Queue: Ira Burak <iburak@google.com>
Reviewed-by: Jesse Doherty <jwd@chromium.org>
Reviewed-by: Lei Zhang <thestig@chromium.org>
Reviewed-by: Alexei Svitkine <asvitkine@chromium.org>
Cr-Commit-Position: refs/heads/master@{#501613}
[modify] https://crrev.com/411c1a6e3ff0586a5f6c931480de270b35a3adf3/base/BUILD.gn
[add] https://crrev.com/411c1a6e3ff0586a5f6c931480de270b35a3adf3/base/metrics/record_histogram_checker.h
[modify] https://crrev.com/411c1a6e3ff0586a5f6c931480de270b35a3adf3/base/metrics/statistics_recorder.cc
[modify] https://crrev.com/411c1a6e3ff0586a5f6c931480de270b35a3adf3/base/metrics/statistics_recorder.h
[modify] https://crrev.com/411c1a6e3ff0586a5f6c931480de270b35a3adf3/base/metrics/statistics_recorder_unittest.cc
[modify] https://crrev.com/411c1a6e3ff0586a5f6c931480de270b35a3adf3/chrome/browser/chrome_browser_main.cc
[modify] https://crrev.com/411c1a6e3ff0586a5f6c931480de270b35a3adf3/components/metrics/BUILD.gn
[add] https://crrev.com/411c1a6e3ff0586a5f6c931480de270b35a3adf3/components/metrics/expired_histograms_checker.cc
[add] https://crrev.com/411c1a6e3ff0586a5f6c931480de270b35a3adf3/components/metrics/expired_histograms_checker.h

Project Member

Comment 10 by bugdroid1@chromium.org, Sep 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d63c2034e96e75a83227372afc5559cd50751630

commit d63c2034e96e75a83227372afc5559cd50751630
Author: Ira Burak <iburak@google.com>
Date: Wed Sep 13 15:36:16 2017

Manually add MAJOR_BRANCH_DATE for M63. For the later versions, this file will be generated automatically.

MAJOR_BRANCH_DATE file contains a date of the last major branch. This date will be used as a base date to compare expiry dates of histograms with.

Bug:  753485 
Change-Id: I2c336df168376c3ac909c877b5b938dad96a4347
Reviewed-on: https://chromium-review.googlesource.com/661068
Reviewed-by: Lei Zhang <thestig@chromium.org>
Reviewed-by: Jesse Doherty <jwd@chromium.org>
Reviewed-by: Alexei Svitkine <asvitkine@chromium.org>
Commit-Queue: Ira Burak <iburak@google.com>
Cr-Commit-Position: refs/heads/master@{#501640}
[add] https://crrev.com/d63c2034e96e75a83227372afc5559cd50751630/chrome/MAJOR_BRANCH_DATE

Blocking: 756948
Project Member

Comment 13 by sheriffbot@chromium.org, Oct 27 2017

Labels: Hotlist-Recharge-BouncingOwner
Owner: ----
Status: Untriaged (was: Assigned)
The assigned owner "iburak@google.com" is not able to receive e-mails, please re-triage.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Owner: jwd@chromium.org
Status: Assigned (was: Untriaged)
Project Member

Comment 15 by bugdroid1@chromium.org, Feb 2 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chrome/tools/release/scripts/+/1c594093b3119ebb3fa89f6b9320a592b59820ca

commit 1c594093b3119ebb3fa89f6b9320a592b59820ca
Author: Gayane Petrosyan <gayane@google.com>
Date: Fri Feb 02 21:06:48 2018

Project Member

Comment 16 by bugdroid1@chromium.org, Mar 20 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chrome/tools/release/scripts/+/8a380589ce2059c11408bb241019050c46220088

commit 8a380589ce2059c11408bb241019050c46220088
Author: Gayane Petrosyan <gayane@google.com>
Date: Tue Mar 20 15:09:26 2018

Owner: gayane@chromium.org

Sign in to add a comment