New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 648647 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner: ----
Closed: Sep 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug



Sign in to add a comment

histograms.xml is too big

Project Member Reported by wfh@chromium.org, Sep 20 2016

Issue description

histograms.xml has grown to a size that:

 1. It takes longer than the "renderer is hung" seconds to load in code search, so I get the "your renderer is hung" dialog. (see attached image).
 2. You cannot view the diff in code reviews, making code reviews harder.

I wonder if we could consider either:

 1. Splitting histograms.xml into smaller files somehow. Perhaps the pretty_print.py script could handle all this automatically?
 2. Making code search handle large files better
 4. Making code review handle large files better

Putting in Metrics component but if we decide we cannot change histograms.xml then I'll assign to infra to look at.
 
histograms_xml_is_too_damn_big.png
9.6 KB View Download
See also some related issues:

https://bugs.chromium.org/p/chromium/issues/detail?id=627574
https://bugs.chromium.org/p/chromium/issues/detail?id=383831

In terms of histograms.xml splitting, there's been some different proposals:
  1. Splitting out the obsolete histograms to a separate file.
  2. Splitting out histograms by code directory that uses them - e.g. content/ could have its own histograms.xml file, net/ could have its own histograms.xml file, etc.

Personally, I think best solution is to just fix the tools (code search, code review, etc). Unfortunately, it's a solution that requires persuading other teams to do stuff, which can be hard to do.

In terms of splitting histograms.xml, solution 1. is reasonable but on its own I don't think it helps much. You're left still with two very large files - and then if someone is obsoleting a histogram, it's more work and harder to review and they now have to touch two large files that probably are plagued by the same problems.

Kind of orthogonal to splitting the files is we should come up with a policy for removing obsolete histograms entirely. For example, if a histogram has been obsolete for over a year, then it's unlikely someone would want to look at it's historical data and we can remove it.

In terms of option "2" of splitting histograms files more granularly, I feel like this requires a lot of overhead - because either we don't split things nearly enough too matter (so files are huge) or we end up with a ton of histograms.xml files which I'm not sure is desirable - things like the presubmits still need to look at the global picture to make sure there's no naming conflicts and we'd have to fetch all of them server-side. Plus it's just a lot more files in repo, possibly cluttering things.

.... now all of those are just my opinions and I can be convinced otherwise, but my intuition is to just push to get the tools fixed and from our side just come up with a policy and tool to periodically clean out obsolete histograms after some time.
Arguably, we should be able to remove all obsolete entries in the file, without affecting what our dashboards can show.  Probably we'd achieve this by keeping some master file checked into google3, which has better support for large files than the Chromium repo.

Comment 3 by wfh@chromium.org, Sep 21 2016

Status: WontFix (was: Unconfirmed)
Thanks for the links to those other issues. it sounds like many of the issues will be fixed with gerrit, and the codesearch issue can be worked around by just viewing histograms.xml in internal codesearch?

I agree maybe removing the obsolete histograms from the copy in chromium, if there is some way of making sure they stick around in the internal histograms.xml.

I'm happy to close this as WontFix. Thanks. (feel free to reopen if you still want to track it).

Sign in to add a comment