Enforce all TBMv2 metrics to require trace categories |
||||
Issue descriptionWith https://github.com/catapult-project/catapult/issues/1818, Ben makes it possible for TBMv2 metrics to declare their required trace categories. I think we should go one step further and require that all tracing metrics must declare their required trace categories. Once that is done, a few good things can happen: 1) In trace viewer UI, we can detect with the current tracing categories, which metrics are applicable and make the metrics trace panel only show those that are applicable. 2) In Telemetry TBMv2 benchmark code, we will no longer need to force user to specify which trace categories are needed (e.g: https://cs.chromium.org/chromium/src/tools/perf/benchmarks/system_health.py?q=system_health.py&l=100). As the metrics are specified, we can just infer which trace categories are required by invoke some script that would spit out the trace categories from those metrics. 3) Given an arbitrary trace, we can know which metrics are applicable for it & be able to fail early with error like "trace does not contain required categories for metrics". This is much better than showing cpu time is 0 for a memory trace, for example. I file this bug so anyone who has bandwidth can just take it.
,
Apr 20 2018
,
Apr 23 2018
A few things we should clarify: When I state that I'm interested in metric X, do we want to include categories required for the diagnostic metrics of X? What about metrics which are computable using a small set of categories, but provide slightly better data with a few extra categories?
,
Apr 23 2018
> When I state that I'm interested in metric X, do we want to include categories required for the diagnostic metrics of X? This is an interesting question! Maybe we have something like "basic_categories_requirement" vs "full_categories_requirement_for_diagnostic"? > What about metrics which are computable using a small set of categories, but provide slightly better data with a few extra categories? We wouldn't block people from adding extra categories, but I think the minimal set should be automagically enabled if possible
,
Apr 23 2018
#4 SGTM.
,
Apr 30 2018
,
Apr 30 2018
tdresser@, re: > What about metrics which are computable using a small set of categories, but provide slightly better data with a few extra categories? I think that we should probably avoid this at all costs: this is the way that the CPU metrics have long worked and it confuses the heck out of people.
,
Apr 30 2018
Hmmm, what should we do in these cases then? Refuse to compute the CPU metrics unless all categories required for breakdowns are present?
,
Apr 30 2018
Instead of having two different types of required categories for a given metric, what about having multiple versions of metrics? "cpuMetric" can require "toplevel", "extendedCpuMetric" can require "toplevel,rail,v8,etc". It's ok if extendedCpuMetric does nothing but call cpuMetric directly. Eventually, the "extended" breakdowns and whatever other functionality that depends on rail/v8/etc can be refactored out of cpuMetric into extendedCpuMetric.
,
Apr 30 2018
#9 sounds correct to me.
,
May 3 2018
That sounds reasonable.
,
Aug 14
|
||||
►
Sign in to add a comment |
||||
Comment 1 by nednguyen@chromium.org
, Apr 20 2018