sysmon crashlooping on invalid cpu % value (string?) |
||
Issue description
NOTICE:root:ts_mon was set up.
INFO:chromite.scripts.sysmon.mainlib:Enabling prod host metric collection.
ERROR:chromite.scripts.sysmon.loop:Error during loop.
Traceback (most recent call last):
File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/loop.py", line 33, in loop_once
self._callback()
File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/mainlib.py", line 53, in __call__
proc_metrics.collect_proc_info()
File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/proc_metrics.py", line 27, in collect_proc_info
collector.collect()
File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/proc_metrics.py", line 47, in collect
self._flush()
File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/proc_metrics.py", line 58, in _flush
metric.flush()
File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/proc_metrics.py", line 96, in flush
_cpu_percent_metric.set(self._cpu_percent, fields=self._fields)
File "/usr/local/google/home/chromeos-test/chromiumos/chromite/third_party/infra_libs/ts_mon/common/metrics.py", line 394, in set
raise errors.MonitoringInvalidValueTypeError(self._name, value)
MonitoringInvalidValueTypeError: Metric "proc/cpu_percent" was given invalid value "0.0" (<type 'float'>).
,
Jul 14 2017
,
Jul 14 2017
Gauge metrics are integer gauges. They do not like float values.
,
Jul 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/0b634e9ed433d285995c33138ad95c9f93959028 commit 0b634e9ed433d285995c33138ad95c9f93959028 Author: Aviv Keshet <akeshet@chromium.org> Date: Fri Jul 14 22:40:29 2017 sysmon: round cpu % to integer BUG= chromium:743199 TEST=None Change-Id: I393cca2ce725ea0d641b1d3809e73a494c541699 Reviewed-on: https://chromium-review.googlesource.com/572360 Tested-by: Aviv Keshet <akeshet@chromium.org> Trybot-Ready: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> [modify] https://crrev.com/0b634e9ed433d285995c33138ad95c9f93959028/scripts/sysmon/proc_metrics.py
,
Jul 15 2017
Believed fixed |
||
►
Sign in to add a comment |
||
Comment 1 by akes...@chromium.org
, Jul 14 2017Correction, it's not crashlooping, but it is fail-looping, and the failure prevents other metrics in the loop from being emitted. Traceback (most recent call last): File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/loop.py", line 33, in loop_once self._callback() File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/mainlib.py", line 53, in __call__ proc_metrics.collect_proc_info() File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/proc_metrics.py", line 27, in collect_proc_info collector.collect() File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/proc_metrics.py", line 47, in collect self._flush() File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/proc_metrics.py", line 58, in _flush metric.flush() File "/usr/local/google/home/chromeos-test/chromiumos/chromite/venv/chromite/scripts/sysmon/proc_metrics.py", line 96, in flush _cpu_percent_metric.set(self._cpu_percent, fields=self._fields) File "/usr/local/google/home/chromeos-test/chromiumos/chromite/third_party/infra_libs/ts_mon/common/metrics.py", line 394, in set raise errors.MonitoringInvalidValueTypeError(self._name, value) MonitoringInvalidValueTypeError: Metric "proc/cpu_percent" was given invalid value "389.9" (<type 'float'>). DEBUG:chromite.scripts.sysmon.loop:Sleeping for 60, 60 remaining ERROR:chromite.scripts.sysmon.loop:Error during loop.