New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 881186 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Oct 7
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Feature

Blocking:
issue 892999


Participants' hotlists:
ML-Service


Sign in to add a comment

ML Service: Basic metrics

Project Member Reported by amoylan@chromium.org, Sep 6

Issue description

We need to add basic metrics tracking to ML Service.
- Memory (current & peak over e.g. a day)
- CPU
- Events (e.g. bootstrap)

 
Project Member

Comment 1 by bugdroid1@chromium.org, Sep 11

Project Member

Comment 2 by bugdroid1@chromium.org, Sep 13

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/be34c8662f78e5076eef3cc42b725f60fe0bab79

commit be34c8662f78e5076eef3cc42b725f60fe0bab79
Author: Andrew Moylan <amoylan@chromium.org>
Date: Thu Sep 13 08:23:09 2018

ml: Add metrics dependency

Also stop using out-of-tree build which causes the wrong version of
platform2/ includes to be found

TEST=Check that setup_board --force then build_packages works
BUG= chromium:881186 

Change-Id: Ia53ba211cfd28690bfa2b53259a5193206ba613a
Reviewed-on: https://chromium-review.googlesource.com/1215502
Commit-Ready: Andrew Moylan <amoylan@chromium.org>
Tested-by: Andrew Moylan <amoylan@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>

[modify] https://crrev.com/be34c8662f78e5076eef3cc42b725f60fe0bab79/chromeos-base/ml/ml-9999.ebuild

Project Member

Comment 3 by bugdroid1@chromium.org, Sep 20

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/40ee4fc920ffda053130717750fccff69b2b8dab

commit 40ee4fc920ffda053130717750fccff69b2b8dab
Author: Andrew Moylan <amoylan@chromium.org>
Date: Thu Sep 20 12:15:19 2018

ml: Collect basic metrics

This CL adds some basic metrics collection ML Service. Future CLs will
expand metrics to cover events and resource-usage of the
TensorFlow-related operations (model loading, inference).

Metrics added:
- D-Bus -> Mojo bootstrap event
- Memory usage and peak memory usage per day
- CPU usage

Seccomp policy additions are for ProcessMetrics, UMA metrics, and
cumulative metrics backing storage (pread64/pwrite64).

CQ-DEPEND=CL:1215502
BUG= chromium:881186 
TEST=manual (checked chrome://histograms)

Change-Id: If6791a4ee0b0ed0e3cb7f57d8e59eeadd1d2c6ea
Reviewed-on: https://chromium-review.googlesource.com/1209022
Commit-Ready: Andrew Moylan <amoylan@chromium.org>
Tested-by: Andrew Moylan <amoylan@chromium.org>
Reviewed-by: Andrew Moylan <amoylan@chromium.org>

[modify] https://crrev.com/40ee4fc920ffda053130717750fccff69b2b8dab/ml/README.md
[modify] https://crrev.com/40ee4fc920ffda053130717750fccff69b2b8dab/ml/seccomp/ml_service-seccomp-arm.policy
[add] https://crrev.com/40ee4fc920ffda053130717750fccff69b2b8dab/ml/metrics.h
[modify] https://crrev.com/40ee4fc920ffda053130717750fccff69b2b8dab/ml/daemon.h
[modify] https://crrev.com/40ee4fc920ffda053130717750fccff69b2b8dab/ml/seccomp/ml_service-seccomp-amd64.policy
[modify] https://crrev.com/40ee4fc920ffda053130717750fccff69b2b8dab/ml/init/ml-service.conf
[add] https://crrev.com/40ee4fc920ffda053130717750fccff69b2b8dab/ml/metrics.cc
[modify] https://crrev.com/40ee4fc920ffda053130717750fccff69b2b8dab/ml/ml.gyp
[modify] https://crrev.com/40ee4fc920ffda053130717750fccff69b2b8dab/ml/daemon.cc

Project Member

Comment 4 by bugdroid1@chromium.org, Sep 24

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/03140b03f0821388708c52c1fc98d3f2091886f1

commit 03140b03f0821388708c52c1fc98d3f2091886f1
Author: Andrew Moylan <amoylan@chromium.org>
Date: Mon Sep 24 01:17:01 2018

Add UMA histograms for Chrome OS ML Service

Add histograms for the following metrics added in Chrome OS CL
crrev.com/c/1209022.
* Numerical histograms for CPU and RAM usage of ML Service
* Enum histogram for Mojo connection/disconnection events

Bug:  881186 
Change-Id: Id649c3bce16bf6ca65871feee1ab741372530669
Reviewed-on: https://chromium-review.googlesource.com/1237017
Reviewed-by: Jesse Doherty <jwd@chromium.org>
Commit-Queue: Andrew Moylan <amoylan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#593465}
[modify] https://crrev.com/03140b03f0821388708c52c1fc98d3f2091886f1/tools/metrics/histograms/enums.xml
[modify] https://crrev.com/03140b03f0821388708c52c1fc98d3f2091886f1/tools/metrics/histograms/histograms.xml

Remaining metrics needed are:
- For model load
- For graph executor creation
- For inference

For each of the above, record:
Events enum: requested, success, failures
Resource cost: cpu time
Resource cost: maybe delta of memory usage
Blocking: 892999
Labels: OS-Chrome
Status: Fixed (was: Started)
Summary: ML Service: Basic metrics (was: ML Service: Metrics)
Split off the remaining work into crbug.com/892999.

Sign in to add a comment