I think this needs to be among the first things we do in rollout. Possibly even before we try to roll out to prod. I got here because I did some sniffing around and was unpleasantly surprised to see that we don't have any metrics instrumentation from the container_pool service. We cannot maintain services or daemons that are not instrumented.

Step 1 of this is to add a SetupTsMonGlobalState context setup in the entry point for container_pool. I believe that is in the lxc_pool_service script. And a tick increment counter in container_pool/service.py 's tick loop.

-> kenobi is going to get the ball rolling

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/9b18c715f9b5b617495ca409cc8c073698307ef7

commit 9b18c715f9b5b617495ca409cc8c073698307ef7
Author: Ben Kwa <kenobi@google.com>
Date: Fri Dec 15 07:57:29 2017

[autotest] Enable container pool metrics.

Step 0: Set up a metrics context, and add a basic tick counter for
checking service aliveness.

BUG= chromium:772142 
TEST=service_unittest.py
TEST=Run lxc_pool_service.py locally; verify that the service starts,
     runs, and shuts down normally.

Change-Id: I528b89385efc608b2ff33a306bd20d49e3360c15
Reviewed-on: https://chromium-review.googlesource.com/825954
Commit-Ready: Jacob Kopczynski <jkop@chromium.org>
Tested-by: Ben Kwa <kenobi@chromium.org>
Reviewed-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Jacob Kopczynski <jkop@chromium.org>

[modify] https://crrev.com/9b18c715f9b5b617495ca409cc8c073698307ef7/site_utils/lxc/container_pool/service.py
[modify] https://crrev.com/9b18c715f9b5b617495ca409cc8c073698307ef7/site_utils/lxc_pool_service.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/18577ae28c8b4d5e9b279fd597eace3191a26f61

commit 18577ae28c8b4d5e9b279fd597eace3191a26f61
Author: Jacob Kopczynski <jkop@google.com>
Date: Wed Dec 20 20:53:54 2017

container_pool: Add metrics for service

BUG= chromium:772142 
TEST=None

Change-Id: Ifcb22ac279da77126e027ec966da23d362a81c19

[modify] https://crrev.com/18577ae28c8b4d5e9b279fd597eace3191a26f61/site_utils/lxc/container_pool/service.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/4647712c1694e6d98fc1c8eddb228a3ee0c74ccf

commit 4647712c1694e6d98fc1c8eddb228a3ee0c74ccf
Author: Jacob Kopczynski <jkop@google.com>
Date: Wed Dec 20 20:53:49 2017

container_pool: Add metrics for pool itself

BUG= chromium:772142 
TEST=pool_unittest

Change-Id: I83ddf6d6a6890b66e160316193d49cf1ad32fe29

[modify] https://crrev.com/4647712c1694e6d98fc1c8eddb228a3ee0c74ccf/site_utils/lxc/constants.py
[modify] https://crrev.com/4647712c1694e6d98fc1c8eddb228a3ee0c74ccf/site_utils/lxc/container_pool/pool.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/c0be8249c92994acefd2461dd0eb848e7d1b1d67

commit c0be8249c92994acefd2461dd0eb848e7d1b1d67
Author: Jacob Kopczynski <jkop@google.com>
Date: Thu Dec 21 21:14:25 2017

container_pool: Add metrics for container_bucket

BUG= chromium:772142 
TEST=None

Change-Id: I755540c7b41d6e5ac92f9da2f4b7cdbc5387f60c

[modify] https://crrev.com/c0be8249c92994acefd2461dd0eb848e7d1b1d67/site_utils/lxc/container_bucket.py