The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/bbcc1c9f3477a602e485c17d65c44aea900903a9

commit bbcc1c9f3477a602e485c17d65c44aea900903a9
Author: Satya Tangirala <satyat@google.com>
Date: Fri Aug 11 03:24:12 2017

autotest: Resource Monitor Framework.

Framework to monitor CPU and memory use. Saves top output periodically
on the client and extracts cpu/memory usage metrics when asked to do so.
Also, saves the raw top output on the server.

BUG=chromium:688160
TEST=resource_monitor_unittest.py

Change-Id: I613f6807bf85a08d6292455eb5794cd5988a2881
Reviewed-on: https://chromium-review.googlesource.com/576667
Commit-Ready: Satya Tangirala <satyat@google.com>
Tested-by: Satya Tangirala <satyat@google.com>
Reviewed-by: Abhishek Bhardwaj <abhishekbh@google.com>

[add] https://crrev.com/bbcc1c9f3477a602e485c17d65c44aea900903a9/server/cros/resource_monitor_unittest.py
[add] https://crrev.com/bbcc1c9f3477a602e485c17d65c44aea900903a9/server/cros/resource_monitor.py
[add] https://crrev.com/bbcc1c9f3477a602e485c17d65c44aea900903a9/server/cros/res_resource_monitor/top_whitespace_ridden.txt
[add] https://crrev.com/bbcc1c9f3477a602e485c17d65c44aea900903a9/server/cros/res_resource_monitor/top_test_data_ans.csv
[add] https://crrev.com/bbcc1c9f3477a602e485c17d65c44aea900903a9/server/cros/res_resource_monitor/top_field_order_changed_ans.csv
[add] https://crrev.com/bbcc1c9f3477a602e485c17d65c44aea900903a9/server/cros/res_resource_monitor/top_field_order_changed.txt
[add] https://crrev.com/bbcc1c9f3477a602e485c17d65c44aea900903a9/server/cros/res_resource_monitor/top_whitespace_ridden_ans.csv
[add] https://crrev.com/bbcc1c9f3477a602e485c17d65c44aea900903a9/server/cros/res_resource_monitor/top_test_data.txt

 Issue 856298  has been merged into this issue.

IIRC, we never settled on what we wanted to add in the netperf tests. 
satyat@ merged this CL, then Harpreet (or someone else) pointed out that autotest already has similar functionality. They were probably pointing to get_cpu_usage()
function in client/bin/utils.py.

Will let Alex make the call on which works best for us. 

I vaguely recall Alex using a netperf command line option to get CPU utilization stats. Alex, reading through the CPU utilization part in the netperf manual (https://hewlettpackard.github.io/netperf/doc/netperf.html#CPU-Utilization)
I am not sure if there's a reliable way to get this, or if we build netperf with that option enabled in --configure. 
"The /proc/stat mechanism under Linux is in what the author would consider an “uncertain” category as it appears to be statistical, which may also have issues with time spent processing interrupts.
In summary, be sure to “sanity-check” the CPU utilization figures with other mechanisms. However, platform tools such as top, vmstat or mpstat are often based on the same mechanisms used by netperf."

If you do go through this route, please double-check utilization numbers with
"turbostat --quiet" output.

If netperf is run with the '-cC' options, we can get both server and client-side CPU utilization. Unfortunately, netperf only provides a single percentage per machine, which doesn't offer much insight when a system has 4 cores. My solution was to simply get the /proc/stat values of the machines through autotest right before and after running netperf, such that we could calculate the utilization values of each core separately. Although it seemed like netperf could provide more accurate numbers, as it collects the initial /proc/stat values after initialization, the difference on average between the two methods was less than a percentage point.

That being said, I will play around with different methods and see the discrepancies between values.