We want to create a semi-automated process where a tester can start video playback on a DUT for overnight/overweekend runs and also start a script in the background that collects statistics about system performance, for instance:
- CPU utilization
- Memory consumption
- Power consumption
Currently, we test videos over long durations but our verification is simply to check for the absence of crashes. The hope is that added profiling will provide richer data and better bugs.
We are OK with moving devices to dev mode or test images for this. We want to start with stress testing non-DRM video or the RTC loopback test pages and later include DRM testing by whitelisting a few of the DUTs.
The desired process would look like this:
1. Select DUT. OK to be in Dev/Test mode.
2. Install build to test.
3. Log in.
4. Start data collection in shell
- Specify which dimensions to record.
- Specify sampling frequency
5. Navigate to video in webapp/play store app and start playback in loop mode.
6. Leave overnight/weekend.
7. Come back and stop video playback.
8. Stop script.
9. Analyze log using simple graphing tools and file bugs if needed
Comment 1 by posciak@chromium.org
, Jan 11 2018