Update longevity_Tracker.py to work with AutoTest & Scheduler, and as part of server-side test. |
||||
Issue descriptionWhereas: 1) longevity_Tracker.py was designed to be run locally on a device, after the device had been manually set up (i.e., update OS, enroll to croste.tv, launch kiosk app, and enter user credentials for kiosk app), preventing automation of all these steps; 2) data sent to the perf dashboard was expected to overwrite previously sent performance data for the same version of CrOS and Chrome, preventing rapid regression detection; 3) control.longevity_x files used unique labels (e.g., longevity_1) to specify on what device to run the test, creating unnecessary proliferation of and redundancy in control files. 4) data for multiple cycles was not retained nor aggregated at the end of each cycle, preventing upload of multi-cycle test data to AFE and to the plx dashboard. The script and control files will be re-factored so that: 1) They shall work with the AutoTest scheduler. 2) Script can be called repeatedly (i.e., cycles) by a single instance of a server-side test (i.e., enterprise_LongevityTrackerServer.py). Note: said server-side test will first update device, enroll, launch app, and sign-in to app (if necessary), before calling longevity_Tracker.py script. 3) Script shall create a unique point_id for each run cycle, so that perf data from later cycles do not overwrite previous cycles. 4) Control file shall not bind the test instance to a specific device, but to a specific Kiosk App. 5) Script shall retain and aggregate performance data and metrics from all cycles for a service-side test instance, such that all data can be pulled up to AutoTest backend at the end of the server-side test, and can thereafter be pushed to the plx dashboard. Note: AutoTest backend will delete this data from device after each test run.
,
Jul 19 2016
Completed #3 and #4. Working on #5, which is a pre-requisite to #1 and #2.
,
Aug 8 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/40140993eb8539c8011a290380f78d317df38eb2 commit 40140993eb8539c8011a290380f78d317df38eb2 Author: Scott Cunningham <scunningham@chromium.org> Date: Tue Jul 19 17:18:02 2016 Update longevity_Tracker.py and control files. Update / refactor script and control files to provide the following: 1) They shall support interoperation with AutoTest backend and scheduler. 2) Script may be called repeatedly (i.e., run cycles) by an instance of the server-side test (i.e., enterprise_LongevityTrackerServer.py). Note: The server-side test will first update device, enroll, launch app, and sign-in to the app (if necessary), before calling longevity_Tracker.py script. 3) Script shall create a unique point_id for each run cycle, so that perf data uploaded from later cycles does not overwrite previous cycles. 4) Control file shall not bind the test instance to a specific device, but to a specific Kiosk App, based on App ID. 5) Script shall retain and aggregate performance data and metrics from all cycles for a service-side test instance, such that all data can be pulled up to AutoTest backend at the end of the server-side test, and can thereafter be pushed to the plx dashboard. Note: AutoTest backend will delete this data from device after each test run. 6) Script can still be run locally, to support testing/debugging. In this mode, single point performance data will still be uploaded to the performance dashboard. But since AFE backend is not involved, it shall not upload data to the backend, nor to plx dashaboard. BUG= chromium:629545 TEST=Run longevity_Tracker/control.riseplayer Change-Id: I4f1d3ebc0605a020a689b02096c98dcf23816c7e Reviewed-on: https://chromium-review.googlesource.com/361587 Commit-Ready: Dan Shi <dshi@google.com> Tested-by: Scott Cunningham <scunningham@chromium.org> Reviewed-by: Scott Cunningham <scunningham@chromium.org> Reviewed-by: Harpreet Grewal <harpreet@chromium.org> [modify] https://crrev.com/40140993eb8539c8011a290380f78d317df38eb2/client/site_tests/longevity_Tracker/longevity_Tracker.py [add] https://crrev.com/40140993eb8539c8011a290380f78d317df38eb2/client/site_tests/longevity_Tracker/control.chrome_sign_builder [modify] https://crrev.com/40140993eb8539c8011a290380f78d317df38eb2/site_utils/attribute_whitelist.txt [modify] https://crrev.com/40140993eb8539c8011a290380f78d317df38eb2/server/site_tests/enterprise_LongevityTrackerServer/control.riseplayer [modify] https://crrev.com/40140993eb8539c8011a290380f78d317df38eb2/test_suites/control.longevity [modify] https://crrev.com/40140993eb8539c8011a290380f78d317df38eb2/server/site_tests/enterprise_LongevityTrackerServer/enterprise_LongevityTrackerServer.py [delete] https://crrev.com/c55d41744cfce18e153a84eca1403d7dabe07c15/client/site_tests/longevity_Tracker/control.longevity_5 [delete] https://crrev.com/c55d41744cfce18e153a84eca1403d7dabe07c15/client/site_tests/longevity_Tracker/control.longevity_4 [delete] https://crrev.com/c55d41744cfce18e153a84eca1403d7dabe07c15/client/site_tests/longevity_Tracker/control.longevity_6 [delete] https://crrev.com/c55d41744cfce18e153a84eca1403d7dabe07c15/client/site_tests/longevity_Tracker/control.longevity_1 [add] https://crrev.com/40140993eb8539c8011a290380f78d317df38eb2/server/site_tests/enterprise_LongevityTrackerServer/control.chrome_sign_builder [delete] https://crrev.com/c55d41744cfce18e153a84eca1403d7dabe07c15/client/site_tests/longevity_Tracker/control.longevity_3 [delete] https://crrev.com/c55d41744cfce18e153a84eca1403d7dabe07c15/client/site_tests/longevity_Tracker/control.longevity_2 [add] https://crrev.com/40140993eb8539c8011a290380f78d317df38eb2/server/site_tests/enterprise_LongevityTrackerServer/control.stratosmedia [add] https://crrev.com/40140993eb8539c8011a290380f78d317df38eb2/client/site_tests/longevity_Tracker/control.riseplayer [add] https://crrev.com/40140993eb8539c8011a290380f78d317df38eb2/client/site_tests/longevity_Tracker/control.stratosmedia
,
Aug 8 2016
,
Sep 6 2016
,
Mar 3 2017
|
||||
►
Sign in to add a comment |
||||
Comment 1 by scunning...@chromium.org
, Jul 19 2016