Create system health smoke test that run on CQ for every chromium commit |
|||
Issue descriptionWe should have a system health smoke test that run on CQ for every chromium commit to reduce the change of some chromium CL breaks the system health stories. Requirement: + This smoke test should cover all the system health stories. + It should be easy to disable some failing system health story to unblock CQ if some of them are failing in TOT or flaky. *Folks: the more we can share the stories between system health benchmarks, the easier it's to maintain this & reduce down the amount of runtime on CQ.
,
Jun 3 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0087d3d5b48bcda9d7173769369c344e33f205c7 commit 0087d3d5b48bcda9d7173769369c344e33f205c7 Author: nednguyen <nednguyen@google.com> Date: Fri Jun 03 12:59:30 2016 [tools/perf] Move system_health_stories to system_health.loading_stories module Rationale: we know that it's almost impossible to have 1 single system_health story set, hence the current direction is to create multiple stories set for key user actions. In the future, the page_sets/system_health_stories/ folder can contain these extra story_sets: loading scrolling & animation multi_tabs ... BUG= 615382 , 589726 Review-Url: https://codereview.chromium.org/2033693005 Cr-Commit-Position: refs/heads/master@{#397692} [add] https://crrev.com/0087d3d5b48bcda9d7173769369c344e33f205c7/tools/perf/page_sets/system_health/__init__.py [rename] https://crrev.com/0087d3d5b48bcda9d7173769369c344e33f205c7/tools/perf/page_sets/system_health/loading_stories.py
,
Jun 17 2016
,
Jun 30 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8095da730db4f644cc7691acca63f32b85536bce commit 8095da730db4f644cc7691acca63f32b85536bce Author: nednguyen <nednguyen@google.com> Date: Thu Jun 30 13:24:28 2016 [tools/perf] Add system_health_smoke_test that run all system health stories BUG= 615382 Review-Url: https://codereview.chromium.org/2110653002 Cr-Commit-Position: refs/heads/master@{#403163} [add] https://crrev.com/8095da730db4f644cc7691acca63f32b85536bce/tools/perf/benchmarks/system_health_smoke_test.py
,
Jun 30 2016
,
Jun 30 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/40e2f62217341b98d4fe394dc51dc9fde2d80983 commit 40e2f62217341b98d4fe394dc51dc9fde2d80983 Author: kelvinp <kelvinp@chromium.org> Date: Thu Jun 30 16:51:37 2016 Revert of [tools/perf] Add system_health_smoke_test that run all system health stories (patchset #8 id:160001 of https://codereview.chromium.org/2110653002/ ) Reason for revert: This change is failing on Android Tests bots. https://build.chromium.org/p/chromium.linux/builders/Android%20Tests%20%28dbg%29/builds/35105/steps/telemetry_perf_unittests Original issue's description: > [tools/perf] Add system_health_smoke_test that run all system health stories > > BUG= 615382 > > Committed: https://crrev.com/8095da730db4f644cc7691acca63f32b85536bce > Cr-Commit-Position: refs/heads/master@{#403163} TBR=petrcermak@chromium.org,perezju@chromium.org,nednguyen@chromium.org,nednguyen@google.com # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 615382 Review-Url: https://codereview.chromium.org/2117533002 Cr-Commit-Position: refs/heads/master@{#403199} [delete] https://crrev.com/da47f994b774d716523f119b9b655ad0ebb22765/tools/perf/benchmarks/system_health_smoke_test.py
,
Jun 30 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ea61739c46e564f0c6651e376dd4d8d9e0a2d0df commit ea61739c46e564f0c6651e376dd4d8d9e0a2d0df Author: nednguyen <nednguyen@google.com> Date: Thu Jun 30 19:05:42 2016 [tools/perf] Add system_health_smoke_test that run all system health stories (Reland) *** NOTE TO SHERRIF *** If there are failing tests in form of benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.* please do not revert this CL but instead disabling failing test by adding them into the _DISABLE_TESTS list in tools/perf/benchmarks/system_health_smoke_test.py 1st patch from issue 2110653002 at patchset 160001 (http://crrev.com/2110653002#ps160001) 2nd patch disable more tests BUG= 615382 TBR=petrcermak@chromium.org Review-Url: https://codereview.chromium.org/2115623002 Cr-Commit-Position: refs/heads/master@{#403237} [add] https://crrev.com/ea61739c46e564f0c6651e376dd4d8d9e0a2d0df/tools/perf/benchmarks/system_health_smoke_test.py
,
Jul 1 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/055373dbdafcd9bfceb0264df2554005cb22f536 commit 055373dbdafcd9bfceb0264df2554005cb22f536 Author: petrcermak <petrcermak@chromium.org> Date: Fri Jul 01 16:55:47 2016 [system-health] Re-enable all passing smoke tests on the CQ All tests except for the following 3 passed in the most recent CQ dry runs: * load:tools:dropbox (crbug.com/624474) * load:media:dailymotion ( crbug.com/624607 ) * load:games:bubbles (crbug.com/624701) *** NOTE TO SHERRIF *** If any of the benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.* tests fails, please do not revert this CL but instead disable only the failing tests by adding them to the _DISABLE_TESTS list in tools/perf/benchmarks/system_health_smoke_test.py. Thanks! BUG=624474,624587, 624607 ,624701, 624840 , 615382 ,589726 TBR=nednguyen@google.com CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq Review-Url: https://codereview.chromium.org/2120443002 Cr-Commit-Position: refs/heads/master@{#403470} [modify] https://crrev.com/055373dbdafcd9bfceb0264df2554005cb22f536/tools/perf/benchmarks/system_health_smoke_test.py
,
Jul 12 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4828a631a7c8914d3c175c63c78fbdcee75da9b0 commit 4828a631a7c8914d3c175c63c78fbdcee75da9b0 Author: nednguyen <nednguyen@google.com> Date: Tue Jul 12 17:10:00 2016 [tools/perf] Remove dead code in system_health_smoke_test.py NOTRY=true BUG= 615382 Review-Url: https://codereview.chromium.org/2140133002 Cr-Commit-Position: refs/heads/master@{#404813} [modify] https://crrev.com/4828a631a7c8914d3c175c63c78fbdcee75da9b0/tools/perf/benchmarks/system_health_smoke_test.py |
|||
►
Sign in to add a comment |
|||
Comment 1 by sheriffbot@chromium.org
, Jun 2 2016