Clean up system health orderfile tests |
|||
Issue descriptionThis has been failing in local testing, and I just discovered on the bot that background:tools:gmail is disabled on Nexus_5 due to crbug.com/784400. As my clients are all currently occupied in testing I'm creating this bug so that I don't drop this issue.
,
Nov 12
And load:tools:gmail. (load:tools:gmail is disabled on Android due to crbug.com/657433)
,
Nov 12
load:news:cnn:2018 seems to be running okay on the bots, but is failing for me. The test in #1 is background:tools:gmail.
,
Nov 13
browse:social:facebook_infinite_scroll:2018 and browse:social:facebook due to crbug.com/780779.
,
Nov 15
Side note: this could be (semi) automated by parsing tools/perf/expectations.config. But I'm not sure about that. For now updating manually may be okay. Additional tests from that file: background:news:nytimes background:tools:gmail browse:chrome:newtab browse:chrome:omnibox browse:news:cnn browse:news:cnn:2018 browse:news:globo browse:news:toi browse:shopping:avito browse:shopping:flipkart browse:social:facebook browse:social:facebook_infinite_scroll browse:social:pinterest_infinite_scroll browse:social:tumblr_infinite_scroll browse:tech:discourse_infinite_scroll:2018 load:media:soundcloud load:news:cnn load:news:washingtonpost load:tools:drive load:tools:gmail long_running:tools:gmail-background long_running:tools:gmail-foreground
,
Nov 22
Possible failing tests found during testing:
[ FAILED ] orderfile_generation.testing/browse:shopping:lazada@{'case': 'browse', 'group': 'shopping'}
[ FAILED ] orderfile_generation.testing/load:games:bubbles@{'case': 'load', 'group': 'games'}
,
Nov 22
The two tests in #6 just seem to be flaky, they passed the second time around.
,
Nov 27
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/bcac190e337ca005d030fb315549ff85785cf4cc commit bcac190e337ca005d030fb315549ff85785cf4cc Author: Matthew Cary <mattcary@chromium.org> Date: Tue Nov 27 16:59:59 2018 Orderfile: update orderfile test suite. The orderfile generation and validation benchmark, based on system_health, does not parse tools/perf/expectations.config and instead manually blacklists failing tests. This CL updates that blacklist. As the blacklist has grown much larger, some assumptions in the tests about how many variations could be fit have been broken, requiring the test to be updated as well. Bug: 904364 Change-Id: I342ed77fb07b3f562eef9fd9e16600a7c424b2eb Reviewed-on: https://chromium-review.googlesource.com/c/1350974 Reviewed-by: Egor Pasko <pasko@chromium.org> Reviewed-by: Benoit L <lizeb@chromium.org> Commit-Queue: Matthew Cary <mattcary@chromium.org> Cr-Commit-Position: refs/heads/master@{#611149} [modify] https://crrev.com/bcac190e337ca005d030fb315549ff85785cf4cc/tools/perf/contrib/orderfile/orderfile.py [modify] https://crrev.com/bcac190e337ca005d030fb315549ff85785cf4cc/tools/perf/contrib/orderfile/orderfile_unittest.py
,
Dec 3
|
|||
►
Sign in to add a comment |
|||
Comment 1 by mattcary@chromium.org
, Nov 12