SettingsAdvancedPageBrowserTest is flaky. |
||||||
Issue descriptionSee https://codereview.chromium.org/2756863002/#ps220001 It failed two times, but the third time was successful. [ RUN ] SettingsAdvancedPageBrowserTest.Load [9412:9412:0412/214538.173438:WARNING:chrome_browser_main_chromeos.cc(393)] Running as stub user with profile dir: test-user [ RUN ] SettingsAdvancedPageBrowserTest.Load [4984:4984:0412/215821.614340:WARNING:chrome_browser_main_chromeos.cc(393)] Running as stub user with profile dir: test-user [4984:4984:0412/215822.150757:WARNING:audio_manager.cc(293)] Multiple instances of AudioManager detected [4984:4984:0412/215822.150774:WARNING:audio_manager.cc(254)] Multiple instances of AudioManager detected [4984:4984:0412/215822.179642:ERROR:logging_chrome.cc(175)] Unable to create symlink /b/s/w/itIL8OqK/.org.chromium.Chromium.TtIINc/ds1mkvj/test-user/chrome_debug.log pointing at /b/s/w/itIL8OqK/.org.chromium.Chromium.TtIINc/ds1mkvj/test-user/chrome_debug_20170412-215822.log: No such file or directory [4984:5005:0412/215822.318927:WARNING:freezer_cgroup_process_manager.cc(61)] Cgroup freezer does not exist or is not writable. Unable to freeze renderer processes. [4984:4984:0412/215822.369203:ERROR:remote_text_input_client.cc(140)] Not implemented reached in virtual void RemoteTextInputClient::OnInputMethodChanged() [4984:4984:0412/215822.369223:ERROR:remote_text_input_client.cc(97)] Not implemented reached in virtual bool RemoteTextInputClient::HasCompositionText() const [4984:4984:0412/215822.369228:ERROR:remote_text_input_client.cc(103)] Not implemented reached in virtual bool RemoteTextInputClient::GetTextRange(gfx::Range *) const [4984:4984:0412/215822.430984:ERROR:render_widget_host_view_aura.cc(1245)] Not implemented reached in virtual base::i18n::TextDirection content::RenderWidgetHostViewAura::GetTextDirection() const [4984:4984:0412/215822.585058:ERROR:render_widget_host_view_aura.cc(1245)] Not implemented reached in virtual base::i18n::TextDirection content::RenderWidgetHostViewAura::GetTextDirection() const [4984:4984:0412/215822.806822:WARNING:CONSOLE(112)] "This file is deprecated. Please use `iron-flex-layout/iron-flex-layout-classes.html`, and one of the specific dom-modules instead", source: chrome://md-settings/crisper.js (112) [4984:4984:0412/215822.806990:WARNING:CONSOLE(112)] "This file is deprecated. Please use `iron-flex-layout/iron-flex-layout-classes.html`, and one of the specific dom-modules instead", source: chrome://md-settings/crisper.js (112) [4984:4984:0412/215822.957965:WARNING:CONSOLE(0)] "/deep/ combinator is deprecated and will be a no-op in M60, around August 2017. See https://www.chromestatus.com/features/4964279606312960 for more details.", source: (0) [4984:4984:0412/215822.990065:INFO:CONSOLE(1221)] "Running TestCase SettingsAdvancedPageBrowserTest.MAYBE_Load", source: test_api.js (1221) [4984:4984:0412/215823.006952:ERROR:CONSOLE(48)] "Mocha test failed: SettingsPage advanced pages TypeError: Cannot read property 'shadowRoot' of null at SettingsAdvancedPageBrowserTest.getSection (settings_page_browsertest.js:91:24) at Context.<anonymous> (advanced_page_browsertest.js:51:28) ", source: mocha_adapter.js (48) [4984:4984:0412/215823.008627:INFO:CONSOLE(125)] "Page load time: 409 ms", source: polymer_browser_test_base.js (125) [4984:4984:0412/215823.008774:INFO:CONSOLE(127)] "Test run time: 16 ms", source: polymer_browser_test_base.js (127) [4984:4984:0412/215823.008953:INFO:CONSOLE(129)] "Total time: 425 ms", source: polymer_browser_test_base.js (129) [4984:4984:0412/215823.009001:ERROR:web_ui_test_handler.cc(76)] Test Errors: 1/2 tests had failed assertions. [4984:4984:0412/215823.009016:ERROR:web_ui_browser_test.cc(463)] CONDITION FAILURE: encountered javascript console error(s): [4984:4984:0412/215823.009019:ERROR:web_ui_browser_test.cc(465)] JS ERROR: '[4984:4984:0412/215823.006952:ERROR:CONSOLE(48)] "Mocha test failed: SettingsPage advanced pages TypeError: Cannot read property 'shadowRoot' of null at SettingsAdvancedPageBrowserTest.getSection (settings_page_browsertest.js:91:24) at Context.<anonymous> (advanced_page_browsertest.js:51:28) ", source: mocha_adapter.js (48) ' [4984:4984:0412/215823.009034:ERROR:web_ui_browser_test.cc(467)] JS call assumed failed, because JS console error(s) found. gen/chrome/test/data/webui/settings/advanced_page_browsertest-gen.cc:44: Failure Value of: RunJavascriptTestF( true, "SettingsAdvancedPageBrowserTest", "MAYBE_Load") Actual: false Expected: true [ FAILED ] SettingsAdvancedPageBrowserTest.Load, where TypeParam = and GetParam() = (1520 ms) [ RUN ] SettingsAdvancedPageBrowserTest.Load
,
Apr 26 2017
@stevenjb: I'll take a look. Having said that, I think that our "integration" tests have been fairly flaky and perhaps we should reconsider whether we should keep them, or just stick with "unit-tests". There was some more extensive discussion about this at https://bugs.chromium.org/p/chromium/issues/detail?id=641400#c13 (see discussion from that comment and below).
,
Apr 26 2017
,
Apr 26 2017
,
Apr 27 2017
I spent quite a bit of effort trying to reproduce the flakiness without success (on ChromeOS for Linux). @stevenjb: The lazy module is loaded before the test even begins running at [1]. I tried adding an artificial delay until the lazy module is loaded, and the test successfully waits for it and then runs, so that does not seem to be an issue. Given the above, and that I also don't see any flakes at [2], I don't know what else to try at this point (perhaps the flakiness only happens at linux_chromium_chromeos_ozone_rel_ng?). I am deferring any further investigation until this flakiness starts happening on ToT and not just on tryjobs. [1] https://cs.chromium.org/chromium/src/chrome/test/data/webui/settings/settings_page_browsertest.js?q&l=78 [2] https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=SettingsAdvancedPageBrowserTest
,
Apr 27 2017
,
Apr 27 2017
There are no instances in the last 200 runs on https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_chromeos_ozone_rel_ng/ We can keep an eye out for this, but I would be fine with resolving this as WontFix until / unless it shows up again.
,
Apr 27 2017
Marking as WontFix for now. Let's keep an eye on it. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by steve...@chromium.org
, Apr 26 2017Status: Assigned (was: Untriaged)