"RequestCoordinatorFactoryTest.BuildRequestCoordinator" is flaky |
||||||||||
Issue description"RequestCoordinatorFactoryTest.BuildRequestCoordinator" is flaky. This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label. We have detected 12 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyQAsSBUZsYWtlIjVSZXF1ZXN0Q29vcmRpbmF0b3JGYWN0b3J5VGVzdC5CdWlsZFJlcXVlc3RDb29yZGluYXRvcgw. Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
,
Jan 10 2017
I believe https://codereview.chromium.org/2624523003 is the cause, given that it ladned but also had this failure in it and touched this code.
,
Jan 10 2017
looks like it was reverted though. please unflake the test before relanding it
,
Jan 10 2017
I wonder if this was caused by https://codereview.chromium.org/2609713002/ instead. That CL landed around 18:50 UTC, and all but one of the flakes started around 19:35 UTC The flakes appear to have the following error: [ERROR:connection.cc(1947)] OfflinePageMetadata sqlite error 1802, errno 0: disk I/O error, sql: CREATE TABLE IF NOT EXISTS offlinepages_v1(offline_id INTEGER PRIMARY KEY NOT NULL, creation_time INTEGER NOT NULL, file_size INTEGER NOT NULL, last_access_time INTEGER NOT NULL, access_count INTEGER NOT NULL, client_namespace VARCHAR NOT NULL, client_id VARCHAR NOT NULL, online_url VARCHAR NOT NULL, file_path VARCHAR NOT NULL, title VARCHAR NOT NULL DEFAULT '', original_url VARCHAR NOT NULL DEFAULT '') [FATAL:connection.cc(1962)] disk I/O error which may be something related to no longer delaying the OfflinePageModel construction when creating the test RequestCoordinator...
,
Jan 10 2017
Prognosis is that the RequestCoordinatorFactoryTest doesn't flush the i/o thread (only flushes browser thread), so causing the flake depending on which order the threads terminate. Will test locally
,
Jan 11 2017
This test is still flakey, and I seem to be hitting it on every run!
,
Jan 11 2017
This is blocking the entire Chromium commit queue for changes which affect the build, because these aren't retried without the patch. Examples: https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_swarming_rel/builds/97761 https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_swarming_rel/builds/97738 Please disable this test right away while the flakiness is being investigated. Thanks.
,
Jan 11 2017
,
Jan 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/95ae77193e1dbd5a7e6048095d0100597e626e83 commit 95ae77193e1dbd5a7e6048095d0100597e626e83 Author: Philip Jägenstedt <foolip@chromium.org> Date: Wed Jan 11 15:46:56 2017 Disable RequestCoordinatorFactoryTest.BuildRequestCoordinator (flaky) BUG= 679844 TBR=petewil@chromium.org Review-Url: https://codereview.chromium.org/2622303002 . Cr-Commit-Position: refs/heads/master@{#442911} [modify] https://crrev.com/95ae77193e1dbd5a7e6048095d0100597e626e83/chrome/browser/android/offline_pages/request_coordinator_factory_unittest.cc
,
Jan 11 2017
,
Jan 11 2017
Detected 180 new flakes for test/step "RequestCoordinatorFactoryTest.BuildRequestCoordinator". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyQAsSBUZsYWtlIjVSZXF1ZXN0Q29vcmRpbmF0b3JGYWN0b3J5VGVzdC5CdWlsZFJlcXVlc3RDb29yZGluYXRvcgw. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jan 12 2017
,
Jan 14 2017
After a few days of debugging and trying things, we are going to remove the test because... 1. The test as it currently stands isn't adding much value in the first place 2. After many efforts described below, the test is still flaky 3. I have yet to see another keyservice factory with a unittest 4. This investigation is turning to be too long with little return value Steps taken to try to solve the flake: 1. Since the issue was always a disk I/O error, tried flushing all the threads, including the IO thread. Turned into consistently failing. 2. Added a DependsOn between the RequestCoordinatorFactory and the OfflinePageModelFactory, which was implicit before but now more explicit. Did nothing regarding the flake 3. Rather than extending from a render harness, create our own thread bundle, profile manager and testing profile. This had marginal effect. Rather than having just a disk I/O error, we now also have "unable to open database file"... but we also have a 50% flake rate. I will send CL to include the DependsOn change and to delete this test shortly.
,
Jan 14 2017
,
Jan 18 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f109c99563e6efe928faee2ba68402f1a3cc06f3 commit f109c99563e6efe928faee2ba68402f1a3cc06f3 Author: chili <chili@chromium.org> Date: Wed Jan 18 05:36:13 2017 [Offline Pages] Add explicit DependsOn clause to request coordinator factory and delete a flaky test BUG= 679844 Review-Url: https://codereview.chromium.org/2635633002 Cr-Commit-Position: refs/heads/master@{#444285} [modify] https://crrev.com/f109c99563e6efe928faee2ba68402f1a3cc06f3/chrome/browser/android/offline_pages/request_coordinator_factory.cc [delete] https://crrev.com/1f89b6d7e48c06aec8f4e2c0723afa229045a402/chrome/browser/android/offline_pages/request_coordinator_factory_unittest.cc [modify] https://crrev.com/f109c99563e6efe928faee2ba68402f1a3cc06f3/chrome/test/BUILD.gn
,
Jan 19 2017
+fgorski, who expressed interest in looking at the broken unittest for science |
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by jam@chromium.org
, Jan 10 2017Status: Assigned (was: Untriaged)