ProfileAttributesStorageTest.LoadAvatarFromDiskTest failing consistently on linux_chromium_asan_rel_ng, blocking CQ |
|||||||||
Issue descriptionThis is causing unit_tests to fail. Example failures: https://ci.chromium.org/buildbot/tryserver.chromium.linux/linux_chromium_asan_rel_ng/511124 https://ci.chromium.org/buildbot/tryserver.chromium.linux/linux_chromium_asan_rel_ng/511116 https://ci.chromium.org/buildbot/tryserver.chromium.linux/linux_chromium_asan_rel_ng/511105 https://ci.chromium.org/buildbot/tryserver.chromium.linux/linux_chromium_asan_rel_ng/511101 https://ci.chromium.org/buildbot/tryserver.chromium.linux/linux_chromium_asan_rel_ng/511100 https://ci.chromium.org/buildbot/tryserver.chromium.linux/linux_chromium_asan_rel_ng/511098 Failure text: [ RUN ] ProfileAttributesStorageTest.LoadAvatarFromDiskTest ../../chrome/browser/profiles/profile_attributes_storage_unittest.cc:808: Failure Value of: base::PathExists(icon_path) Actual: false Expected: true [ FAILED ] ProfileAttributesStorageTest.LoadAvatarFromDiskTest (25 ms) This is preventing jobs from making it through the CQ. Marking P1.
,
Dec 14 2017
The test must be *really* flaky ... in some cases it is failing on the retries, but in other cases it isn't.
,
Dec 14 2017
Strange. My last check in was about 2 months ago (was busy with kids), and the failure happened only for one day. Is the Linux ASAN bot the only one with flakiness? I suspect something is wrong with a recent check in. Is it possible to do a bisect? BTW, do you agree to disable the test temporarily?
,
Dec 14 2017
It turns out that the flakiness started today and is visible on this waterfall bot: https://ci.chromium.org/buildbot/chromium.memory/Linux%20ASan%20LSan%20Tests%20%281%29/?limit=200 Here was the first failing build: https://ci.chromium.org/buildbot/chromium.memory/Linux%20ASan%20LSan%20Tests%20%281%29/40854 lwchkg@, please take ownership of this – if necessary, disable the test until the root cause can be isolated. I glanced through the CLs of the failing build and the few builds leading up to it and didn't see any obvious suspects.
,
Dec 14 2017
,
Dec 14 2017
The test fails because a file writing operation failed. It looks like a problem in the testing infrastructure. grunell@: Can you please help me to pass the bug to the people managing the linux asan bot?
,
Dec 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c9f5613bb33ced5aaa9244004770ee8e6f3b6306 commit c9f5613bb33ced5aaa9244004770ee8e6f3b6306 Author: Henrik Grunell <grunell@chromium.org> Date: Thu Dec 14 12:52:33 2017 Disable flaky ProfileAttributesStorageTest.LoadAvatarFromDiskTest on Linux ASAN. TBR= lwchkg@chromium.org Bug: 794821 Change-Id: Ie3fc909acee14e5bf8b8b80e8c8e0e2b3bf45945 Reviewed-on: https://chromium-review.googlesource.com/827006 Reviewed-by: Henrik Grunell <grunell@chromium.org> Reviewed-by: Mihai Sardarescu <msarda@chromium.org> Commit-Queue: Henrik Grunell <grunell@chromium.org> Cr-Commit-Position: refs/heads/master@{#524048} [modify] https://crrev.com/c9f5613bb33ced5aaa9244004770ee8e6f3b6306/chrome/browser/profiles/profile_attributes_storage_unittest.cc
,
Dec 14 2017
,
Dec 14 2017
Infra troopers can pick it up.
,
Dec 14 2017
lwchkg@chromium.org: what is the specific file writing operation (relevant log messages, other info like what is it trying to write to) so infra trooper can investigate? I am not familiar with the implementation details of these tests or what they are testing.
,
Dec 14 2017
https://cs.chromium.org/chromium/src/chrome/browser/profiles/profile_attributes_storage_unittest.cc?type=cs&q=ProfileAttributesStorageTest&sq=package:chromium&l=815 The test starts by writing a string literal (mock icon) to a predefined path in the user data directory (forgot what is it, needs time to search). That operation has failed so the main part of the test did not start. I am quite surprised that another test ( https://cs.chromium.org/chromium/src/chrome/browser/profiles/profile_attributes_storage_unittest.cc?type=cs&q=profile_attributes_storage_unittest:691&sq=package:chromium&l=691 ) has written another file to the same directory but did not become flaky. Maybe the two tests are executed in different shards, and only one of the shards failed.
,
Dec 14 2017
seanmcculllough@ Please check #11. I've forgotten to mention you there. :-(
,
Dec 15 2017
Few concerns: 1. This is probably unrelated to the flakiness, but https://cs.chromium.org/chromium/src/chrome/browser/profiles/profile_attributes_storage_unittest.cc?type=cs&q=ProfileAttributesStorageTest&sq=package:chromium&l=814 writes only 8 bytes to the file, because sizeof(bitmap) is 8 (with 'bitmap' being a pointer). You should change "const char* bitmap" to "const char bitmap[]". 2. I don't really know how we execute tests, but is it possible that DownloadHighResAvatarTest and LoadAvatarFromDiskTest are sometimes executed in parallel? If so, they race for the same file path. Consider changing kIconIndex to be different in different tests.
,
Dec 15 2017
Issue 795068 has been merged into this issue.
,
Dec 15 2017
@vadimsh Thanks for pointing out (1). I'll make a CL ASAP. For (2), I'm still investigating. What I'm sure is that DownloadHighResAvatarTest (shard #2) and LoadAvatarFromDiskTest (shard #0) are executed in two different shards. Do you know whether it is possible for the different shards to access the same file?
,
Dec 20 2017
(removing from trooper queue, this doesn't look to be caused by infrastructure)
,
Dec 21 2017
,
Jan 14 2018
I thought the test writes the icon to the user_data_dir of the TestingProfileManager, but it turns out to write to the real user_data_dir, i.e. <settings_dir>/Chromium/User Data/Avatars. So the race in #13 appears to be possible. I'll try to fix by adding a path service override for TestingProfileManager.
,
Jan 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6c602d4eb8a37e78a3ae81428b8981e58af807dd commit 6c602d4eb8a37e78a3ae81428b8981e58af807dd Author: WC Leung <lwchkg@chromium.org> Date: Tue Jan 23 22:32:18 2018 Bug fixes of ProfileAttributesStorageTest.LoadAvatarFromDiskTest The captioned test was disabled in Linux ASAN because of flakiness. Two possible causes were identified: - Possible race between DownloadHighResAvatarTest and LoadAvatarFromDiskTest because the path returned by GetPathOfHighResAvatarAtIndex is not inside the unique temp directory. - The directory holding the test icon is not created before running base::WriteFile. This CL fixes the above issues, and changed "const char* bitmap" to "const char bitmap[]" because of issue 795518 . Bug: 794821 , 795518 Change-Id: I0deb2c8acddea86fb94ca27a363afa132f0ac6fc Reviewed-on: https://chromium-review.googlesource.com/868411 Reviewed-by: Lei Zhang <thestig@chromium.org> Reviewed-by: Bernhard Bauer <bauerb@chromium.org> Reviewed-by: Scott Violet <sky@chromium.org> Commit-Queue: WC Leung <lwchkg@chromium.org> Cr-Commit-Position: refs/heads/master@{#531370} [modify] https://crrev.com/6c602d4eb8a37e78a3ae81428b8981e58af807dd/chrome/browser/component_updater/supervised_user_whitelist_installer_unittest.cc [modify] https://crrev.com/6c602d4eb8a37e78a3ae81428b8981e58af807dd/chrome/browser/printing/cloud_print/test/cloud_print_proxy_process_browsertest.cc [modify] https://crrev.com/6c602d4eb8a37e78a3ae81428b8981e58af807dd/chrome/browser/profiles/profile_attributes_storage_unittest.cc [modify] https://crrev.com/6c602d4eb8a37e78a3ae81428b8981e58af807dd/chrome/browser/profiles/profile_info_cache_unittest.cc [modify] https://crrev.com/6c602d4eb8a37e78a3ae81428b8981e58af807dd/chrome/browser/profiles/profile_info_cache_unittest.h [modify] https://crrev.com/6c602d4eb8a37e78a3ae81428b8981e58af807dd/chrome/test/base/testing_profile_manager.cc [modify] https://crrev.com/6c602d4eb8a37e78a3ae81428b8981e58af807dd/chrome/test/base/testing_profile_manager.h
,
Jan 26 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5121fb3ecfcecb1693ee4113707532fa5673aa93 commit 5121fb3ecfcecb1693ee4113707532fa5673aa93 Author: WC Leung <lwchkg@chromium.org> Date: Fri Jan 26 19:05:52 2018 Enable ProfileAttributesStorageTest.LoadAvatarFromDiskTest on Linux ASAN The captioned test is now fixed, so we enable the test again. This is a revert of https://chromium-review.googlesource.com/827006. Bug: 794821 Change-Id: Icabf8e2fed02d0b2c3220af7e1a1001afbaadbfd Reviewed-on: https://chromium-review.googlesource.com/883821 Reviewed-by: Lei Zhang <thestig@chromium.org> Reviewed-by: Bernhard Bauer <bauerb@chromium.org> Commit-Queue: Bernhard Bauer <bauerb@chromium.org> Cr-Commit-Position: refs/heads/master@{#532034} [modify] https://crrev.com/5121fb3ecfcecb1693ee4113707532fa5673aa93/chrome/browser/profiles/profile_attributes_storage_unittest.cc
,
Jan 28 2018
|
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 by kbr@chromium.org
, Dec 14 2017