"BlacklistTest.LoadBlacklistedLibrary" is flaky |
||||||||||||||||
Issue description"BlacklistTest.LoadBlacklistedLibrary" is flaky. This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label. We have detected 14 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLwsSBUZsYWtlIiRCbGFja2xpc3RUZXN0LkxvYWRCbGFja2xpc3RlZExpYnJhcnkM. Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
,
Apr 14 2017
Sample failure log from https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/422616: [ RUN ] BlacklistTest.LoadBlacklistedLibrary e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(84): error: Value of: dll_blacklisted.is_valid() Actual: true Expected: false e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(85): error: Value of: ::GetEnvironmentVariableW(test_data[i].dll_beacon, 0, 0) Actual: 2 Expected: 0u Which is: 0 e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(92): error: Value of: array_size Actual: 0 Expected: 1 + num_initially_blocked_ Which is: 1 e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(93): error: Value of: blocked_dlls[num_initially_blocked_] Actual: NULL Expected: test_data[i].dll_name Which is: L"blacklist_test_dll_3.dll" e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(112): error: Value of: dll_blacklisted_different_case.is_valid() Actual: true Expected: false e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(113): error: Value of: ::GetEnvironmentVariableW(test_data[i].dll_beacon, 0, 0) Actual: 2 Expected: 0u Which is: 0 [ FAILED ] BlacklistTest.LoadBlacklistedLibrary (52 ms) [ RUN ] BlacklistTest.LoadBlacklistedLibrary e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(84): error: Value of: dll_blacklisted.is_valid() Actual: true Expected: false e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(85): error: Value of: ::GetEnvironmentVariableW(test_data[i].dll_beacon, 0, 0) Actual: 2 Expected: 0u Which is: 0 e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(92): error: Value of: array_size Actual: 0 Expected: 1 + num_initially_blocked_ Which is: 1 e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(93): error: Value of: blocked_dlls[num_initially_blocked_] Actual: NULL Expected: test_data[i].dll_name Which is: L"blacklist_test_dll_3.dll" e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(112): error: Value of: dll_blacklisted_different_case.is_valid() Actual: true Expected: false e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(113): error: Value of: ::GetEnvironmentVariableW(test_data[i].dll_beacon, 0, 0) Actual: 2 Expected: 0u Which is: 0 [ FAILED ] BlacklistTest.LoadBlacklistedLibrary (16 ms) [ RUN ] BlacklistTest.LoadBlacklistedLibrary e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(84): error: Value of: dll_blacklisted.is_valid() Actual: true Expected: false e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(85): error: Value of: ::GetEnvironmentVariableW(test_data[i].dll_beacon, 0, 0) Actual: 2 Expected: 0u Which is: 0 e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(92): error: Value of: array_size Actual: 0 Expected: 1 + num_initially_blocked_ Which is: 1 e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(93): error: Value of: blocked_dlls[num_initially_blocked_] Actual: NULL Expected: test_data[i].dll_name Which is: L"blacklist_test_dll_3.dll" e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(112): error: Value of: dll_blacklisted_different_case.is_valid() Actual: true Expected: false e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(113): error: Value of: ::GetEnvironmentVariableW(test_data[i].dll_beacon, 0, 0) Actual: 2 Expected: 0u Which is: 0 [ FAILED ] BlacklistTest.LoadBlacklistedLibrary (16 ms) [ RUN ] BlacklistTest.LoadBlacklistedLibrary e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(84): error: Value of: dll_blacklisted.is_valid() Actual: true Expected: false e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(85): error: Value of: ::GetEnvironmentVariableW(test_data[i].dll_beacon, 0, 0) Actual: 2 Expected: 0u Which is: 0 e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(92): error: Value of: array_size Actual: 0 Expected: 1 + num_initially_blocked_ Which is: 1 e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(93): error: Value of: blocked_dlls[num_initially_blocked_] Actual: NULL Expected: test_data[i].dll_name Which is: L"blacklist_test_dll_3.dll" e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(112): error: Value of: dll_blacklisted_different_case.is_valid() Actual: true Expected: false e:\b\c\b\win\src\chrome_elf\blacklist\test\blacklist_test.cc(113): error: Value of: ::GetEnvironmentVariableW(test_data[i].dll_beacon, 0, 0) Actual: 2 Expected: 0u Which is: 0 [ FAILED ] BlacklistTest.LoadBlacklistedLibrary (16 ms)
,
Apr 14 2017
Adding in a few people that have touched this test in the past. Anyone know why it might have started being flaky since 4/11?
,
Apr 16 2017
Detected 3 new flakes for test/step "BlacklistTest.LoadBlacklistedLibrary". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLwsSBUZsYWtlIiRCbGFja2xpc3RUZXN0LkxvYWRCbGFja2xpc3RlZExpYnJhcnkM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Apr 17 2017
I'll go ahead and disable the test for now, as it keeps flaking: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_elf_unittests&tests=BlacklistTest.LoadBlacklistedLibrary krstnmnlsn@: can you please help triage this failure?
,
Apr 17 2017
Actually, it appears that emails to krstnmnlsn@ are bouncing. csharp@: can you please help triage this instead?
,
Apr 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6bb5dfd77a43f4c55e1bdc463a6568b27cd6de05 commit 6bb5dfd77a43f4c55e1bdc463a6568b27cd6de05 Author: alexmos <alexmos@chromium.org> Date: Mon Apr 17 17:26:53 2017 Disable BlacklistTest.LoadBlacklistedLibrary due to being flaky. BUG= 711651 TBR=csharp@chromium.org CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.win:win10_chromium_x64_rel_ng Review-Url: https://codereview.chromium.org/2822993002 Cr-Commit-Position: refs/heads/master@{#464942} [modify] https://crrev.com/6bb5dfd77a43f4c55e1bdc463a6568b27cd6de05/chrome_elf/blacklist/test/blacklist_test.cc
,
Apr 17 2017
Detected 8 new flakes for test/step "BlacklistTest.LoadBlacklistedLibrary". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyLwsSBUZsYWtlIiRCbGFja2xpc3RUZXN0LkxvYWRCbGFja2xpc3RlZExpYnJhcnkM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Apr 17 2017
-Sheriff-Chromium, test already disabled
,
Apr 18 2017
No idea why this test just started flaking, assigning to pennymac@ since she's done more work on this code recently and might have some idea what is going on.
,
May 3 2017
,
May 3 2017
,
May 6 2017
,
May 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b1d09797bdb4b4b4340662a5bc25a0d611cc42ba commit b1d09797bdb4b4b4340662a5bc25a0d611cc42ba Author: pennymac <pennymac@chromium.org> Date: Mon May 08 19:52:16 2017 [chrome_elf] blacklist flaky tests Possible flake fix. Overriding chrome_elf test launch options. BUG= 711651 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.win:win10_chromium_x64_rel_ng Review-Url: https://codereview.chromium.org/2861793003 Cr-Commit-Position: refs/heads/master@{#470086} [modify] https://crrev.com/b1d09797bdb4b4b4340662a5bc25a0d611cc42ba/chrome_elf/blacklist/test/blacklist_test.cc [modify] https://crrev.com/b1d09797bdb4b4b4340662a5bc25a0d611cc42ba/chrome_elf/run_all_unittests.cc
,
May 8 2017
Adjustment landed. Setting this to fixed. Please re-open if you still see the same flakes.
,
May 10 2017
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng?numbuilds=200 This test is failing on many builds (search for chrome_elf_unittests in the above link). Please disable it in the meantime.
,
May 10 2017
btw in the future, if you have a possible fix please keep a close eye on the bots to confirm that flakiness has gone away. I see 60 flakes in the last 2 days since this relanded per the first link in this bug.
,
May 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2435ed2540c29f9c76e0b35c96b72e0d531daa92 commit 2435ed2540c29f9c76e0b35c96b72e0d531daa92 Author: jam <jam@chromium.org> Date: Wed May 10 19:37:29 2017 Revert of [chrome_elf] blacklist flaky tests (patchset #3 id:60001 of https://codereview.chromium.org/2861793003/ ) Reason for revert: This is still causing lots of flake. Original issue's description: > [chrome_elf] blacklist flaky tests > > Possible flake fix. Overriding chrome_elf test launch options. > > BUG= 711651 > CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.win:win10_chromium_x64_rel_ng > > Review-Url: https://codereview.chromium.org/2861793003 > Cr-Commit-Position: refs/heads/master@{#470086} > Committed: https://chromium.googlesource.com/chromium/src/+/b1d09797bdb4b4b4340662a5bc25a0d611cc42ba TBR=robertshield@chromium.org,pennymac@chromium.org BUG= 711651 NOTREECHECKS=true NOPRESUBMIT=true NOTRY=true Review-Url: https://codereview.chromium.org/2869273005 Cr-Commit-Position: refs/heads/master@{#470670} [modify] https://crrev.com/2435ed2540c29f9c76e0b35c96b72e0d531daa92/chrome_elf/blacklist/test/blacklist_test.cc [modify] https://crrev.com/2435ed2540c29f9c76e0b35c96b72e0d531daa92/chrome_elf/run_all_unittests.cc
,
May 10 2017
This delayed the landing of a v8 roll that's supposed to unwedge the cq (https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/440945)
,
May 11 2017
fyi: some of the mayhem appears to be a separate problem with the Win bots affecting many devs. https://bugs.chromium.org/p/chromium/issues/detail?id=721245 I still the flakes of this ticket remain, so I'll look into them again locally. But things are muddied. <internal link> https://chromium-swarm.appspot.com/task?id=36046366ecc71d10&refresh=10&show_raw=1 </internal link> ---- John, thank you for pointing out the link in the first comment on this ticket. I was unaware of a good place to watch for waterfall flakes. chromium-try-flakes.appspot.com does not seem to be linked from anywhere obvious, including the chromium waterfall. Is it a well known dashboard? The only link I found was https://build.chromium.org/f/chromium/flakiness/ off of the chromium waterfall main page (https://build.chromium.org/p/chromium/waterfall).
,
May 11 2017
Penny: the tool is meant to file bugs automatically based on flakes seen on the CQ, so that's why it's not advertised directly.
,
May 15 2017
,
Jul 27 2017
Leaving the test turned off for now. Changing to P3. Blacklist implementation is changing very soon. Will delete these tests at that time.
,
Jun 18 2018
Obsolete. These tests are gone now. https://chromium-review.googlesource.com/1076972 |
||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||
Comment 1 Deleted