Several tests in org.chromium.components.crash.browser.CrashDumpManagerTest are flaky |
||||||||||||
Issue description"org.chromium.components.crash.browser.CrashDumpManagerTest#testUploadMinidump_NullMinidumpPath" is flaky. This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label. We have detected 3 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyaQsSBUZsYWtlIl5vcmcuY2hyb21pdW0uY29tcG9uZW50cy5jcmFzaC5icm93c2VyLkNyYXNoRHVtcE1hbmFnZXJUZXN0I3Rlc3RVcGxvYWRNaW5pZHVtcF9OdWxsTWluaWR1bXBQYXRoDA. Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
,
May 24 2017
The test was added recently by isherman@.
,
May 24 2017
I am disabling the test, the CL is https://codereview.chromium.org/2902103004.
,
May 24 2017
It looks like we're trying to delete all the files in the cache-directory - maybe there are files (or directories) in the cache-directory, that aren't cleaned before running this test? Ilya: if it makes sense I think we should use a temporary directory for the tests instead of using the cache-directory directly.
,
May 24 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/406f05e73b87ed54dd6b6ace45cea9385208c5ac commit 406f05e73b87ed54dd6b6ace45cea9385208c5ac Author: vitaliii <vitaliii@chromium.org> Date: Wed May 24 15:08:26 2017 Disable CrashDumpManagerTest#testUploadMinidump_NullMinidumpPath (flaky) TBR=gsennton@chromium.org,isherman@chromium.org BUG=725379 Review-Url: https://codereview.chromium.org/2902103004 Cr-Commit-Position: refs/heads/master@{#474289} [modify] https://crrev.com/406f05e73b87ed54dd6b6ace45cea9385208c5ac/components/crash/android/javatests/src/org/chromium/components/crash/browser/CrashDumpManagerTest.java
,
May 26 2017
Issue 726604 has been merged into this issue.
,
May 26 2017
testUploadMinidump_NoCallback is also flaky (see Issue 726604 ). Disabling and duping that bug into this.
,
May 26 2017
,
May 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/73fce28b25fd97073601ecb04ee5f88c16cc4ad2 commit 73fce28b25fd97073601ecb04ee5f88c16cc4ad2 Author: Matt Giuca <mgiuca@chromium.org> Date: Fri May 26 07:44:45 2017 Disable CrashDumpManagerTest#testUploadMinidump_NoCallback (flaky). TBR=gsennton@chromium.org,isherman@chromium.org Bug: 725379 Change-Id: I20682c13346285b427b94f04300eb2056f0b5835 Reviewed-on: https://chromium-review.googlesource.com/517003 Reviewed-by: Matt Giuca <mgiuca@chromium.org> Commit-Queue: Matt Giuca <mgiuca@chromium.org> Cr-Commit-Position: refs/heads/master@{#474957} [modify] https://crrev.com/73fce28b25fd97073601ecb04ee5f88c16cc4ad2/components/crash/android/javatests/src/org/chromium/components/crash/browser/CrashDumpManagerTest.java
,
May 30 2017
,
May 30 2017
,
May 31 2017
Issue 727506 has been merged into this issue.
,
May 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6b0e73063c6d65e7172cd247e05006078a60cf23 commit 6b0e73063c6d65e7172cd247e05006078a60cf23 Author: isherman <isherman@chromium.org> Date: Wed May 31 15:52:15 2017 Fix some flaky tests: Use a dedicated temp dir. BUG=725379 TEST=CrashDumpManagerTest.* R=gsennton@chromium.org Review-Url: https://codereview.chromium.org/2911983008 Cr-Commit-Position: refs/heads/master@{#475924} [modify] https://crrev.com/6b0e73063c6d65e7172cd247e05006078a60cf23/components/crash/android/javatests/src/org/chromium/components/crash/browser/CrashDumpManagerTest.java
,
May 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f6c031634d0e361e90534d410121d25c991b3a80 commit f6c031634d0e361e90534d410121d25c991b3a80 Author: pkasting <pkasting@chromium.org> Date: Wed May 31 20:02:39 2017 Revert of Fix some flaky tests: Use a dedicated temp dir. (patchset #1 id:1 of https://codereview.chromium.org/2911983008/ ) Reason for revert: Looks to be causing consistent failures on Android, e.g.: https://luci-milo.appspot.com/buildbot/chromium.linux/Android%20Tests/42292 Original issue's description: > Fix some flaky tests: Use a dedicated temp dir. > > BUG=725379 > TEST=CrashDumpManagerTest.* > R=gsennton@chromium.org > > Review-Url: https://codereview.chromium.org/2911983008 > Cr-Commit-Position: refs/heads/master@{#475924} > Committed: https://chromium.googlesource.com/chromium/src/+/6b0e73063c6d65e7172cd247e05006078a60cf23 TBR=gsennton@chromium.org,isherman@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG=725379 Review-Url: https://codereview.chromium.org/2915883003 Cr-Commit-Position: refs/heads/master@{#475991} [modify] https://crrev.com/f6c031634d0e361e90534d410121d25c991b3a80/components/crash/android/javatests/src/org/chromium/components/crash/browser/CrashDumpManagerTest.java
,
May 31 2017
Issue 726138 has been merged into this issue.
,
May 31 2017
,
Jun 1 2017
Hmm, I'm not sure why the revert in #14 was necessary -- that is, I'm not sure why these tests are still failing in some situations. Gustav, any ideas?
,
Jun 1 2017
,
Jun 1 2017
I couldn't repro this on an O device. The failures (at least here: https://chromium-swarm.appspot.com/task?id=3678480a2cb49510&refresh=10&show_raw=1 ) are running hammerheads on KTU84P (KitKat MR2), I'll flash one of those and see if I can repro. I would say that the different failures here: https://00e9e64bac2af4321a0dfa9ff8ca56d0e4f36a9588adea6da0-apidata.googleusercontent.com/download/storage/v1/b/chromium-result-details/o/html%2Fchrome_public_test_apk_Android%20Tests_42292_2017_05_31_T17_22_29-UTC?qk=AD5uMEvLa4a-rdo-67M3KerbsggFKuFZsjI7GuR5l6Nd4ttewrriTvZErrFz8AWxXvr6DI7mFMQeDI-RevS5-Hk_TK7v-ajPEFtYJ6WqZMhdgil88ZC91uccskLHx6UN1v8xlYCrOquDhbUcDbu6TnNY6Pf4ZHOAfMW8jgXzuKO6f-qEYkxx5wffbMr-wtHJLrT_axFonof3f2gPtA5NFJxDhnI-vuTsxJcEZak0td6_oD1BjsD8j8wZqejyeMh5Hs4Pr6lzEBfYvFZ6SViz7iQC_8RTYZcOtfeczVUV1ZmrA9JJkwdTSljbuNNO6JNVoEQ0YPXmj8Yg2KT8Y27FecCcH6AKJR5E2-m9XlJYbECFd0EdeCv9FLQVxUaBeuNkyxQcB8Gmw0oGuvpctKvNhKdwdVsZD7VO_NgQ6VFv6JVmJHXtgR2JAsXxm76vM1yYA-NArPV47d8rH2Qa4e1lCT564BK8CpbkS0hfyAU5TESd8JHbtL4f1_QI6yizQ9AfxejC6D_qXKp_ERhHUGk-IxVX1yU43iFFVguaawGMenYCfYGQHAGpakQrSbrktlg7fPM4sk2GTnjWEqcx2wKkZFmQRE5uO3grNQXq3EUhZ-7zasMhXll8PcS7HZIMASo144Tw43O8wulxZftBrOLczStQuf0kmFOVOIkORimD8lZHAuHBrRXZXuySyrcEDkr-bZJT7CENlk-GsWECSSMlZfpLG99LzKHJ-8ADwjymrzl5pBm-mOJ-egKygLZ9k81XL8xXjKZam6mB_HYDzWZ0zfON1Kp_5IbcHPpOlK5jVkpJn85QMss9r-zzBZBfD-AkZrSFLPkR7-wssmoItetIDEANDU6V8rSnQqCzqczF9h4xD728cy9Il9E suggest that the temporary directory is never created. java.io.IOException: open failed: ENOENT (No such file or directory) at java.io.File.createNewFile(File.java:946) at org.chromium.components.crash.browser.CrashDumpManagerTest.testUploadMinidump_MinidumpPathIsValid(CrashDumpManagerTest.java:99) at java.lang.reflect.Method.invokeNative(Native Method) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199) at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:129) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701) Caused by: libcore.io.ErrnoException: open failed: ENOENT (No such file or directory) at libcore.io.Posix.open(Native Method) at libcore.io.BlockGuardOs.open(BlockGuardOs.java:110) at java.io.File.createNewFile(File.java:939) ... 15 more junit.framework.AssertionFailedError at org.chromium.components.crash.browser.CrashDumpManagerTest.testUploadMinidump_MinidumpPathIsDirectory(CrashDumpManagerTest.java:83) at java.lang.reflect.Method.invokeNative(Native Method) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199) at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:129) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701) java.io.IOException: open failed: ENOENT (No such file or directory) at java.io.File.createNewFile(File.java:946) at org.chromium.components.crash.browser.CrashDumpManagerTest.testUploadMinidump_NoCallback(CrashDumpManagerTest.java:47) at java.lang.reflect.Method.invokeNative(Native Method) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199) at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:129) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701) Caused by: libcore.io.ErrnoException: open failed: ENOENT (No such file or directory) at libcore.io.Posix.open(Native Method) at libcore.io.BlockGuardOs.open(BlockGuardOs.java:110) at java.io.File.createNewFile(File.java:939) ... 15 more
,
Jun 1 2017
Ran these tests on a hammerhead with KTU84P and repeated 20 times - the tests pass each time. I don't know what's different on the tryserver bots. Do we have a way of running these tests on those bots?
,
Jun 1 2017
#20: try jobs :) but please don't kick a bunch of try jobs on linux_android_rel_ng There are indeed other ways to run tests on the same hardware. I'll follow up off-thread; it's kinda complicated at the moment.
,
Jun 1 2017
Detected 7 new flakes for test/step "org.chromium.components.crash.browser.CrashDumpManagerTest#testUploadMinidump_MinidumpPathIsDirectory". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNycAsSBUZsYWtlImVvcmcuY2hyb21pdW0uY29tcG9uZW50cy5jcmFzaC5icm93c2VyLkNyYXNoRHVtcE1hbmFnZXJUZXN0I3Rlc3RVcGxvYWRNaW5pZHVtcF9NaW5pZHVtcFBhdGhJc0RpcmVjdG9yeQw. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jun 2 2017
I agree that it seems the temp directory is failing to be created. I'm not sure what might cause that -- file permissions? Full disk? Thanks for helping to investigate! Please include me on off-thread discussion as well -- if nothing else, I might learn something =)
,
Jun 2 2017
,
Jun 2 2017
Not a trooper bug, fwiw. Writing up the instructions now.
,
Jun 2 2017
Ugh, I just realized that the bot pkasting@ is referring to (that's failing with Ilya's CL) is 'Android Tests' on 'chromium.linux'. John, is it OK to use your instructions on that builder? (I tried running on linux_android_rel_ng / tryserver.chromium.android and that run succeeds).
,
Jun 2 2017
Or is it possible to see runs from 31st of May for linux_android_rel_ng? From here: https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng I can only see the 200 most recent runs, and that's not enough to see results from when Ilya landed his CL.
,
Jun 2 2017
#26: Yeah, that uses the same devices (hammerhead/KTU84P) as linux_android_rel_ng. #27: Again, I'll follow up offline.
,
Jun 3 2017
Detected 3 new flakes for test/step "org.chromium.components.crash.browser.CrashDumpManagerTest#testUploadMinidump_MinidumpPathIsDirectory". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNycAsSBUZsYWtlImVvcmcuY2hyb21pdW0uY29tcG9uZW50cy5jcmFzaC5icm93c2VyLkNyYXNoRHVtcE1hbmFnZXJUZXN0I3Rlc3RVcGxvYWRNaW5pZHVtcF9NaW5pZHVtcFBhdGhJc0RpcmVjdG9yeQw. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jun 5 2017
https://chromium-review.googlesource.com/c/524042/ is out to disable the remaining tests.
,
Jun 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/119740b795871b7c4fc374c3e13d15ebf20ced4f commit 119740b795871b7c4fc374c3e13d15ebf20ced4f Author: Colin Blundell <blundell@chromium.org> Date: Mon Jun 05 09:51:28 2017 Disable remaining CrashDumpManager tests due to flake At least one of these tests continues to exhibit flake, and there's no reason to suppose that the other isn't subject to the same problems as the rest of the tests in this suite, as it looks like it's going through the same basic flow. TBR=isherman@chromium.org Bug: 725379 Change-Id: I1fa0747d9554e64b67f4c6ee11777d9096ff5572 Reviewed-on: https://chromium-review.googlesource.com/524042 Reviewed-by: Colin Blundell <blundell@chromium.org> Commit-Queue: Colin Blundell <blundell@chromium.org> Cr-Commit-Position: refs/heads/master@{#476964} [modify] https://crrev.com/119740b795871b7c4fc374c3e13d15ebf20ced4f/components/crash/android/javatests/src/org/chromium/components/crash/browser/CrashDumpManagerTest.java
,
Jun 5 2017
According to the flakiness dashboard these tests did fail a couple of times when run using swarming: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=testUploadMinidump just after Ilya's CL landed. AFAICT they failed consistently (2/2 times) until pkasting@ landed the revert. |
||||||||||||
►
Sign in to add a comment |
||||||||||||
Comment 1 by vitaliii@chromium.org
, May 24 2017