New issue
Advanced search Search tips

Issue 607169 link

Starred by 0 users
Status: Fixed
Owner:
agrieve@chromium.org
Closed: Jun 2016
Cc:
jbudorick@chromium.org
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug

Blocking:
issue 617213



Sign in to add a comment

Android tests should not pass when runtime_deps is incorrect

Project Member Reported by agrieve@chromium.org, Apr 27 2016 
Since all tests push their runtime_deps to $EXTERNAL_STORAGE and do not delete them afterwards, devices tend to accumulate the union of all test data. This leads to tests that have the wrong runtime_data set to pass locally and then fail on bots where the devices are routinely wiped.

I think the simplest way to fix this would be to have the test runner use PushChangedFiles(delete_device_stale=True). However, to do this we'd also need to change the test data directory from $EXTERNAL_STORAGE to $EXTERNAL_STORAGE/subdir.
Project Member

Comment 1 by bugdroid1@chromium.org, May 26 2016 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0fe567512193cc8fbfcfd006ff1d504bd6346290

commit 0fe567512193cc8fbfcfd006ff1d504bd6346290
Author: agrieve <agrieve@chromium.org>
Date: Thu May 26 13:50:18 2016

Move side-loaded test data /sdcard -> /sdcard/chromium_tests_root

And use delete_device_stale=True when pushing. This will prevent tests
mistakenly passing when they depend on stale data files from a previous test.

BUG= 607169 

Review-Url: https://codereview.chromium.org/1926793002
Cr-Commit-Position: refs/heads/master@{#396169}

[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/base/base_paths_android.cc
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/base/test/android/javatests/src/org/chromium/base/test/util/UrlUtils.java
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/base/test/test_support_android.cc
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/base/test/test_support_android.h
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/build/android/pylib/local/device/local_device_gtest_run.py
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/build/android/pylib/local/device/local_device_instrumentation_test_run.py
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/build/android/pylib/remote/device/remote_device_test_run.py
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/build/apk_test.gypi
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/chrome/common/chrome_paths.cc
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/components/cronet/android/BUILD.gn
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/components/cronet/android/test/mock_cert_verifier.cc
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/components/cronet/android/test/native_test_server.cc
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/components/cronet/android/test/quic_test_server.cc
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/components/cronet/android/test/src/org/chromium/net/MockCertVerifier.java
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/components/cronet/android/test/src/org/chromium/net/NativeTestServer.java
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/components/cronet/android/test/src/org/chromium/net/QuicTestServer.java
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/net/test/android/javatests/src/org/chromium/net/test/EmbeddedTestServer.java
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/net/test/android/javatests/src/org/chromium/net/test/EmbeddedTestServerImpl.java
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/net/test/android/javatests/src/org/chromium/net/test/util/CertTestUtil.java
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/net/test/embedded_test_server/android/embedded_test_server_android.cc
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/testing/android/native_test.gyp
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/testing/android/native_test/BUILD.gn
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/testing/android/native_test/java/src/org/chromium/native_test/NativeTestActivity.java
[modify] https://crrev.com/0fe567512193cc8fbfcfd006ff1d504bd6346290/testing/android/native_test/native_test_launcher.cc
Project Member

Comment 2 by bugdroid1@chromium.org, May 26 2016 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5a7a6cd176bd3c1a9524893801b8d177ebe0f63a

commit 5a7a6cd176bd3c1a9524893801b8d177ebe0f63a
Author: agrieve <agrieve@chromium.org>
Date: Thu May 26 14:53:18 2016

Revert of
Project Member

Comment 3 by bugdroid1@chromium.org, May 27 2016 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2524a0db714a44fbce723e00f6ed914e6d143cb6

commit 2524a0db714a44fbce723e00f6ed914e6d143cb6
Author: agrieve <agrieve@chromium.org>
Date: Fri May 27 15:57:57 2016

Reland of Move side-loaded test data /sdcard -> /sdcard/chromium_tests_root

Reason for reland:
Updated webview tests that broke (WebViewLayoutTest.java)

And use delete_device_stale=True when pushing. This will prevent tests
mistakenly passing when they depend on stale data files from a previous test.

TBR=jbudorick@chromium.org,thakis@chromium.org,mef@chromium.org
BUG= 607169 

Review-Url: https://codereview.chromium.org/2018663002
Cr-Commit-Position: refs/heads/master@{#396479}

[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/android_webview/tools/system_webview_shell/layout_tests/src/org/chromium/webview_shell/test/WebViewLayoutTest.java
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/android_webview/tools/system_webview_shell/test/data/run_tests.sh
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/base/base_paths_android.cc
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/base/test/android/javatests/src/org/chromium/base/test/util/UrlUtils.java
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/base/test/test_support_android.cc
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/base/test/test_support_android.h
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/build/android/pylib/local/device/local_device_gtest_run.py
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/build/android/pylib/local/device/local_device_instrumentation_test_run.py
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/build/android/pylib/remote/device/remote_device_test_run.py
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/build/apk_test.gypi
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/chrome/common/chrome_paths.cc
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/components/cronet/android/BUILD.gn
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/components/cronet/android/test/mock_cert_verifier.cc
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/components/cronet/android/test/native_test_server.cc
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/components/cronet/android/test/quic_test_server.cc
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/components/cronet/android/test/src/org/chromium/net/MockCertVerifier.java
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/components/cronet/android/test/src/org/chromium/net/NativeTestServer.java
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/components/cronet/android/test/src/org/chromium/net/QuicTestServer.java
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/net/test/android/javatests/src/org/chromium/net/test/EmbeddedTestServer.java
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/net/test/android/javatests/src/org/chromium/net/test/EmbeddedTestServerImpl.java
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/net/test/android/javatests/src/org/chromium/net/test/util/CertTestUtil.java
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/net/test/embedded_test_server/android/embedded_test_server_android.cc
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/testing/android/native_test.gyp
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/testing/android/native_test/BUILD.gn
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/testing/android/native_test/java/src/org/chromium/native_test/NativeTest.java
[modify] https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6/testing/android/native_test/native_test_launcher.cc

Comment 4 by agrieve@chromium.org, Jun 1 2016

Status: Fixed (was: Started)

Comment 5 by joh...@chromium.org, Jun 6 2016

Blocking: 617213
Project Member

Comment 6 by bugdroid1@chromium.org, Jun 6 2016 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb

commit 55ff9438e11da5a78a6ba5d100cebfcfdbc07deb
Author: agrieve <agrieve@chromium.org>
Date: Mon Jun 06 22:13:55 2016

Revert of Reland of Move side-loaded test data /sdcard -> /sdcard/gtestdata (patchset #3 id:300001 of https://codereview.chromium.org/2018663002/ )

Reason for revert:
Breaking quota-related tests in content_browsertests
https://crbug.com/617213

Original issue's description:
> Reland of Move side-loaded test data /sdcard -> /sdcard/chromium_tests_root
>
> Reason for reland:
> Updated webview tests that broke (WebViewLayoutTest.java)
>
> And use delete_device_stale=True when pushing. This will prevent tests
> mistakenly passing when they depend on stale data files from a previous test.
>
> TBR=jbudorick@chromium.org,thakis@chromium.org,mef@chromium.org
> BUG= 607169 
>
> Committed: https://crrev.com/2524a0db714a44fbce723e00f6ed914e6d143cb6
> Cr-Commit-Position: refs/heads/master@{#396479}

TBR=jbudorick@chromium.org,thakis@chromium.org,mef@chromium.org,torne@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG= 607169 ,617213,616155

Review-Url: https://codereview.chromium.org/2044523003
Cr-Commit-Position: refs/heads/master@{#398137}

[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/android_webview/tools/system_webview_shell/layout_tests/src/org/chromium/webview_shell/test/WebViewLayoutTest.java
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/android_webview/tools/system_webview_shell/test/data/run_tests.sh
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/base/base_paths_android.cc
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/base/test/android/javatests/src/org/chromium/base/test/util/UrlUtils.java
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/base/test/test_support_android.cc
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/base/test/test_support_android.h
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/build/android/pylib/local/device/local_device_gtest_run.py
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/build/android/pylib/local/device/local_device_instrumentation_test_run.py
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/build/android/pylib/remote/device/remote_device_test_run.py
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/build/apk_test.gypi
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/chrome/common/chrome_paths.cc
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/components/cronet/android/BUILD.gn
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/components/cronet/android/test/mock_cert_verifier.cc
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/components/cronet/android/test/native_test_server.cc
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/components/cronet/android/test/quic_test_server.cc
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/components/cronet/android/test/src/org/chromium/net/MockCertVerifier.java
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/components/cronet/android/test/src/org/chromium/net/NativeTestServer.java
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/components/cronet/android/test/src/org/chromium/net/QuicTestServer.java
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/net/test/android/javatests/src/org/chromium/net/test/EmbeddedTestServer.java
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/net/test/android/javatests/src/org/chromium/net/test/EmbeddedTestServerImpl.java
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/net/test/android/javatests/src/org/chromium/net/test/util/CertTestUtil.java
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/net/test/embedded_test_server/android/embedded_test_server_android.cc
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/testing/android/native_test.gyp
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/testing/android/native_test/BUILD.gn
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/testing/android/native_test/java/src/org/chromium/native_test/NativeTest.java
[modify] https://crrev.com/55ff9438e11da5a78a6ba5d100cebfcfdbc07deb/testing/android/native_test/native_test_launcher.cc
Project Member

Comment 7 by bugdroid1@chromium.org, Jun 8 2016 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d4d66d4d0353020ddf9f1423d08cf39ed911c329

commit d4d66d4d0353020ddf9f1423d08cf39ed911c329
Author: agrieve <agrieve@chromium.org>
Date: Wed Jun 08 16:53:39 2016

Reland #2: Move side-loaded test data /sdcard -> /sdcard/gtestdata

Reason for reland:
- content_browsertests and component_browsertests updated to
clear the correct private data directory between test runs.

Uses delete_device_stale=True when pushing. This will prevent tests
mistakenly passing when they depend on stale data files from a previous test.

TBR=jbudorick@chromium.org,thakis@chromium.org,mef@chromium.org,torne@chromium.org
BUG= 607169 ,617213,616155

Review-Url: https://codereview.chromium.org/2043803003
Cr-Commit-Position: refs/heads/master@{#398601}

[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/android_webview/tools/system_webview_shell/layout_tests/src/org/chromium/webview_shell/test/WebViewLayoutTest.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/android_webview/tools/system_webview_shell/test/data/run_tests.sh
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/base/base_paths_android.cc
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/base/test/android/javatests/src/org/chromium/base/test/util/UrlUtils.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/base/test/test_support_android.cc
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/base/test/test_support_android.h
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/build/android/pylib/local/device/local_device_gtest_run.py
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/build/android/pylib/local/device/local_device_instrumentation_test_run.py
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/build/android/pylib/remote/device/remote_device_test_run.py
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/build/apk_test.gypi
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/chrome/common/chrome_paths.cc
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/components/BUILD.gn
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/components/cronet/android/BUILD.gn
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/components/cronet/android/test/mock_cert_verifier.cc
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/components/cronet/android/test/native_test_server.cc
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/components/cronet/android/test/quic_test_server.cc
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/components/cronet/android/test/src/org/chromium/net/MockCertVerifier.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/components/cronet/android/test/src/org/chromium/net/NativeTestServer.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/components/cronet/android/test/src/org/chromium/net/QuicTestServer.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/components/test/android/browsertests_apk/src/org/chromium/components_browsertests_apk/ComponentsBrowserTestsActivity.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/content/shell/android/browsertests_apk/src/org/chromium/content_browsertests_apk/ContentBrowserTestsActivity.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/content/test/BUILD.gn
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/net/test/android/javatests/src/org/chromium/net/test/EmbeddedTestServer.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/net/test/android/javatests/src/org/chromium/net/test/EmbeddedTestServerImpl.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/net/test/android/javatests/src/org/chromium/net/test/util/CertTestUtil.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/net/test/embedded_test_server/android/embedded_test_server_android.cc
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/testing/android/native_test.gyp
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/testing/android/native_test/BUILD.gn
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/testing/android/native_test/java/src/org/chromium/native_test/NativeTest.java
[modify] https://crrev.com/d4d66d4d0353020ddf9f1423d08cf39ed911c329/testing/android/native_test/native_test_launcher.cc
►

Sign in to add a comment