Audit existing Files.app tests |
|||||||||
Issue descriptionAudit the current Files.app tests we have (and which are enabled) to help us get a better understanding of our testing coverage, and where we need to improve. Ideally, also produce a doc on how to add new tests, mainly for UI integration tests (such as touch bug tests) as well as how to run/watch/debug them.
,
Mar 8 2018
,
Mar 8 2018
First glance shows that we have: * unit tests: - ui/file_manager/**/*_unittest.[js|html] - Run in chrome/browser/chromeos/file_manager/file_manager_jstest.cc * integration tests: - ui/file_manager/integration_tests/* - included in ui/file_manager/integration_tests/file_manager_test_manifest.json - Run in chrome/browser/chromeos/file_manager/file_manager_browsertest.cc * Run tests (currently 45): out/Default/browser_tests --gtest_filter=FileManager*
,
Mar 8 2018
40 Unit tests run with --gtest_filter=FileManagerJSTest.* 102 Integration tests run with --gtest_filter=*/FileManagerBrowserTest.* Summary of tests: https://docs.google.com/spreadsheets/d/1iuDr0thscc5LPM2UJUfaaSAOPXL7MceFKsIepg8uiDc/edit#gid=0
,
Mar 21 2018
Document added with disabled state of integration tests.
,
Mar 21 2018
Unit Tests: see #c3 Integration Tests: see #c3 API tests: - c++ unit tests for chrome APIs such as chrome.fileManagerPrivate. - https://cs.chromium.org/chromium/src/chrome/browser/chromeos/extensions/file_manager/file_manager_private_apitest.cc - runs as InProcBrowserTest
,
Mar 22 2018
,
Mar 22 2018
I see the remaining work here is to better understand the integration tests. 1/ What is causing the flakes? 2/ What amount of the backend is using fakes/mocks, and how much of actual backend code is running?
,
Mar 28 2018
,
Apr 15 2018
,
Apr 17 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/df79950240771a48655a06a1037c64a96c01d5a4 commit df79950240771a48655a06a1037c64a96c01d5a4 Author: Joel Hockey <joelhockey@chromium.org> Date: Tue Apr 17 03:12:44 2018 Pause FileManagerUITests. Turn off FileManagerUITests from CQ until current audit of tests by noel@ is complete. Bug: 819478 Change-Id: I14dca567de4fb7919d94b1d89a0cb2dada5b8d67 Reviewed-on: https://chromium-review.googlesource.com/1013668 Reviewed-by: Sasha Morrissey <sashab@chromium.org> Commit-Queue: Joel Hockey <joelhockey@chromium.org> Cr-Commit-Position: refs/heads/master@{#551233} [modify] https://crrev.com/df79950240771a48655a06a1037c64a96c01d5a4/chrome/browser/chromeos/file_manager/file_manager_uitest.cc
,
May 21 2018
This work is going great - lowering priority to P2.
,
Jul 18
,
Jul 19
1/ What is causing the flakes? answ: Starting tests before the test resources were set up. 2/ What amount of the backend is using fakes/mocks, and how much of actual backend code is running? answ: In chrome, unit tests mock the needed FileApps/FSP features, integration tests mock the needed Volumes (Drive, MTP, USB whatever). There is no need to bring up "actual backend" code [1]. [1] back-end code systems should provide/maintain the mock, and it should come with unit tests (see ToTT #528).
,
Jul 19
The flake cause fixed on crbug.com/831074#c15 , and all tests re-enabled. Feel I inherited this bug somehow due to that, but I have no clue what "Audit" means or actual intent or work needed on this bug. I can list || count them and compare to # FileManagerPrivateApiTest "unittests" 5 => 6 tests for f in *FileManagerPrivateApiTest*; do ./out/Release/browser_tests --gtest_filter="$f" --gtest_list_tests | \ sed "s| #.*$||g" done FileApp JS "unittests" 40 => 42 tests for f in *FileManagerJs*; do ./out/Release/browser_tests --gtest_filter="$f" --gtest_list_tests | \ sed "s| #.*$||g" done FileApp "integration tests" 102 => 322 tests for f in *FilesAppBrowser* \ *AudioPlayerBrowserTest* \ *GalleryBrowserTest* \ *VideoPlayerBrowserTest* \ *MultiProfileFileManagerBrowserTest* ; do ./out/Release/browser_tests --gtest_filter="$f" --gtest_list_tests | \ sed "s| #.*$||g" done
,
Jul 19
> FileApp "integration tests" 102 => 322 tests Seems folks know how to write integration tests, and have added a few. Still dunno what "Audit" means in this bug, re-assigning back to the owner.
,
Jul 19
No worries, I think this bug is complete given that we all know how to write tests (and they are documented at go/xf-noogler). We also have a much better idea of coverage. Marking as fixed.
,
Jul 20
For "coverage", as in real coverage reports, filed issue 865376. |
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 by sashab@chromium.org
, Mar 7 2018