Improve Files.app audio file manger browser test speed |
||||
Issue descriptionhttps://bugs.chromium.org/p/chromium/issues/detail?id=836254#c75 suggests one way to make Files.app browser tests faster. Another way for the slower tests, is to try and make those tests faster. The Audio file manager browser tests are a good example: there are two groups - AudioPlayerBrowserTests, which are generally fast. We should look them and do some clean-up, and see why they are fast [1]. - OpenAudioFiles/FilesAppBrowserTest are slow on ASAN [2], and disabled on MSAN since they timeout there. We should find out why they are slow, compared with the AudioPlayerBrowserTest group (I suspect they try to do too much). [1] Good on ASAN and MSAN https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=AudioPlayerBrowserTest [2] Too many grey boxes on ASAN, disabled on MSAN https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=OpenAudioFiles%2FFilesAppBrowserTest Main goal in this issue is to get group [2] working fast on ASAN and MSAN. +cc fukino.
,
May 28 2018
Related issue 837551 re the OpenAudioFile/FilesAppBrowserTest and MSAN.
,
May 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/be1d9b3a082d4284aaaa6b65e5d92792f88c9446 commit be1d9b3a082d4284aaaa6b65e5d92792f88c9446 Author: Noel Gordon <noel@chromium.org> Date: Tue May 29 05:13:40 2018 FilesAppBrowserTest: add audioOpenClose test case Add minimal test of opening and closing Audio Player from Files.App to work out the minimum tests run-time for this test group. Since we test that the window closes in audioOpenClose*, there is no need to test it again in all the other audio test cases. Remove that code. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: I675bca0318db0877014c402f2593c476ec86ff29 Reviewed-on: https://chromium-review.googlesource.com/1075448 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Naoki Fukino <fukino@chromium.org> Cr-Commit-Position: refs/heads/master@{#562342} [modify] https://crrev.com/be1d9b3a082d4284aaaa6b65e5d92792f88c9446/chrome/browser/chromeos/file_manager/file_manager_browsertest.cc [modify] https://crrev.com/be1d9b3a082d4284aaaa6b65e5d92792f88c9446/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
May 29 2018
,
May 29 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/06b9fe48cdf3660a7d7124937fee1c7ff026553e commit 06b9fe48cdf3660a7d7124937fee1c7ff026553e Author: Noel Gordon <noel@chromium.org> Date: Tue May 29 12:50:35 2018 OpenAudioFiles/FilesAppBrowserTest: use file system URL helper Add a helper to convert a file name to a file system URL and use it to do that conversion in all test cases. No change in test speed here, it is just easier to read. Minor: update comments for some test steps. Test: browser_test --gtest_filter="OpenAudioFiles/FilesApp*" Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: I089f07a230f4a9817016b834d067fa07b0b1bc3b Reviewed-on: https://chromium-review.googlesource.com/1075959 Reviewed-by: Stuart Langley <slangley@chromium.org> Reviewed-by: Naoki Fukino <fukino@chromium.org> Commit-Queue: Noel Gordon <noel@chromium.org> Cr-Commit-Position: refs/heads/master@{#562397} [modify] https://crrev.com/06b9fe48cdf3660a7d7124937fee1c7ff026553e/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
May 31 2018
Running the OpenAudioFiles tests locally % xvfb-run -s "-screen 0 1024x1024x24" \ ./out/Release/browser_tests --gtest_filter="OpenAudioFiles*FilesApp*" --test-launcher-jobs=26 IMPORTANT DEBUGGING NOTE: each test is run inside its own process... Using 13 parallel jobs. [1/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioOpenDownloads_GuestMode (14694 ms) [2/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioOpenCloseDownloads (14960 ms) [3/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioOpenCloseDownloads_GuestMode (14962 ms) [4/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioOpenDownloads (15213 ms) [5/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioOpenDrive (15214 ms) [6/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioOpenCloseDrive (15486 ms) [7/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioAutoAdvanceDrive (19932 ms) [8/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioNoRepeatModeSingleFileDrive (19931 ms) [9/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioRepeatAllModeSingleFileDrive (19934 ms) [10/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioRepeatOneModeSingleFileDrive (19934 ms) [11/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioRepeatAllModeMultipleFileDrive (20192 ms) [12/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioNoRepeatModeMultipleFileDrive (20193 ms) [13/13] OpenAudioFiles/FilesAppBrowserTest.Test/audioRepeatOneModeMultipleFileDrive (20453 ms) show Advance and Repeat tests take 5 seconds longer than the minimal test case (openClose).
,
May 31 2018
,
May 31 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1da5b76270996932a3f2a9d553da3c12c1c8f3b5 commit 1da5b76270996932a3f2a9d553da3c12c1c8f3b5 Author: Noel Gordon <noel@chromium.org> Date: Thu May 31 13:27:44 2018 OpenAudioFiles/FilesAppBrowserTest: code consistency clean-up - run git cl format over this file - add more documentation to getTrackTest() - add use helper routines for checking a track is playing - ensure the same test steps have the same comment - add a TODO to fix a broken test Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: I40063f0153147c4f160bdf51003cd717d94c50f0 Reviewed-on: https://chromium-review.googlesource.com/1080569 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Stuart Langley <slangley@chromium.org> Cr-Commit-Position: refs/heads/master@{#563204} [modify] https://crrev.com/1da5b76270996932a3f2a9d553da3c12c1c8f3b5/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
May 31 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/11e40b318e80d32e435af8c5eb7fa912f2566cc4 commit 11e40b318e80d32e435af8c5eb7fa912f2566cc4 Author: Noel Gordon <noel@chromium.org> Date: Thu May 31 13:33:22 2018 Use a track class selector in audioOpen test A /deep/ query selector is not needed to select a track from the Audio Player. Like AudioPlayerBrowserTest use a .track class selector to get the [active] track, and a :not[active] to get the inactive track. This removes the repeatUntil() polling, which wasn't needed, and the /deep/ combinator, which is deprecated (see issue 498405 ). No change in behavior, no new tests. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: I109d7954ae50ea97081292b5fee0fb66a542a4ce Reviewed-on: https://chromium-review.googlesource.com/1080585 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Stuart Langley <slangley@chromium.org> Cr-Commit-Position: refs/heads/master@{#563210} [modify] https://crrev.com/11e40b318e80d32e435af8c5eb7fa912f2566cc4/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
May 31 2018
So re #6, where does the extra 5 seconds come from? See [1], the Advance test for example is playing 'Beautiful Song' and waiting for it to finish to test that the Audio Player advances to the next song. 'Beautiful Song' is exactly 5 secs long. The bot is waiting for it to finish before the test can check the "advance". Surely the bots do not listen to the music, and won't mind if we "skip the boring bits" and move the playing song real close to the end of the song, and play from there ... [1] https://chromium-review.googlesource.com/c/chromium/src/+/1080667#message-ed1c278e6e0923fae82fe1649c336a0d615d2079
,
May 31 2018
FTR landed "OpenAudioFiles FilesAppBrowserTest: click repeat buttons with class" [1] to click the repeat buttons without using /deep/ combinator, issue 498405 . [1] https://chromium-review.googlesource.com/c/chromium/src/+/1080667
,
May 31 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3dec30816333b9c0f82beb2e27cd81afa7e74803 commit 3dec30816333b9c0f82beb2e27cd81afa7e74803 Author: Noel Gordon <noel@chromium.org> Date: Thu May 31 16:05:40 2018 OpenAudioFiles FilesAppBrowserTest: add track time leap forward helper Add a helper routine to move the current track foward in time (to ~90% of the track durataion for now). In follow-up, the helper will be used to speed up browser test-time by making audio track time leap forward to skip the boring (test) bits. No change in behavior, no new tests. Tbr: slangley, fukino-san No-presubmit: true Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: Ibf5d4afc7f90667935433dd71e2ae8303cb29607 Reviewed-on: https://chromium-review.googlesource.com/1080671 Reviewed-by: Noel Gordon <noel@chromium.org> Commit-Queue: Noel Gordon <noel@chromium.org> Cr-Commit-Position: refs/heads/master@{#563260} [modify] https://crrev.com/3dec30816333b9c0f82beb2e27cd81afa7e74803/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
May 31 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e59a1573fac0ff443a46b5c1cfadd4c7eaa26327 commit e59a1573fac0ff443a46b5c1cfadd4c7eaa26327 Author: Noel Gordon <noel@chromium.org> Date: Thu May 31 16:14:39 2018 Speed up OpenAudioFiles audioAutoAdvance browser tests Waiting for the next song takes 5 seconds because that is the duration of the current track, but that wastes bot-time. On slow bots like ASAN MSAN, multiply by a factor of 5 or 6, add Files.App start-up time etc, and the result is near/over the bot time-out => test flakes by design. Use audioTimeLeapForward() to "leap" the current track forward in time to 90% of its duration. This reduces browser test-time of this test by 25%. The last 10% of that track still plays, which allows this fixture to correctly test for "AutoAdvance" to the next audio track. No change in test behavior, just more speed. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: Ic5622c0de27e895d1532596f39d3b029b6b51b05 Reviewed-on: https://chromium-review.googlesource.com/1080672 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Stuart Langley <slangley@chromium.org> Cr-Commit-Position: refs/heads/master@{#563262} [modify] https://crrev.com/e59a1573fac0ff443a46b5c1cfadd4c7eaa26327/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
Jun 1 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/df24f5ba47cd43d780856ed2e2ab342dc1419f3d commit df24f5ba47cd43d780856ed2e2ab342dc1419f3d Author: Noel Gordon <noel@chromium.org> Date: Fri Jun 01 01:35:39 2018 Speed up OpenAudioFiles audioRepeatAllModeSingleFile browser tests Once the test track starts playing, click the "repeat-all" button, and leap forward in time to 90% of the track duration and let it play out. Then test the track gets repeated. Again, reduces browser test-time by 25% (fast is better than slow). No change in test behavior, just more speed. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: Iaeec0242a9aa1566a3e2b009335c03b0980e9150 Reviewed-on: https://chromium-review.googlesource.com/1080907 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Stuart Langley <slangley@chromium.org> Cr-Commit-Position: refs/heads/master@{#563475} [modify] https://crrev.com/df24f5ba47cd43d780856ed2e2ab342dc1419f3d/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
Jun 5 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c8cef1324056682b92349e98ab41983a4cf4e5f2 commit c8cef1324056682b92349e98ab41983a4cf4e5f2 Author: Noel Gordon <noel@chromium.org> Date: Tue Jun 05 11:09:18 2018 Speed up OpenAudioFiles audioNoRepeatModeSingleFile browser tests Once the test track starts playing, leap forward in time and let track play out, then test the Audio Player stops playing (no-repeat). Again, reduces browser test-time by 25%. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: I0fca8aee0f55a65d4fb8384fdc9bb8b640da0937 Reviewed-on: https://chromium-review.googlesource.com/1086527 Reviewed-by: Naoki Fukino <fukino@chromium.org> Commit-Queue: Noel Gordon <noel@chromium.org> Cr-Commit-Position: refs/heads/master@{#564441} [modify] https://crrev.com/c8cef1324056682b92349e98ab41983a4cf4e5f2/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
Jun 5 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/04051aa5f5f1c457c93c54299a3eb2e276901ae2 commit 04051aa5f5f1c457c93c54299a3eb2e276901ae2 Author: Noel Gordon <noel@chromium.org> Date: Tue Jun 05 11:35:33 2018 Speed up OpenAudioFiles audioRepeatOneModeSingleFile browser tests Once the test track starts playing, leap forward in time and let track play out, then test that Audio Player replays the track (repeat-once). Again, reduces browser test-time by 25%. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: Ic0918eb2b68b7b92cf8a80974e67cecae6bae7ab Reviewed-on: https://chromium-review.googlesource.com/1086551 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Naoki Fukino <fukino@chromium.org> Cr-Commit-Position: refs/heads/master@{#564446} [modify] https://crrev.com/04051aa5f5f1c457c93c54299a3eb2e276901ae2/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
Jun 5 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4cab6f35cdd60c27074a2cf59554428855e78636 commit 4cab6f35cdd60c27074a2cf59554428855e78636 Author: Noel Gordon <noel@chromium.org> Date: Tue Jun 05 11:56:06 2018 Speed up OpenAudioFiles audioRepeatAllModeMultipleFileDrive browser tests Once the first track starts playing, leap forward in time and wait for the track to play out. Test that Audio Player plays the next track. Once the next track is playing, leap forward in time again and let the track play out. Address TODO: check that the first track replays since repeat-all is active. Again, reduces browser test-time by 25%. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: I956943dde3523c97a7a6f2733a37878eeb5ba8f7 Reviewed-on: https://chromium-review.googlesource.com/1086667 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Naoki Fukino <fukino@chromium.org> Cr-Commit-Position: refs/heads/master@{#564455} [modify] https://crrev.com/4cab6f35cdd60c27074a2cf59554428855e78636/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
Jun 5 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5c5febb9f621ce1c7ddf2265b7794e1a1d2fb622 commit 5c5febb9f621ce1c7ddf2265b7794e1a1d2fb622 Author: Noel Gordon <noel@chromium.org> Date: Tue Jun 05 12:46:30 2018 Speed up OpenAudioFiles audioNoRepeatModeMultipleFile browser tests Once the test track starts playing, leap forward in time and let track play out, then test the Audio Player stops playing (no-repeat). Again, reduces browser test-time by 25%. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: I7fd2c0e7939d9d4e42e584b8aa9493019418de97 Reviewed-on: https://chromium-review.googlesource.com/1086673 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Stuart Langley <slangley@chromium.org> Cr-Commit-Position: refs/heads/master@{#564463} [modify] https://crrev.com/5c5febb9f621ce1c7ddf2265b7794e1a1d2fb622/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
Jun 5 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ad0186d02394c2aba2ae2fc23bc66fec9646aa5e commit ad0186d02394c2aba2ae2fc23bc66fec9646aa5e Author: Noel Gordon <noel@chromium.org> Date: Tue Jun 05 12:54:45 2018 Speed up OpenAudioFiles audioRepeatOneModeMultipleFile browser tests Once the test track starts playing, leap forward in time and let track play out, then test that Audio Player replays the track (repeat-once). Again, reduces browser test-time by 25%. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: Ie271e783a2a6977c4651c7ae669a5d1625cc750a Reviewed-on: https://chromium-review.googlesource.com/1086681 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Naoki Fukino <fukino@chromium.org> Cr-Commit-Position: refs/heads/master@{#564464} [modify] https://crrev.com/ad0186d02394c2aba2ae2fc23bc66fec9646aa5e/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
Jun 5 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f6736e6766a2e168d8e13bf334453da617b05616 commit f6736e6766a2e168d8e13bf334453da617b05616 Author: Noel Gordon <noel@chromium.org> Date: Tue Jun 05 12:59:47 2018 Address open_audio_files.js TODO: remove unused audioFileSystemURL audioFileSystemURL() is now unused: remove it. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: Ie30d8e3cea8ab2869a82beaa029fd05d798e990e Reviewed-on: https://chromium-review.googlesource.com/1086685 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Naoki Fukino <fukino@chromium.org> Cr-Commit-Position: refs/heads/master@{#564467} [modify] https://crrev.com/f6736e6766a2e168d8e13bf334453da617b05616/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
Jun 5 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ab415f8e575f5ca6a29a92f06994320c881f6182 commit ab415f8e575f5ca6a29a92f06994320c881f6182 Author: Noel Gordon <noel@chromium.org> Date: Tue Jun 05 16:47:44 2018 ES6 open_audio_files.js: ditch var, use let and const Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: I25c2100efce0f934660cc548a76ef5bb70565eef Reviewed-on: https://chromium-review.googlesource.com/1087190 Commit-Queue: Noel Gordon <noel@chromium.org> Reviewed-by: Stuart Langley <slangley@chromium.org> Cr-Commit-Position: refs/heads/master@{#564540} [modify] https://crrev.com/ab415f8e575f5ca6a29a92f06994320c881f6182/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
Jun 6 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8cfa4dad126839e107af244ebc4b1dcaa9f2fa73 commit 8cfa4dad126839e107af244ebc4b1dcaa9f2fa73 Author: Noel Gordon <noel@chromium.org> Date: Wed Jun 06 01:39:06 2018 OpenAudioFiles browser tests: add a getExpectedFileEntryRows helper Adding a file to a path's basic file entry set is common in these test fixtures: common enough to warrant a helper routine. Also don't bother sorting the row entries: remoteCall.waitForFiles does that. Bug: 847172 Cq-Include-Trybots: master.tryserver.chromium.linux:closure_compilation Change-Id: I529015eb9a5aa7fda59d9e33babee752875db5ab Reviewed-on: https://chromium-review.googlesource.com/1087189 Reviewed-by: Stuart Langley <slangley@chromium.org> Commit-Queue: Noel Gordon <noel@chromium.org> Cr-Commit-Position: refs/heads/master@{#564763} [modify] https://crrev.com/8cfa4dad126839e107af244ebc4b1dcaa9f2fa73/ui/file_manager/integration_tests/file_manager/open_audio_files.js
,
Jun 26 2018
OpenAudioFiles/FilesAppBrowserTest.Test/audioOpenDownloads is still flaky on MSan: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20ChromiumOS%20MSan%20Tests/7763
,
Jul 4
,
Jul 9
All green RELEASE/DEBUG/MSAN/ASAN https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=OpenAudioFiles%2FFilesAppBrowserTest.Test%2FaudioOpen |
||||
►
Sign in to add a comment |
||||
Comment 1 by noel@chromium.org
, May 28 2018