clear_system_cache is slow on windows |
||||||
Issue descriptionThe clear_system_cache.exe is really slow when running on windows (about few minutes by run) The way the cache is cleared got fixed here: https://bugs.chromium.org/p/chromium/issues/detail?id=527254 https://chromium.googlesource.com/chromium/src/+/f90a68046a970ba841c532b781992681ee0aaab9 Unfortunately, the default executable provided with chrome is still the old one. Which is quite slow for a chrome developer. The sha1 of the file to be downloaded seems to be the old one. The old way to clear the cache was to copy the files. Telemetry is clearing the cache for the temporary folder and for the chrome output directory (recursively). Which means every files (even the huge PDB files and temporary objects files) are copied. The new way to clear the cache is to change the modification dates. 1) We should use the new executable 2) We should not clear the cache for every files under out\build More details: https://drive.google.com/open?id=1Vrl2Ck3wcLJM_A-DZyGC3AdufxlaSC-T_Ot8zlapv1k Probably related bug: https://bugs.chromium.org/p/chromium/issues/detail?id=682829 https://bugs.chromium.org/p/chromium/issues/detail?id=702669
,
Mar 30 2017
,
Mar 30 2017
I think this script is not specific to Windows, right? What about Mac & Linux? Also how would we test the effect of clearing system cache, otherwise I worry that the refactoring may make it no longer does what it supposed to do.
,
Mar 30 2017
The executable is here: https://cs.chromium.org/chromium/src/tools/perf/clear_system_cache/clear_system_cache_main.cc?sq=package:chromium&dr It's using the |EvictFileFromSystemCache| from base: https://cs.chromium.org/chromium/src/base/test/test_file_util_linux.cc?sq=package:chromium&dr&l=17 On windows: https://cs.chromium.org/chromium/src/base/test/test_file_util_win.cc?sq=package:chromium&dr&l=107 Speed is apparently fine on other plateform. There is no refactoring to do. The fix is already landed years ago. As I get it, this should be changed and the executable uploaded to the drive. https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/bin/win/AMD64/clear_system_cache.exe.sha1?q=clear_system_cache&dr
,
Mar 30 2017
Thanks, based on the implementation, it looks like the posix platform has some primitive to flush & free cached page for a file, which make it much faster compared with copying the file way on Windows. I think 1) is a definitely must do. Though it's unclear whether we should still do 2) if running the script on out\build is fast enough. Wouldn't doing 2) risk making browser cold startup not "fully cold" due to us not clear all the browser file cache? To update the clear_system_cache binary in Telemetry, the actual way is updating the entry in https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/internal/binary_dependencies.json?rcl=94d1bb9f4ab0a7fbbabe5018ffa0578b054fa89c&l=49 I think https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/bin/win/AMD64/clear_system_cache.exe.sha1?q=clear_system_cache&dr is just legacy way of doing it which should be removed by now.
,
Mar 31 2017
Randy: can you take ownership of upgrading this script in telemetry binary_dependencies?
,
Mar 31 2017
I agree with #5 let do 1) and then, I'll check the speed of the script and we can decide whether or not to do 2)
,
Apr 5 2017
Is there some progress on that issue?
,
Apr 5 2017
Sorry, this fell off my plate a little bit. I do not have a windows machine with a chromium checkout on it. If someone can build the executable for me, and send it to me I will gladly update the file in the deps manager. If it requires me to get a windows checkout I probably wont have time to do this until next week.
,
Apr 6 2017
CL is up here: https://codereview.chromium.org/2800803003/
,
Apr 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9b373d62207990ea3702931b04a622840c1af9b1 commit 9b373d62207990ea3702931b04a622840c1af9b1 Author: catapult-deps-roller <catapult-deps-roller@chromium.org> Date: Thu Apr 06 18:08:28 2017 Roll src/third_party/catapult/ 80a58afe7..3680efbfb (1 commit) https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/80a58afe727a..3680efbfbcc7 $ git log 80a58afe7..3680efbfb --date=short --no-merges --format='%ad %ae %s' 2017-04-06 rnephew [Telemetry] Update windows clear_system_cache executable. Created with: roll-dep src/third_party/catapult BUG= 706906 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, see: http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel TBR=catapult-sheriff@chromium.org Review-Url: https://codereview.chromium.org/2799933002 Cr-Commit-Position: refs/heads/master@{#462545} [modify] https://crrev.com/9b373d62207990ea3702931b04a622840c1af9b1/DEPS
,
Apr 6 2017
If the change is effective, we should see a drop in https://chromeperf.appspot.com/report?sid=b47746886cc2c454c7532a619f558c242492196a48768ec0b8bb945d8bf9307a
,
Apr 7 2017
Yup, we are seeing a huge improvement from 5.6 minutes to .8 minutes! This tells me that we probably don't need to do 2) of #0. tienneb@: if you disagree with my assessment, please feel free to reopen this bug. Thanks Randy for pushing this through!
,
Apr 7 2017
I'm fine with this for now.
,
Jan 16
,
Jan 16
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by erikc...@chromium.org
, Mar 30 2017