Performance evaluation/comparison for creating temp files using GetTempFileName API and GUID-based method |
|||
Issue descriptionAs documented in https://msdn.microsoft.com/en-us/library/windows/desktop/aa364991(v=vs.85).aspx "Due to the algorithm used to generate file names, GetTempFileName can perform poorly when creating a large number of files with the same prefix. In such cases, it is recommended that you construct unique file names based on GUIDs." Because of this, I have recently updated CreateTemporaryFileInDir() from using GetTempFileName() to using GUID-based method, as in crrev.com/2788483005/. However, the performance gain is unclear. We need to evaluate and compare these two methods.
,
Apr 17 2017
,
Apr 19 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4cb75df89a5fabfb4a83717cdc0f9289eb3b0596 commit 4cb75df89a5fabfb4a83717cdc0f9289eb3b0596 Author: chengx <chengx@chromium.org> Date: Wed Apr 19 00:36:18 2017 A tool to evaluate methods in creating temp files This "CreateTempFilesPerfEval" tool is to compare the time cost of creating temporary files between using GetTempFileName Windows API and using the GUID-based method, especially when there are already tens of thousands of temp files in the target directory. Sample results from a corp desktop (Windows 10 Enterprise, version 1511) are recorded in GetTempFileNamePerfExample.txt and GuidPerfExample.txt in this CL. It shows the time cost of creating every 500 temp files until there are 65535 temp files created in a directory for each method. The last row shows the time cost of creating the very last 35 temp files. Note that GetTempFileName can create at most 65535 files in a single directory. The sample results (the last row ommited) are also plotted in https://docs.google.com/spreadsheets/d/1j5nHneABic4-Ziz7KjjXPAbvr2aERyZB0qwbLYqyKGE/edit#gid=0 From the data/figure, we can see that GetTempFileName is working fine when the amount of temp files in the folder is under ~36000. It takes ~400 ms to create 500 temp files. However, 1) As the temp files accumulate to 36500, it performs ~4x worse; 2) As the temp files accumulate to 52000, it performs ~9x worse; 3) As the temp files accumulate to 60000, it performs ~15x worse; 4) As the temp files accumulate to 64500, it performs ~50x worse; 5) As the temp files accumulate to 65000, it performs ~110x worse; -- Now it takes 43+ seconds to create 500 temp files. 6) As the temp files accumulate to 65500, it performs ~1570x worse; -- It takes 44+ seconds just to create the remaining 35 temp files. In comparison, creating temp files using GUID-based method doesn't have this performance issue. It always takes 400 ms ~ 500 ms to create 500 temp files. The time complexity of GUID-based method for creating temp files is O(1) w.r.t the amount of existing temp files in the target directory. BUG= 711534 ,103737 Review-Url: https://codereview.chromium.org/2810333008 Cr-Commit-Position: refs/heads/master@{#465438} [add] https://crrev.com/4cb75df89a5fabfb4a83717cdc0f9289eb3b0596/tools/win/CreateTempFilesPerfEvaluation/.gitignore [add] https://crrev.com/4cb75df89a5fabfb4a83717cdc0f9289eb3b0596/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc [add] https://crrev.com/4cb75df89a5fabfb4a83717cdc0f9289eb3b0596/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.sln [add] https://crrev.com/4cb75df89a5fabfb4a83717cdc0f9289eb3b0596/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.vcxproj [add] https://crrev.com/4cb75df89a5fabfb4a83717cdc0f9289eb3b0596/tools/win/CreateTempFilesPerfEvaluation/GetTempFileNamePerfExample.txt [add] https://crrev.com/4cb75df89a5fabfb4a83717cdc0f9289eb3b0596/tools/win/CreateTempFilesPerfEvaluation/GuidPerfExample.txt [add] https://crrev.com/4cb75df89a5fabfb4a83717cdc0f9289eb3b0596/tools/win/CreateTempFilesPerfEvaluation/ReadMe.txt
,
Apr 20 2017
Nice analysis!
,
Apr 21 2017
|
|||
►
Sign in to add a comment |
|||
Comment 1 by chengx@chromium.org
, Apr 14 2017