Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(545)

Issue 2810333008: A tool to evaluate methods in creating temp files (Closed)

Created:
3 years, 8 months ago by chengx
Modified:
3 years, 8 months ago
Reviewers:
stanisc, brucedawson
CC:
chromium-reviews
Target Ref:
refs/heads/master
Project:
chromium
Visibility:
Public.

Description

A tool to evaluate methods in creating temp files This "CreateTempFilesPerfEval" tool is to compare the time cost of creating temporary files between using GetTempFileName Windows API and using the GUID-based method, especially when there are already tens of thousands of temp files in the target directory. Sample results from a corp desktop (Windows 10 Enterprise, version 1511) are recorded in GetTempFileNamePerfExample.txt and GuidPerfExample.txt in this CL. It shows the time cost of creating every 500 temp files until there are 65535 temp files created in a directory for each method. The last row shows the time cost of creating the very last 35 temp files. Note that GetTempFileName can create at most 65535 files in a single directory. The sample results (the last row ommited) are also plotted in https://docs.google.com/spreadsheets/d/1j5nHneABic4-Ziz7KjjXPAbvr2aERyZB0qwbLYqyKGE/edit#gid=0 From the data/figure, we can see that GetTempFileName is working fine when the amount of temp files in the folder is under ~36000. It takes ~400 ms to create 500 temp files. However, 1) As the temp files accumulate to 36500, it performs ~4x worse; 2) As the temp files accumulate to 52000, it performs ~9x worse; 3) As the temp files accumulate to 60000, it performs ~15x worse; 4) As the temp files accumulate to 64500, it performs ~50x worse; 5) As the temp files accumulate to 65000, it performs ~110x worse; -- Now it takes 43+ seconds to create 500 temp files. 6) As the temp files accumulate to 65500, it performs ~1570x worse; -- It takes 44+ seconds just to create the remaining 35 temp files. In comparison, creating temp files using GUID-based method doesn't have this performance issue. It always takes 400 ms ~ 500 ms to create 500 temp files. The time complexity of GUID-based method for creating temp files is O(1) w.r.t the amount of existing temp files in the target directory. BUG=711534, 103737 Review-Url: https://codereview.chromium.org/2810333008 Cr-Commit-Position: refs/heads/master@{#465438} Committed: https://chromium.googlesource.com/chromium/src/+/4cb75df89a5fabfb4a83717cdc0f9289eb3b0596

Patch Set 1 #

Total comments: 7

Patch Set 2 : Use TEMP directory, delete all created temp folder/files automatically #

Total comments: 15

Patch Set 3 : Address comments #

Patch Set 4 : Fix evaluation of execution time for CreateFilesUsingGuid #

Patch Set 5 : Better printf to make the output more readable #

Unified diffs Side-by-side diffs Delta from patch set Stats (+780 lines, -0 lines) Patch
A tools/win/CreateTempFilesPerfEvaluation/.gitignore View 1 chunk +4 lines, -0 lines 0 comments Download
A tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc View 1 2 3 4 1 chunk +272 lines, -0 lines 0 comments Download
A tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.sln View 1 chunk +28 lines, -0 lines 0 comments Download
A tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.vcxproj View 1 2 1 chunk +153 lines, -0 lines 0 comments Download
A tools/win/CreateTempFilesPerfEvaluation/GetTempFileNamePerfExample.txt View 1 2 3 4 1 chunk +134 lines, -0 lines 0 comments Download
A tools/win/CreateTempFilesPerfEvaluation/GuidPerfExample.txt View 1 2 3 4 1 chunk +134 lines, -0 lines 0 comments Download
A tools/win/CreateTempFilesPerfEvaluation/ReadMe.txt View 1 2 1 chunk +55 lines, -0 lines 0 comments Download

Messages

Total messages: 104 (89 generated)
chengx
PTAL~
3 years, 8 months ago (2017-04-14 01:10:07 UTC) #10
chengx
Adding Stan. PTAL~
3 years, 8 months ago (2017-04-14 17:41:07 UTC) #16
stanisc
This is a great experiments that shows that we should consider switching to a GUID ...
3 years, 8 months ago (2017-04-14 18:46:00 UTC) #19
stanisc
https://codereview.chromium.org/2810333008/diff/80001/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc File tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc (right): https://codereview.chromium.org/2810333008/diff/80001/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc#newcode173 tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc:173: out += sprintf(out, "%.2hhX", id->Data4[i]); On 2017/04/14 18:46:00, stanisc ...
3 years, 8 months ago (2017-04-14 18:47:58 UTC) #20
chengx
On 2017/04/14 18:46:00, stanisc wrote: > This is a great experiments that shows that we ...
3 years, 8 months ago (2017-04-14 22:27:22 UTC) #31
brucedawson
This is very useful information, worth sharing. However: 1) You should put the conclusions in ...
3 years, 8 months ago (2017-04-14 23:07:24 UTC) #35
stanisc
https://codereview.chromium.org/2810333008/diff/140001/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc File tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc (right): https://codereview.chromium.org/2810333008/diff/140001/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc#newcode183 tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc:183: std::cout << i << " / " << count ...
3 years, 8 months ago (2017-04-14 23:30:55 UTC) #36
brucedawson
https://codereview.chromium.org/2810333008/diff/140001/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc File tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc (right): https://codereview.chromium.org/2810333008/diff/140001/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc#newcode228 tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc:228: FILE_ATTRIBUTE_DIRECTORY) { On 2017/04/14 23:30:55, stanisc wrote: > Do ...
3 years, 8 months ago (2017-04-14 23:36:15 UTC) #37
chengx
To Stan & Bruce: all code comments are addressed in the new patch set. To ...
3 years, 8 months ago (2017-04-17 19:00:57 UTC) #59
stanisc
https://codereview.chromium.org/2810333008/diff/140001/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc File tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc (right): https://codereview.chromium.org/2810333008/diff/140001/tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc#newcode183 tools/win/CreateTempFilesPerfEvaluation/CreateTempFilesPerfEval.cc:183: std::cout << i << " / " << count ...
3 years, 8 months ago (2017-04-17 22:36:10 UTC) #62
chengx
The evaluation of execution time is fixed for CreateFilesUsingGuid in the new patch set. Also, ...
3 years, 8 months ago (2017-04-17 23:12:16 UTC) #64
stanisc
lgtm
3 years, 8 months ago (2017-04-18 01:49:41 UTC) #83
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2810333008/260001
3 years, 8 months ago (2017-04-19 00:17:55 UTC) #100
brucedawson
lgtm
3 years, 8 months ago (2017-04-19 00:22:24 UTC) #101
commit-bot: I haz the power
3 years, 8 months ago (2017-04-19 00:36:59 UTC) #104
Message was sent while issue was closed.
Committed patchset #5 (id:260001) as
https://chromium.googlesource.com/chromium/src/+/4cb75df89a5fabfb4a83717cdc0f...

Powered by Google App Engine
This is Rietveld 408576698