0

I wrote a script that checks to see if a folder exists, and if so, empties it. it then waits 5 seconds, then proceeds to recopy new (or maybe the same) files back into the folder.

When the folders don't already exist, it'll take 15-20 seconds to copy the files. but when the folders already exist, I can watch them being emptied, I see it wait, then I see it zip through the copy commands in less than 1 second. And the files show up in the list.

Can someone explain this to me? It would almost appear that the copy is actually looking to see if the file HAD existed, and if so, if it was somehow identical, it just restores the file rather than recopy it.

I haven't tested the "restore" theory, but wanted to know if this was even probable (it's scary either way).

I added the 5 second pause thinking that maybe the drive needed time to stabilize.

Thanks in advance.

Example of script:

@echo off

if exist c:\some\folder erase /q c:\some\folder
erase /q C:\another\folder\file*.exe
ping -n 6 -w 1000 127.0.0.1 > nul

copy /y some\folder\sample.exe c:\some\folder\sample.exe
copy /y another\folder\file001.exe c:\some\folder\file001.exe

make your test files large so they take time to copy, and you'll see what I mean.

  • Are you saying it's working too fast for your liking? I'm not sure what the question is. Can't you just check the file and see if it changed as you expected? What are you testing? – MC10 Jul 31 '15 at 1:34
  • No, I'm saying if it takes 15 seconds to copy it the first time, why doesn't it take 15 seconds the second time, after the files have been deleted. – jnewman Jul 31 '15 at 1:35
  • by the way, I replaced "erase" with "rmdir /s" and it does the same thing. – jnewman Jul 31 '15 at 1:36
  • the goal is to update a folder with a new set of files, and keep it clean. It will frequently be the case that the new files will be identical to the old files, but just in case, all files are removed and new files copied in to prevent buildup. – jnewman Jul 31 '15 at 1:39
  • 1
    you have relative paths eg some\folder\sample.exe what directory are you in there? and re what is happening, perhaps another possibility is a cache somewhere maybe hdd designed but you need to test erasing the file properly with an erasing properly program – barlop Jul 31 '15 at 1:55
2

Sounds like you are witnessing the filesystem cache doing its job.

First of all, when you copy a file the computer must read that file into memory before it can write it back out to the copy. Reading that file represents a time cost, so the in-memory copy is kept around in the cache in case you need it again. Like if you open that file you just copied, or you copy it somewhere else. The OS does not have to waste time reading it from the disk again.

Second of all, the cake is a lie. When a file gets "written", it does not actually go to the disk. It goes instead to a memory buffer which then gets committed to the disk at some later time (usually immediately but not necessarily -- it depends on how busy the disk is). Either way, the OS reports that the file has finished copying when in fact it has not.

Both read caching and write buffering are optimization techniques that all modern operating systems use. In your case (assuming nothing went wrong and the files are identical), your script is hitting a sweet spot. The file(s) merely got copied from one block of cache memory to another block of buffer memory and that's why it didn't take very long.

Incidentally... this is why you should never turn a computer off without properly shutting it down or remove a flash drive without unmounting it. Part of that process is to commit the write cache to the disk.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.