Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a text file that has ~500,000 full paths. I want to copy them to a NAS once. It's about 1TB of data. Should I make a zip file with all the files or copy them individually? How can I do either automatically?

share|improve this question

1 Answer 1

up vote 2 down vote accepted

You can do this with a For statement command from the command prompt or a batch script

For /f %f in (list.txt) do xcopy "%f" \\NasName\dest [switches]

Typical switches:

  • /C Continues copying even if errors occur.
  • /I If destination does not exist and copying more than one file, assumes that destination must be a directory.
  • /H Copies hidden and system files also.
  • /R Overwrites read-only files.
  • /Y Suppresses prompting to confirm you want to overwrite an existing destination file.

Do the above command and walk away for a few hours. If the path's are relitive path's in your list file replace %f with C:\path\to\source\%f

Don't forget you will need privileges to read the file to copy them so if any of your files are in restricted folders (like other users Document's folders) you will need to run the command prompt/batch file as Administrator. Also if you run this in a batch file you will need to replace the single % with %%


If you list of 500,000 files are just all the files under c:\somepath you can just use Robocopy instead of xcopy + for statement

robocopy c:\somepath \\NasName\dest [switches]

I would recommend the following switches

  • /s Copies subdirectories. Note that this option excludes empty directories.
  • /zb Uses Restart mode. If access is denied, this option uses Backup mode.
  • /MT Creates multi-threaded copies (Works only on Server 2008 R2 and Windows 7)
  • /LOG:<logfile> Log the results to a file instead of the console (this will boost performance of /MT)
share|improve this answer
    
Thanks for your help. I forgot to ask about this so does your solution maintain the directory structure of the paths on the NAS? And there are spaces in the paths but they are not " wrapped. Is that addressed in the "%f" part or should I do that before running this script? Also even though its a 1Gb/s card & switch the NAS seems to be limited to 10MB/s. IF thats the case would it sway you towards making the zip first? It'll be about 30hrs to transfer –  Paul DeBruicker Aug 1 '12 at 10:35
    
1) The for does not re-create the directory structure, but robocopy does. 2) The " in the "%f" are wrapping the path's with quotes. 3) 10MB/s is not the greatest transfer speed but it is still decent. You are likely limited by a slow hard drive somewhere 4) If you zip it first, do you plan on leaving it zipped on the nas or how to you plan to un zip it (If you unzip it over the network you are transfering the uncompressed file over the wire anyway during the unzipping process and you are just adding extra time) –  Scott Chamberlain Aug 1 '12 at 13:34

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.