Copying large number of files accross home network - crashing

Question

I'm trying to copy a number of files from one machine to another on a wireless home network. There are around 5,000 files, each roughly 3-5 MB (mp3s).

My problem is that the network keeps crashing - both machines report being disconnected and my router needs to be manually reset.

Copying is done using the following command line statement:

xcopy \\My-PC\converted C:\Music

After some 20 files are copied the network goes down.

I assume this is a limitation/fault of the network, not being able to handle a such a large load all at once. If that's the case, is there a work around? Possibly set a transfer speed limit?

Master machine: Windows 7

Client machine: Windows Vista

Answer 1

Other answers are nice and useful but they address the symptoms and not the root of the problem. 15 GB network transfer between two computers on a home network is not a heavy load and any switch/router must handle this without any problem. If your router consistently crashes in the middle of it, your router is faulty and needs to be either fixed or replaced.

How to troubleshoot

The only three issues you can easily fix are these:

1. Overheating: if your router is hot to the touch, it may be overheating. If it has fans inside, check that they are working. Use compressed air can to blow the dust out if it, if it appears to be dusty. If it has insufficient air circulation (in a closet, etc.), open a closet door and see if that makes any difference, etc.

2. Faulty connection or network port. If part of your network is wired, and the router have several network ports which are not used, try moving a wired network connection to a different port (not applicable to WAN connection that goes to your internet modem, if you have one).

3. Faulty power supply. If the router has an external power supply (wall wart, etc.), and you have a spare one with the exact same voltage, max. power, and connector shape and polarity, try using it and see if it helps.

If these steps do not help, I suspect the most cost-effective solution would be to get a new router.

Answer 2

I'm not certain if you are open to 3rd party software as a solution but there is a utility that seems like it's a perfect fit to the task. It sets up a resumable file transfer queue.

FF Copy. http://www.ffprojects.net/ffcopy/

A more native approach might be robocopy, especially if you're comfortable using command line utilities.

http://technet.microsoft.com/en-us/library/cc733145(WS.10).aspx

http://ss64.com/nt/robocopy.html

  ROBOCOPY Source_folder Destination_folder [files_to_copy] [options]

these switches in particular:

            /Z : Copy files in restartable mode (survive network glitch).
            /B : Copy files in Backup mode.
           /ZB : Use restartable mode; if access denied use Backup mode.
        /IPG:n : Inter-Packet Gap (ms), to free bandwidth on slow lines.

          /R:n : Number of Retries on failed copies - default is 1 million.
          /W:n : Wait time between retries - default is 30 seconds

Answer 3

Maybe you can try 'Microsoft Richcopy' a free tool from Microsoft.

Similar functionality to Robocopy but with a GUI. Many parameters can be configured. Transfers can be paused and resumed.

Answer 4

I'm guessing that xcopy somehow holds the directory open on the remote computer, and repeated references to the directory cause "garbage" to accumulate in the network management software, only released when the xcopy finishes.

So you need some sort of software that does more of a file at a time, or, at most, small bursts of files. There are probably freeware tools to do this, or you can easily write your own in Java or whatever.

The other option would be to zip up the files and transmit them as one large file.

Answer 5

On large copy operations it is always good to have the possibility to resume after the process has been interrupted. And also check the state of the parts that have already been transfered.

I suggest you use a different tool for this.

One easy soulution would be to download the free evaluation copy of TotalCommander http://www.ghisler.com/download.htm and use its "Synchronise Directories" feature. I have used it a lot for moving and copying mp3s around the LAN or between partitions and it is really easy and safe:

It allows you to have 2 windows side by siode and compare the already copied files and continue copying the rest. At the end you can do a size based check or even a content based check (check the "by content" option), to see if everything went fine. I suggest doing the content check only once in the very end, because it could take some time.

I use this gui based tool a lot on such occasions because it can handle interruptions, errors and because at the end you can be sure all has been copied 1:1

Other possible tools are the command line based "robocopy" from microsoft. There is also the commandline based "rsync" which is very good at such tasks, but it is from the linux world and I have preferred robocopy over it on windows.

There are other gui tools like Beyond Compare or WinMerge, here is a list:

http://alternativeto.net/software/winmerge/?platform=windows

If you still think you are having a network problem afterwards, let me say I think it is very hard to track down a network related problem like yours without having real access to the machine. Could be anything. You could reduce the setup step by step and see at which point the problem disappears (Remove other machines from the lan or switch them off, remove switches and cables between the 2 machines and replace them with cables that work, switch off and/or uninstall the firewalls (be careful then) and other additional software on both machines one by one. In the end connect the machines directly or over a router that you know works. You can exchnge the network cards or add additional network cards and use those to find out if a broken nic is the source of the error. You could also boot the one or the other system from a live CD like ubuntu or knoppix live cd and try heavy copy operations with them. They can integrate into your existing LAN.

Remember to always only change ony part of your setup and repeat your tests until yoou know if the error is still there. So you can encircle and track down the source of the problem.)

Answer 6

I recently also had to transfer a lot of files over the wireless with the same issue, the solution I had to use was to simply use a wired connection (the 'server' was a network disk I moved to my computer) I got a 10 fold increase on the transfer rate

you should be able to do the same thing but you might need to change the IP settings so they would recognize each other though, turn of the wireless connection on one and set the other with a shared internet connection

or use a FTP client that lets you set a transfer limit (Filezilla being one)