2

Troubleshooting a tricky performance problem and wondering if this behavior is normal/expected.

This seems to be related to SMB2, as switching to SMB1 resolves things. The question is whether this is expected (i.e. design of SMB2) and, if so, is there a solution/workaround?

Here’s a simplified version of the problem:

1.Connect two Windows 7 machines via crossover cable.

2.Start a large file copy.

3.While that copy is underway, start a small file copy. The second copy process is very slow.

4.However, if you instead connect to the other machine “another” way (i.e. via IP instead of FQDN, or via Netbios name instead of IP, etc.), copy performance is relatively good.

It appears that the large copy is “bottlenecking” the original connection – i.e. as soon as you pause the large copy, the small copy finishes quickly.

It’s not saturating the bandwidth/stack completely though (since the second copy via a "different connection” finishes quickly).

How to reproduce the problem:

I've created some batch files that demonstrate this (all the code is listed at the bottom of the question – you need to create 5 separate batch files called go.bat, test.bat, copy1.bat, copy2.bat, copybig.bat):

Setup:

  1. Create test directory on remote ComputerB (c:\test)
  2. Create test directory on local ComputerA (c:\test)
  3. Put the 5 test batch files in ComputerA’s c:\test directory
  4. Edit go.bat and modify the last line as follows: The format is “call test.bat 1 2 3”, where: 1 = remote path via hostname 2 = local folder 3 = remote path via IP (e.g.: call test.bat \\ComputerB\c$\test c:\test \\ComputerB’sIPAddress\c$\test)

Execution:

  1. Open a command prompt as an administrator (so it can create the empty test files) and run go.bat

Here’s what the batch files do:

-Checks ComputerA to see if the three files exist (1GB, 10MB-1, 10MB-2) . If the files aren’t there, creates them.

-Deletes the local log file and ComputerB test files

-Starts copying the 1GB file from ComputerA to ComputerB

-Waits two seconds

-Starts copying the 10MB-1 file from ComputerA to ComputerB

-Starts copying the 10MB-2 file from ComputerA to ComputerB (but using its IP address)

-Logs, with timestamps, what’s happening to log.txt

Results:

What seems to happen consistently is this:

-The big 1GB file copy establishes a connection with the remote machine

-The small 10MB-1 file copy starts and eventually finishes but is really slow because it’s “sharing” the existing connection with the 1GB copy connection

-The small 10MB-2 file copy that uses the IP address finishes quickly because it establishes a “new/separate” connection

-At any point, if you hit pause during the big copy, the small 10MB-1 copy finishes quickly.

Batch file code:

--go.bat--
cls
rem The format is test.bat 1 2 3, where:
rem 1 = destination path with hostname
rem 2 = local path
rem 3 = destination path with IP

call test.bat \\ComputerB\c$\test c:\test \\ComputerB'sIP\c$\test
--go.bat--


--test.bat--
if exist %2\1GB goto 1
fsutil file createnew 1GB 1048576000

:1
if exist %2\10MB-1 goto 2
fsutil file createnew 10MB-1 10485760

:2
if exist %2\10MB-2 goto 3
fsutil file createnew 10MB-2 10485760

:3

del /q /f log.txt
del /q /f %1\10MB-1
del /q /f %1\10MB-2
del /q /f %1\1GB

set var1=%1
set var2=%2
set var3=%3

echo Calling CopyBig.bat %time% >> log.txt
start CopyBig.bat %var1% %var2% %var3%

echo Pausing for two seconds %time% >> log.txt
ping 127.0.0.1 -n 3

echo Calling Copy1.bat %time% >> log.txt
start copy1.bat %var1% %var2% %var3%

echo Calling Copy2.bat %time% >> log.txt
start copy2.bat %var1% %var2% %var3%
--test.bat--


--copybig.bat--
echo Starting Big Robocopy %time% >> log.txt
robocopy %2 %1 1GB
echo Done Big Robocopy %time% >> log.txt
--copybig.bat--


--copy1.bat--
echo Starting 1st Robocopy %time% >> log.txt
robocopy %2 %1 10MB-1
echo Done 1st Robocopy %time% >> log.txt
--copy1.bat--


--copy2.bat--
echo Starting 2nd Robocopy %time% >> log.txt
robocopy %2 %3 10MB-2
echo Done 2nd Robocopy %time% >> log.txt
--copy2.bat--
5
  • The usual preliminary question: Have you ensured that all network drivers are up-to-date according to Windows Update and the manufacturer's website, and that Windows is fully patched?
    – harrymc
    Jun 5, 2014 at 19:00
  • SMB2 has some buggy issues as I'm experiencing several issues like this when utilizing SMB2.
    – Travis
    Jun 5, 2014 at 20:14
  • Good question @harrymc - yes, have tried updating NIC drivers and the machines are well patched.
    – Craig H
    Jun 6, 2014 at 22:04
  • Have you tried the original xcopy (for comparison)? Also, have you tried playing with the Interpacket Gap? See here for an interesting article which may shed some light. Jun 11, 2014 at 19:40
  • Same issue with copy, xcopy or Windows explorer. Have tried lots of tuning and tweaking, with no noticeable improvement. Baffled.
    – Craig H
    Jun 12, 2014 at 19:57

2 Answers 2

0

Note: I haven't tried the batch files, but instead looked at details of how Robocopy works.

Per Mark Minasi's blog, setting the IPG flag will have a dramatic effect of how much bandwidth is used by robocopy. He states:

Waiting Between Blocks Finally, with the /ipg option, Robocopy lets me reduce the amount of network bandwidth the tool can consume. Like all copy routines, Robocopy copies data in 64KB blocks. Typically, Robocopy just sends one block after another, with no pause between the blocks. Adding the /ipg:n option, however, tells Robocopy to wait n milliseconds between blocks. But what value do you use? Well, 64KB is about half a million bits. Therefore, over a 100Mb Ethernet connection, that 64KB file copy would take about 1/200 of a second—5 milliseconds.

Five milliseconds is a theoretical best-case figure. In a busy real-world network with normal software overhead, you might expect no more than 20 percent of that speed, making the actual time perhaps 25 milliseconds. Assuming, then, that Robocopy can effectively transfer only one block every 25 milliseconds, setting /ipg:25 would cut Robocopy's network usage by 50 percent. Setting /ipg:50 would cut Robocopy's network usage by 66 percent, and setting /ipg:75 would reduce network usage by 75 percent of its unencumbered speed. You get the idea.

You can change the copybig.bat to:

--copybig.bat--
echo Starting Big Robocopy %time% >> log.txt
robocopy /IPG:750 %2 %1 1GB 
echo Done Big Robocopy %time% >> log.txt
--copybig.bat--

I chose 750 because it should roughly limit it to 1Mbps. You can use the Robocopy IPG delay calculator to figure out what delay is best for your setup.

Another option to try is the /mt flag (for multithreading) -- but be careful as this will not work with /ipg.

1
  • Thanks-unfortunately it's not just a robocopy issue. I used that to demonstrate things in a repeatable, scriptable way. The same issue occurs if you copy a large file with Windows Explorer & then try open a small file from the same share (i.e. until the large copy is done, additional SMB connections are very slow). At this point, it looks like a fundamental Windows 7/SMB2 problem - the issue seems to be there on fresh W7 builds. It's hard to believe this is "normal" behaviour (i.e. I would expect that there would be more people complaining about this) but perhaps that's just the way it is!
    – Craig H
    Jun 12, 2014 at 19:56
0

I know that this may seem like a cop-out; however, since you say that if the IP is entered in then the copy goes just fine therefore on the computer running this batch file you can try to modify the HOSTS file (C:\Windows\System32\HOSTS) and add x.x.x.x hostname for some of the hosts that you are trying.

1
  • Thanks - I'd try anything at this point, but unfortunately the hosts solution won't solve the issue. The root issue isn't name resolution or how it connects. No matter which method you use to connect to the destination (i.e. via IP or FQDN or Netbios name), any additional connections made the same way will be slow - and any additional connections made a different way will be fast.
    – Craig H
    Jun 12, 2014 at 19:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .