13

The windows tool robocopy has a command line switch /NOOFFLOAD. What does this switch actually do, and under what circumstances would I use it?

The documentation for the switch says

/NOOFFLOAD :: copy files without using the Windows Copy Offload mechanism.

Further researching of the 'Windows Copy Offload mechanism' leads to this msdn document, but I have trouble understanding what effect disabling offload actually would have.

If a network file share is on a server which uses 'window copy offload', what noticeable benefits/drawbacks would using /NOOFFLOAD have on the copying process?
For example, would it increase or decrease the load on the server, or speed up the transfer at the cost of other resources?

13

In theory the load on the remote server will (by aggregate) be the same. It is the same amount being downloaded from one server and the same amount being uploaded to another, but the focus point of that load is shifted.

From your link:

enter image description here

By offloading the copy to the server it becomes the server itself that does the copy. For two servers in the same datacenter it can be a lot faster, as the copy can happen at the local datacenter link speed. If you have a 10mbps connection to both servers, but they have 10gbps local connections, then copying locally will be incredibly slow by comparison. It might cause a higher instant CPU or network load, but the transfer can happen faster. It might even be that the transfer is more efficient, due to lower latency caused by various network appliances in between you and the server.

As mentioned in that link by offloading the copy to the server itself you almost entirely remove your local machine from the equation:

A source file and a destination file can be on the same volume, two different volumes hosted by the same machine, a local volume and a remote volume through Server Message Block (SMB2 or SMB3), or two volumes on two different machines through SMB2 or SMB3

So it doesn't matter if it is the same server, or different servers, the copy operation will be a lot more efficient and will use whatever is available.

On a server or cluster with deduplication a copy might be effectively zero cost in terms of both CPU and disk time as no actual file data would need to be copied until changes were made. Only a file reference would need to be created. In this case using /NOOFFLOAD would be a massive waste of resources as you would download the data and re-upload it, while forcing the server to recheck and de-duplicate the data.

Disabling the offload will force the download to go through your machine. You will download the data from one server and upload it to the other. There are some situations where this might be desirable, particularly if you know that you are in between two servers and have a more effective link, perhaps with less routing or management appliances.

| improve this answer | |
  • In the graph, the actual copy does not take place on the servers but on the storage array, internally. Theoretically, this could be “ref copy” which is basically instant. I’m not sure if servers could copy over the local network with this technology. – Daniel B Jun 16 at 11:15
  • 2
    @DanielB good point, have added some words about local copies and deduplication. I suspect within a cluster that is deduplicated it could do a ref copy as you mention, but I doubt it would work for distinct separate servers or clusters. – Mokubai Jun 16 at 11:52
  • 2
    I think a more likely scenario for the use of the flag would be troubleshooting: if you are getting unexpected errors or other strange behaviour, you would use the flag to disable offloading, and if the issue goes away, you know where the problem lies, and if it doesn't, at least you've eliminated one possible cause. – Harry Johnston Jun 17 at 6:13
10

Offload is an optimization mechanism: when you're copying a file between two locations that are not on your machine, there's no need for your machine to see the data iff the locations can agree with each other to perform the transfer directly. This is an optional optimization, and will be used when possible. But /NOOFFLOAD disables it, so even if the optimization could be used - it won't be. It's basically a premature pessimization. There are very few reasons to actually use it. Sometimes it is insisted that /NOOFFLOAD will ensure that a copy of the data was actually performed, instead of e.g. hardlinking the file. Let it be clear that this is NOT the case, and using this option in lieu of backups is a fool's errand - often with a price tag attached.

In situations where offload mechanism is available, the storage is often deduplicated, and even though the data will make a roundtrip through your system, it'll do that only to reach the block deduplication filter that will discard the duplicate blocks. In other words: /NOOFFLOAD doesn't make anything better, but it can make it lots worse, with the end result being exactly the same - if you're lucky (read on).

The only use case I know for /NOOFFLOAD that is of any concern to technically astute but otherwise "regular" power users is network stress testing. If you want to put some load on the network and the storage system, /NOOFFLOAD will ensure you get the worst case possible under the circumstances in terms of the amount of data shuffled around. This isn't free though: the use of /NOOFFLOAD increases the probability of data corruption in the copy. There's no end-to-end data integrity protection in the Windows virtual filesystem, and file copies done without offload have measurable potential for bit-flips. This is something that hits marginal systems and networks especially hard. So if you think you want to do stress testing, do it using data you don't care about, i.e. delete the copies once they have synced to disk.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.