Highspeed data sharing between two computers

Question

I often need to move/access data (several GB at a time) between my Laptop and Desktop computers (it's on the laptop for mobility, and desktop so it can process it in 10 minutes rather than 2 hours).

The local network is only 100Mbps and gives me around 500KB/s. I tried linking them with an Ethernet crossover cable (1Gbit) but this only hit 20MB/s (I calculated that it should be up around 120MB/s...) which is still not that great and it prevents the desktop accessing the internet or the rest of the network while it is at it (only one Ethernet port).

Why is the crossover cable so much slower than I expected? What other options do I have for moving/accessing/etc. stuff?

Available Ports:
Laptop: 1Gbit Ethernet, USB 2.0.
Desktop: 1394a, eSATA 3.0Gb/s, USB 2.0, USB 3.0

I can't use a local router to connect both computers with 1Gbit as their disallowed on the network :(

Answer 1

I can't use a local router to connect both computers with 1Gbit as their disallowed on the network :(

As a professional network adminsitrator, I have to agree with that rule. However, as a short-term option you could add a network switch and be just fine.

A switch means your devices are still part of the same business network, so your admin can see them and manage them correctly. It won't interfere with any wireless access points that might be deployed or inadvertently open your internal network up to outside wifi users. It will also switch packets directly between your laptop and desktop at gigabit rates such that the data is never sent to the rest of the network (unlike a hub). In networking terms, you want a layer 2 device, not layer 1 or 3. That means no hubs and no consumer-level routers.

I'm talking about something like this. It should be okay, as it's not a router:
http://www.newegg.com/Product/Product.aspx?Item=N82E16833156250

Note that this is not an endorsement of that specific item. I've never used that model. It's just an example and I didn't shop around that much. But it is the type of device I'm talking about. Whether that will be able to outperform something like a direct firewire or usb2/3 link is another matter.

Answer 2

Your bottleneck is disk I/O (input/output, aka read and write). Plattered hard drives normally max out between 20-50 MBps. You can CONCEIVABLY transfer data faster, but if you are sending a large file, you can only send as fast as the drive can access it, and only as fast as the SLOWER of the two drives can either read or write.

Answer 3

Why not get an fast external hard disk that has a eSATA and a USB connection, then put the data on the hard disk. Just plug the hard disk into the machine that needs access to the data.

Answer 4

I would like to offer some ideas to get better transfer of low-medium amounts of information between PC-PC laptop-PC.

Factors to consider:

• The info in each HDD is stored and read in segments according with the cache in the HDD (higher cache = higher performance) of fix size according with the options (file system type and block size) selected when the disk is formatted at a rate of speed according with the RPMs of the HDD (higher RPM = higher speed) and transmitted through the port of connection (for low cost PC's speeds SATA 3 > SATA 2 > SATA 1 > PATA/IDE).

• Any process to the information during its transfer make it slower: buffers, port, media, file system (PC source/PC target), cryptography, antivirus check, etc.

• Quality of the media: networking cable have a specification base (cat 3, cat 5, cat 5e, cat 6m, cat 6e) but the speed limit (the rest of info about the wire that we never read) is the key (over 500 MHz is ok for a Ethernet passcord crossover) so 6 ft (1.8 m) of a good crossover passcord could save hours in the process.

So consider apply this, order by cost - speed....

1. To prepare the PCs for quick transfer:

   Create a data partition in yours HDDs (source - target), maybe you had to reduce the main partition to accomplish this task. For the new partition select the most efficient (usually the newest) file system, higher block size (as similar as possible in both PC in order to reduce process over data, but in concordance with the the HDD's physical cache specification).

   Pros:

   • Cost = 0$
   • Impact = high
   • Technical level required = low
   • Stability: higher (if an independent data partition gets full the OS it's not so affected)
   • Independent cache: A different partition have its own cache in the operating system
   • Avoid some traffic issues with the memory and process (of the OS's kernel)
   • Reduced processing of the information: this partition can be excluded of the antivirus check, Bitlocker, crypting, user rights validation, étc, during the transfer process.
   • Make easier the transfer to different or similar operating systems (by extracting the source HDD from the source PC and directly connecting it to the target PC)
   • Left the hardware to reach its potential with minimal software interference

   Cons:

   • Data Lost probability = medium if we work without external backup / UPS
   • Hardware lost probability = low (no hardware changes involved)

2. Reinstallation from the scratch your operating systems in both source PC - target PC selecting newest and similar file systems with the higer block size available in the list during format. Before start with the OS installation, update the BIOS in each PCs and selecting overclocking options in BIOS Setup to the most stable option but with the HDD data port in most advanced option (for HDD in SATA port; select the most advanced or native option, for HDD in IDE port; activate if its supported bus mastered 32-bit, LBA, etc) in any case S.M.A.R.T. option reduce speed but provide some mind peace, avoiding transfer data from/to a defective HDD. Remember to have on hand all required drivers according with the operating system and follow the first recommendation about the data partition and formatting process.

   Pros:

   • Cost = 0$
   • Impact = medium
   • Technical level required = medium
   • Stability: good
   • Independent cache: a different partition have its own cache in the operating system
   • Avoid some traffic issues with the memory and process (of the OS's kernel)

   Cons:

   • Data lost probability = high, it's mandatory to work with external backup / UPS
   • Hardware lost probability = medium (bad BIOS update will destroy your motherboard)
   • BIOS update is a dangerous process but may improve stability and performance of the motherboard and devices so (PRTFM) please read the original manual of the motherboard and the user forums about the BIOS update process and its known effect over devices (motherboards, memory, HDD, ports, etc) and its impact over operating systems.

3. The right media for the transfer

   • cost 0$: plug in the laptop HDD directly in the PC's motherboard (or the sata-data cord maybe)
   • A small piece of Good Networking Cable for crossover (cat 6e) 500 MHz and the right RJ-45 connectors.