2,867 reputation
716
bio website scottmcclenning.blogspot.com
location Oxnard, CA
age 40
visits member for 4 years, 6 months
seen Jun 27 at 5:00

Jun
20
comment Brand new HDD acts very slow
I was wondering what version of Windows, and Model of hard drive. Also, how much RAM does the machine have, because if sleep mode actually shutdown the hardware, you are basically doing an awake from hibernate. If you have a lot of RAM, it has to load it all in. If you had a bad or under rated cable (SATA1, vice what you need), all disk access would be slow (e.g. transferring large files to/from the disk).
May
18
comment ZFS + dedup: copy many small files fast
Something else that may help is if the two servers you are transferring between both have ZFS, and you can use snapshots, you may be able to use ZFS Send and ZFS Receive to transport the snapshot(s) from one zpool to another, even between machines. In that case, rsync wouldn't be required for the transfer.
May
18
awarded  Custodian
May
18
comment ZFS + dedup: copy many small files fast
I too believe snapshots could save you from needing to do the copy for backup purposes (and should be much faster). That said, the copy speed does seem slow. ZFS takes a lot of CPU because all the checksums. Dedupe takes a lot of RAM. I can't over state having gobs of RAM and adding L2ARC cache drives to the pool to improve dedupe performance. The more RAM & cache you have, the more file hashes the computer can keep in memory, the faster the file system finds the duplicate files to dedupe. (Also, if compression is set high, that would require more CPU, but that is the tradeoff for backups.)
May
18
comment ZFS + dedup: copy many small files fast
+1 for snapshots
May
4
awarded  Custodian
May
4
reviewed Looks OK Win8.1: how to Get fingerprint sensor EgisTec ES603 to work?
Apr
22
awarded  Caucus
Mar
8
revised Replace NTFS hardlinks with files
added hardlink tag
Feb
1
awarded  Necromancer
Dec
31
awarded  Yearling
Nov
7
comment FreeNAS planning - RAID controllers & expanding volumes
If you want flexibility, I wouldn't recommend ZFS. There are a lot of limitations to expanding and upgrading drives. What you may want to look into is LVM. I wanted a zfs drobo-like box, and never found the perfect solution. If you want zfs, you will have to make trade-offs.
Jul
5
awarded  Nice Answer
May
13
awarded  Enlightened
May
13
awarded  Nice Answer
Dec
31
awarded  Yearling
Oct
16
comment ZFS & Deduplicating FLAC Data
Agreed, what jasongullickson needs is something like Audio Dedupe (mindgems.com/products/Duplicate-MP3-Finder/…) for Linux.
May
12
comment Duplicate File Scanner
What programs have you tried without success?
Mar
28
answered Are file copy utilities really useful?
Mar
28
answered VMWare Workstation 8, Sharing a VM without copying them to “Shared VMs location”