Fastest Way to Copy Files

tar pipe

When you don't want to copy a whole file system, the experts agree that most efficient seems to be with a 'tar pipe'


From one disk to another on the same system:

(cd /src; tar cpf - .) | pv -trab -B 500M | (cd /dst; tar xpf -)

Across the network

You can add netcat to the mix (as long as you don't need encryption) and it's still fairly expedient.

On the receiver: 
(change to the directory you want to receive the files or directories in)

nc -l -p 8989 | tar -xpzf -

On the sender:
(change to the directory that has the file or directory - like 'pics' - in it)

tar -czf - pics | nc some.server 8989

With ssh

You can use ssh when netcat isn't appropriate or you want to automate with a SSH key and limited interaction with the other side. This examples 'pulls' from a remote server.

 (ssh user@some.server tar -czf - /srv/http/someSite) | (tar -xzf -)


If you already have a NFS server on one of the systems though, it's basically just as fast. At least in informal testing, it behaves more steadily as opposed to a tar pipe's higher peaks and lower troughs. A simple cp -a will suffice though for lots of little files a tar pipe still may be faster. 


rsync is generally best if you can or expect the transfer to be interrupted. In my testing, rsync achieved about 15% less throughput with about 10% more processor overhead. See the rsync page.