Effectively copy files between servers using scp and compression

From ezUnix
Jump to: navigation, search
                                    pdf_icon.png Download this article as a single PDF document 


Introduction

When copying a large number of small files, it can be faster to use tar over an ssh pipe rather than something like rsync or scp. This is because tar sends all the data as a single stream, and doesn't have to make a remote test for each file.


Execution

This is how you would copy files over SSH/SCP from a local site to a remote site using bzip as the compression mechanism:

# tar -jcf - /some/local/directory | ssh root@remote_server.tld  "tar -C /some/remote/directory -jxf -"



That's all folks.
Marcin


<comments />
Talk:Effectively copy files between servers using scp and compression