The fastest way to transfer files over a network (FTP, HTTP, RSync, etc.)

file-uploadftphttpnetworkingrsync

I'm trying to figure out the best way to transfer large amounts of data over a network between two systems. I am currently looking into either FTP, HTTP, or RSync, and I am wondering which one is the fastest. I've looked online for some answers and found the following sites:

The problem is that these are old, and talk more about the theoretical differences between how the protocols communicate. I am more interested with actual benchmarks, that can say that for a specific setup, when transferring files of varying sizes one protocol is x% faster then the others.

Has anyone test these and posted the results somewhere?

Best Solution

Alright, so I setup the following test:

  • Hardware: 2 desktops Intel Core Duo CPU @ 2.33GHz, with 4G of RAM.
  • OS: Ubuntu 11.10 on both machines
  • Network: 100Mb dedicated switch, both machines are connect to it.
  • Software:

I uploaded the following groups of files to each server:

  1. 1 100M file.
  2. 10 10M files.
  3. 100 1M files.
  4. 1,000 100K files.
  5. 10,000 10K files.

I got the following average results over multiple runs (numbers in seconds):

|-----------+---------+----------|
| File Size | FTP (s) | HTTP (s) |
|-----------+---------+----------|
|      100M |       8 |        9 |
|       10M |       8 |        9 |
|        1M |       8 |        9 |
|      100K |      14 |       12 |
|       10K |      46 |       41 |
|-----------+---------+----------| 

So, it seems that FTP is slightly faster in large files, and HTTP is a little faster in many small files. All in all, I think that they are comparable, and the server implementation is much more important then the protocol.