Improving File Transfer Service ▬ FTS3

Abstract 

This project deals with two aspects of improving the file transfer service – FTS3. The first one is the selection of best source site for file transfers. Since files are replicated at different sites, the selection of the best source site based on the networks throughput and success rate can have a major impact on FTS3. The second one is maximizing the file throughput across WLCG network by increasing the TCP buffer sizes. TCP is the only transport layer protocol used widely for data transfers; it was originally designed with focus on reliability and long-term fairness. In high bandwidth networks, the system administrators have to manually optimize/tune the TCP configurations. Some of these configurations have a major impact on throughput. TCP buffer size is one such setting, which sets a limit on TCP congestion window size. With the release of Linux Kernel 2.6, a new feature “Linux TCP Auto-Tuning” was introduced, which selects the optimal TCP buffer sizes based on system resource usage. Another way to increase the TCP buffer size is to use setsocketopts system call. Since FTS3 implements gridFTP protocol, it gives us the flexibility to set TCP buffer sizes manually. This project evaluates the pros and cons of different techniques for setting TCP buffer sizes. 

Keywords: FTS3, TCP , Linux TCP Auto-Tuning