The DTN (Data Transfer Node) Focus Group has performed several tests on GTS (GÉANT Testbed Service) in order to get useful results that the NRENs can compare and replicate for their own tests. The following matrix shows a summary of the set-up or each test, the parameters tuned, the software installed and the performance achieved, as well as the links to the information on how to install each software in GTS. Finally, some comments related to the setup and the test are included.

The DTN Focus Group has performed several tests on GTS in order to get useful results that the NRENs can compare and replicate for their own tests using up to four Bare Metal Servers (BMS) with two different setups. Both BMS setups were connected directly with 10Gbps links. It has also produced examples of tests and setups in Bare Metal Server (BMS) and virtualised environments using both VMs (provided as setup from the GTS testbed administration page) and Dockers.

More test details:

The tests started with DTNs using several software tools installed in virtual machines (VMs) with 1, 2 or 4 CPUs and followed with the same software tools in Bare Metal Servers (BMSs).

Hardware: two types of Bare Metal Servers (BMS) were used:

• Dell R430 (2x20C/40T Intel® Xeon® E5-2660v3 @ 2,6 GHz 25 MB cache, 128 Gb ECC DDR4 2133 MHz RAM, 6xSSD, 372 GB, 6,0 Gb/s HDD)  
• Dell R520 (1x8C/16T Intel® Xeon® E5-2450v2 @ 2,5 GHz 10 MB cache, 32Gb ECC DDR3 1600 MHz RAM, 2xSSD, 372 GB, 6,0 Gb/s HDD). 

O/S used For Client and server: Ubuntu-18.04-server

Performance metric: Performance measured as the effective transfer rate (Data File Size / Elapsed Real Time). For the measuring of the transfers, this investigation measured the time that a specific huge (i.e. 250GB, 420GB) transfer is taking and it calculated the bandwidth utilization, based on the size of the file and the time it took to move from the client to server.

Parameters tuned:

sudo sysctl -w net.ipv4.tcp_rmem="4096 16777216 2147483647"

sudo sysctl -w net.ipv4.tcp_wmem="4096 16777216 2147483647"

In the first investigation, DTNs on virtualised environments using VMs were tested. For the direct measurement of the links between the client site and the server, ipref was used. The results in virtualised environments showed that all programs evaluated achieve high bandwidth utilization except XRootd. The reason for not achieving high bandwidth utilization was the virtualisation itself. Xrootd is highly dependant on hardware resources and the virtualised environment with the virtual hypervisor on the physical system and the lack of direct access of Xrootd to resources (i.e. the physical link, not direct access to disks and buffer memory) had as result the low bandwidth utilization. On the other hand, the rest of the DTN services and client programs investigated (i.e. FDT, gridFTP) achieved higher bandwidth utilization. This first experiment was executed in virtual servers with different distances between the client and the server. In this case, both VMs were in Amsterdam for the shortest distance tests and one VM was in Amsterdam and the other one in London for a the longest distance test.Additionally, our investigation shown that in VM testing better results are achieved with VM with 2xCPU than 4xCPU. The aforesaid differences in results are due to different reasons. More specifically, it is due to different server architecture, Hyper-V VM Load Balancing, or that DTN tools do not know "how to" better use the CPU resources. It is important to note that this investigation did  not have any access to change VM parameters.

As mentioned before, the results may be affected by the fact that the links are not dedicated and other tests could have been running in GTS simultaneously. This could explain why the tests run between Amsterdam and London with 4 CPUs offered worse results than all the rest, as they were run in different days. When doing the tests, they were repeated at different time slots, to see if a pattern could be found regarding performance, but no significant changes were found.