Measuring Internal Bandwidth with iperf

The internal maximum bandwidth of a specific machine provides a good notion about the internal basic limits. Exceeding this bandwidth while communicating over real NICs is impossible.

The machine in this example has the following properties:

  • Intel Core i7 processor
  • 32 GB RAM
  • 512 GB M.2 SSD
  • Debian 10 (server only)

The NICs do not matter in any way in that case, since we are here only interested in the maximum achievable internal bandwidth (the iperf client sending to the iperf server over the loopback interface).

Starting iperf in server mode

t@u31:~$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------

Starting iperf client on the same machine

t@u31:~$ iperf -c localhost
------------------------------------------------------------
Client connecting to localhost, TCP port 5001
TCP window size: 2.50 MByte (default)
------------------------------------------------------------
[  3] local 127.0.0.1 port 44018 connected with 127.0.0.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   107 GBytes  91.8 Gbits/sec
t@u31:~

Running the same test over a longer time should give approximately the same results, here’s a test run for 5 minutes (300 seconds):

t@u31:~$ iperf -c localhost -t 300
------------------------------------------------------------
Client connecting to localhost, TCP port 5001
TCP window size: 2.50 MByte (default)
------------------------------------------------------------
[  3] local 127.0.0.1 port 44020 connected with 127.0.0.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-300.0 sec  3.10 TBytes  90.8 Gbits/sec
t@u31:~$

Conclusions

  • This internal maximum bandwidth tells you immediately if the machine is basically capable to process your bandwidth requirements internally (or not).
  • Knowing that limit you can then add and test the NICs for your application focussing on the IO, PCIe and networking capabilities.

We recommend to measure and compare this internal maximum bandwidth as the first step on selected machines during your initial planning phase.