OIT has deployed a number of perfSONAR (PS) nodes around campus and have found that using PSfor bandwidth testing has identified a number of opportunities for bandwidth improvements as well as clearly showing the ongoing improvements to the Duke core network.

The initial data shows the performance of the network when connected via a 1 Gbps network connection to the original OIT core.  In mid-May, the network was migrated to the new core and connected at a speed of 10 Gbps.  However, at that time the Fitz East data center was not yet connected to the new core.  The reduction of network capacity from 1Gbps to 500 Mbps was tracked down and found to be related to the througput of a single stream through the Cisco Source Fire IPS devices used in the new core.   After whitelisting the traffic through the new IPS, the performance between the two servers was improved to about 1.5 Gbps although there was some asymmetry in the traffic rates between the two servers.  On June 9th, the Fitz East data center was moved to the new core network and bandwidth improved to a reliable 2.5 Gbps in each direction.  The only things in the path between the two servers were the SourceFire IPS (which was not inspecting the traffic) and a virtual firewall context.  It was surmised that the 2.5 Gbps limitation on bandwidth was due to the firewall.  The bio-perfsonar-02.biology.duke.edu server was taken off-line and migrated to a new IP address that was in the biology data center but not on the Biology VRF.  When the server was restored to service at the new IP address on July 9th, performance immediately improved to an expected 5-7 Gbps.

Similar data is shown below for the network connection between two servers on the same VRF but in different buildings.

Physics has servers in both the Physics Building as well as the BioSci building data center.  The graph below shows traffic flowing between phys-perfsonar-02.phy.duke.edu and bio-perfsonar-04.phys.duke.edu

Both the Physics building and BioSci building were upgraded to 10Gbps and connected to the new core on May 15th (shown in more detail below).  The network between the two buildings immediately improved to delivery of 6-8 Gbps of bandwidth from the earlier limit of 1 Gbps.

It is important to remember that PS shows the available bandwidth, these graphs do not directly show how much is being used.