As  of the ramp up of the SDN project at Duke we have learned a number of important lessons.

1. Performance testing on a production network can have unexpected impact.
2. Your network may not be as fast as you think it is.
3. Your network’s performance may change over time – oversubscription for ports may creep in over time.
4. Connecting 10G ports to a network core with 20 Gbps of capacity should be reviewed with scrutiny – in the past connecting a 1 Gbps port to a 20 Gbps core was less risky.
5. Span ports on the network may not be your friend
6. perfSONAR is fairly un-intrusive and provides lots of useful information about capacity and latency, but may not put enough of a load on the network to show the weak points.
7. Spanning Tree is your friend – don’t turn it off without a lot of planning – 10G loops are a bad thing.