Challenges and Solutions in AV Network Management

 

Creating a well-designed, resilient, and secure AV over IP network that is integrated into the overall technology infrastructure of a university campus requires funding for equipment, training for staff and faculty, and the creation of a diverse and dedicated AV support teams to ensure proper maintenance and operation of the AV systems. All this adds up to a seemingly unsurmountable series of tasks.

In a session of the DDMC, Dr. Joe Way, the Director of Learning Environments at the University of Southern California, along with his senior team of Raj Singh and Chi Lo, discussed the challenges they faced and their approach to resolving them in their own AV network.

 

 

Raj Singh kicked off our session with pointing out initial challenges of managing multicast traffic and traversing between different switches and VLANs for the AV system in USC. They performed a stress test with about 40 rooms to create a playbook for replication and put network and security team guardrails to contain the multicast traffic. Challenges also include securing IP traffic, issues with bandwidth, multicast routing, and multi-MAC exemptions. USC’s approach to network AV is a top-down approach, designing rules and systems first and then propagating down to individual rooms. Data hygiene is important in network-based AV environments, unlike traditional AV where devices can be easily connected and swapped out.

Chi Lo further discussed stress testing a network to ensure it can handle high bandwidth traffic, such as 4K video. Recommend testing all scenarios and maximizing bandwidth to guarantee network readiness for future needs. The importance of direct connection of encoders and decoders to the distribution switcher is emphasized. Firmware updates that allow traversing across multiple fabrics and implementing software based VLAN separation.

The conversation continued with a question about USCs networking approach for distributing AV over IP system. Layer 2 or 3 and why? With a flat layer 2 network and the need for a layer 3 model with edge routing to avoid the whole system going down if a point in the network fails. Raj spoke to the use of router stacks on each floor of the building and keeping the AV system separate to that floor. The approach includes a mix of layer 2 and layer 3 with direct connections to network switches for multicast devices, and a registration and monitoring system for security.

The importance of security measures in the network and how USC went through an approval process with its own network security team to ensure the firewall rules and ports were secured was also discussed in detail. They partnered with the CEO’s office and IT infrastructure early on to get their approval and blessing, which slowed down the process initially but ultimately fast-tracked everything else. They were able to develop policies and tools to monitor the system and detect anomalies. They also used software-based tools to see what type of process is causing issues in the network.

Tracking metrics becomes the priority once an AV over IP network is in place, one on the engineering side and the other on the IT side. They are looking at things like network bandwidth, CPU cycles, packet loss, latency, and connectivity. Exploration of self-healing solutions for issues like cameras going offline, which can be fixed by power cycling. The team is also focused on ensuring that all their devices are running the latest software to enable them to connect to different platforms like Microsoft Teams, Zoom,and Webex. Compliance and security risks are also taken seriously, and upgrades are made promptly when necessary.

In conclusion, the DDMC session featuring Dr. Joe Way, Raj Singh, and Chi Lo provided valuable insights into the challenges faced and solutions implemented in the AV network at the University of Southern California. From network security and software updates to equipment maintenance and user support, the USC team demonstrated a comprehensive approach to managing a complex AV infrastructure.

While we have summarized some key points from the session, there are many more details to be learned from watching the recorded video. Overall, this session serves as a valuable resource for anyone involved in AV network management, particularly in an educational or large organizational setting.

Wolfvision CYNAP Pro

App-free dongle-free screen sharing!

Connect and share your screen using the wireless technology that’s built into your own mobile device. Our wireless BYOD solutions suit all iOS, iPadOS, Android, Chrome OS, Windows and Mac devices – with full support for AirPlay, Chromecast, and Miracast screen mirroring.

Record your content

Cynap Pro lets you record all your multi-window, multimedia content. Everything is captured in high definition and saved internally – perfect for use as part of your online educational program. The included Capture feature pack enables operation as capture agent for Panopto, and other compatible video management platforms.

Multi-platform web conferencing

Cynap Pro‘s multi-platform web conferencing solution runs directly on the device itself, and is designed to solve many of the issues most commonly experienced with BYOM web conferencing systems.

Complex multi-step setup, and bandwidth issues are eliminated, and Zoom, MS Teams or WebRTC-based wireless conferencing sessions are easily started and controlled, using a simple workflow, from a touchscreen, or any laptop, smartphone, or tablet.

Stream & record to mobile

Our unique vSolution App for iOS, iPadOS, Android, and Windows lets your audiences receive and record a live stream of presentation or lecture content from Cynap Pro onto their own smartphones and tablets.

Freedom to present

Cynap Pro plays, displays, records, and streams all commonly used media at the same time, giving you unlimited choice of materials during presentations, lectures, and active learning classes.

Access your data easily via cloud, network drive or from mobile devices – even your laptop is no longer essential – you can simply bring your content on a USB stick, or download it directly from the cloud!

Annotate over any open window

Add to your content material using our built-in annotation features – or note down your ideas using the digital whiteboard, and save the output of both for future use.

If you are looking for an all-in-one with only needing to add audio processing? The CYNAP line of products is a good way to go.

Crestron NVX Training

Duke is considering deploying Crestron’s NVX network-based AV solution in a unique active learning/gaming environment, so we sent a few members of Duke’s AV community to attend Crestron’s NVX Design and Application (DM-NVX) and DigitalMedia Networking Certification (DM-NVX-N) classes. This was a unique class as it was highly compressed to accommodate our group’s schedule and needs.

Why NVX?
Crestron’s NVX platform is an AV over IP solution that replaces the need for more traditional 8×8, 16×16, 32×32, 64×64 and 128×128 DM switchers. NVX is more flexible, scalable, and brings AV into parity with modern IT practices. In essence, it’s the future of hardware-based AV.

The class started as many Crestron classes do with general introductions, some background on the devices, where to start when researching help (psss, it’s Crestron’s website)… and from that point on, we were thrown head first into the deep end of networking. Yes, very little of the class had to do with traditional audio or video. We covered the OSI model, IP addressing, subnet masks, port numbers, IP transport protocols, hubs, switches, routers, and DNS. For those that have a networking background, this was a nice refresher course. But, for more traditional AV folks, networking can sound like a foreign language. I won’t bore you with the details, but the first slide deck was over 180 slides, and the information was dense. After being bombarded with all of the information for roughly six hours, we took our first test. Most everyone passed on the first try… even if it was by a single question.

After the test and a bit of time to recover, we started a hands-on exploration of the hardware. We started off by connecting one NVX directly to another NVX, setting one as a transmitter and the other as a receiver. After a few minutes, we had a very basic AV system! The next phase was to connect the two NVX units to a local switch. That took a bit of switch configuring/setup, but again, it was easy. As we started the second day, we connected our local switches to a core switch, so we could share any of our NVX transmitters to any of the receivers. While more complicated, it wasn’t that difficult to configure. During the final hours of the last day, we chatted about programming for the NVX and how and why you may want to consider a DigitalMedia XiO Director, a Virtual Switching Appliance Crestron offers to simplify programming for more complex NVX setups. We had another test, and the class was over.

A few takeaways:

  • NVX, or AV over IP, is here to stay and AV groups should get comfortable with the future
  • While AV folks don’t need to throw away their old skills, networking is a core part of the future of AV
  • Start befriending your networking folks… today
  • IP over AV has a range of network security concerns, so you should also befriend your networking security folks
  • The future is exciting and complicated. Lean into the new way of doing things (or at least understand them)

 

 

NewTek Connect Spark Review

The good folks at NewTek were nice enough to send Duke’s Office of Information Technology a demo unit of their new Connect Spark. The Connect Spark is a difficult device to explain to someone that has never produced or recorded a multi-camera live event. In the past, you’d need to run at least one cable from each camera to feed video and audio to the switcher before capturing and streaming that content out. Those cables had length limitations (~15 meters for HDMI and ~100 meters for SDI), not to mention being rather inconvenient. With the Connect Spark, instead of running AV cables throughout the event space, this unit leverages the local wired or wireless network to stream content either directly to a computer or an NDI capable video switcher (more on that in a bit).

Out of the box, the device is rather simple to setup and configure from the perspective of an AV professional with reasonable networking chops. I downloaded the accompanying app and was communicating with the device within minutes, able to make adjustments and confirm settings as needed. I was then able to stream that content to a local computer for importing into Telestream Wirecast (and a number of other streaming applications). Beyond that, the device could also be streamed to a Network Device Interface (NDI) capable video switcher.

In academic environments, this device could easily be deployed in large event spaces to simplify the cabling necessary to support a large multi-camera event. Also, due to the flexible/modular nature of the hardware, this same equipment could quickly be redeployed in a different location with different cameras and minimal technician involvement. Beyond that, it frees up the production team to work anywhere with a connection to the network backbone. So, in theory, you could have videographers in one building filming an event, and the director and production team in another building working their magic.

Network Device Interface (NDI)
I mentioned NDI above. NDI is a royalty-free standard developed by NewTek to enable video-compatible products to communicate, deliver, and receive broadcast quality video in a high quality, low latency manner that is frame-accurate and suitable for switching in a live production environment, or so says Wikipedia. So, this device works wonderfully with NewTek’s very popular TriCaster… but it also works with other NDI switchers such as Panasonic’s new switchers or web based virtual switches.

The Gotchas:
It wouldn’t be a DDMC article if it was all positive. Most, if not all, of the “gotchas” with the NewTek are outside of NewTek’s control. First, if you have a complicated network topology, you may experience issues. For example, in some situations, I wasn’t able to communicate with the device as it was on a different subnet or vlan. Again, not a problem if you understand your network… but for a technician that has no idea what a subnet or vlan is… it could be a show stopper. I was quickly able to quickly work around this issue, but you may need to work with your networking folks to get this all to work seamlessly. Second, if you don’t have a robust network, you may experience dropout issues, specifically when using somewhat inexpensive switches. While the device worked perfectly on our enterprise network, I experienced minor issues with my (admittedly old) home network. Infrequently, I’d see a dropped frame or hesitation. Again, I don’t blame this on the Connect Spark, but be aware that you may want to upgrade to a more modern router/switch if you are on older equipment.

Overall, I really enjoyed the device, and it underlines the coming “AV on IP” reality for AV folks.

 

 

New Progressive Download Capability for Warpwire

One of the cool features we rolled out out in our last major Warpwire upgrade is an option for progressive download. This work has grown out of our involvement with Duke global programs, such as our campus at Duke Kunshan University. Specifically, we’ve noted that in situations that combine low bandwidth with high network latency, we can reduce video buffering and significantly improve the end user experience by delivering the files via progressive download rather than the standard HTTP Live delivery method Warpwire currently uses by default.

warpwireSecurity

 

We’ve updated our Warpwire FAQ (https://duke.service-now.com/kb_view.do?sysparm_article=KB0017639) to reflect this new capability as well as to include minimum technical requirements for using Warpwire. I’ve clipped the relevant info below:

What are the minimum technical requirements for using Warpwire?

Network Speed:
  • Viewing videos at 360p requires 1.0 Megabit per second
  • Viewing videos at 480p (SD quality) requires 3.0 Megabits per second
  • Viewing videos at 720p (HD quality) requires 5.0 Megabits per seconds
Latency (sometimes called “ping”):
To stream videos effectively via Warpwire’s default method (currently Apple’s HLS protocol), your network latency (“ping”) should be no greater than 150ms.
Test your network speed and latency (“ping”) here: http://www.speedtest.net/)

My Warpwire videos keep buffering. What can I do to improve performance?

If your videos keep buffering (i.e., showing three moving dots on top of the video), test your network connection for bandwidth and latency via http://www.speedtest.net/). If your bandwidth and/or latency are close to our minimum recommended thresholds, switching the media libraries to Progressive Download may help. To do that, contact the OIT Service Desk (oit.duke.edu/help) if you own the Media Library or Libraries containing the problem files and provide the name and URL of the Libraries in question. If you are not the owner, contact that person (the person who shared the files with you) and request that they contact the OIT Service Desk to switch to Progressive Download or make other arrangements to share the file/s with you.

Please note that if you do not have a bandwidth of at least 1 Megabit per second, your best option may be to request that the owner of your Media Library or Libraries move the files to Duke’s box.com site where you can download the videos and view them on your local hard drive.