The Impact of Artificial Intelligence on Video

Big advances are taking place in intersection of video with AI (Artificial Intelligence). I ran across an interesting article in Streaming Media Magazine called The State of Video and AI 2018 that takes stock of some of these changes and I wanted to share it with you as we look toward what’s ahead for Duke.

 

We’ve been following trends in this area from a number of directions, including video captioning. As many of you are aware, the needs for captioning videos we produce at Duke are increasing, but the costs of captioning services, most of which rely on intensive manual labor, are high. However, new tools like IBM’s Watson, which includes more than 60 AI services, including machine captioning (with accuracy advertised as a whopping 96%), seem poised to shift the balance and make it possible for us to caption videos on a wider scale. We demoed Watson recently and will continue to monitor it as well as other tools in this space.

In this context I also wanted to point out that we recently began offering ASR (Automatic Speech Recognition) for Panopto, Duke’s lecture capture service. We are excited about the opportunities this new functionality will offer students and other viewers who are looking to drill down to points in videos where specific terms are found. This feature adds to Panopto’s already healthy set of features built around in-video search, including OCR (Optical Character Recognition) for slide content, and user-created time-stamped notes and bookmarks.

New Features at Rev.com

Rev.com, currently the most widely utilized caption service provider at Duke, just announced some new features we wanted to let you know about. All are included at no extra charge in their standard $1.00 per minute service. For more information about getting started with Rev or another caption provider, you can visit https://oit.duke.edu/what-we-do/services/captioning. You may also be interested in attending A Hands-on Guide to Captioning at Duke, a Learn IT@Lunch session scheduled for Wednesday, January 31, 2018 in which OIT’s Joel Crawford Smith and Todd Stabley will discuss video captioning at Duke and help you set up an account with Rev and get started captioning your videos.

Browser-based Captions Editor: It makes minor fixes, converting formats and frame rates. You can access it on any Order Detail page by clicking “edit”. Or give it a test run here: https://www.rev.com/captions-editor/sandbox

Rev's new browser-based caption editor

Rev’s new browser-based caption editor

Browser-based Transcript Editor: Allows changes like formatting, speaker labels, etc. If you order timestamps, Rev gives you a transcript with word by word timestamps that play along with your file. You can test it out here: https://www.rev.com/transcript-editor/sandbox

Turnaround: Rev reduced transcription turnaround by 25% and caption turnaround by 50% over the last 12 months.

Revver network: Rev crossed 14,000 monthly active Revvers (freelancers who transcribe and caption). 90% are based in US/Canada. This allows us to turn around large volumes with high quality. Rumor has it a Duke staffer who thought they were quite qualified to be a Rev captions editor and was rejected. Say it isn’t so!

Support coverage: Rev expanded their 24/7 support to the weekends as well.

Additional improvements: custom timestamp offset for transcripts, PDF and TXT transcript outputs, and improved Rev API support.

If any of our peer Universities are interested in speaking with Rev, feel free to reach out to us and we’ll connect you.

Duke’s Disability Management System Visits the TEC

Last week, three members of Duke’s Disability Management System (DMS) visited the Technology Engagement Center to provide an AV focused Americans with Disabilities Act (ADA) overview and to highlight their services to the Duke Digital Media Community (DDMC). DMS Executive Director, Leigh Fickling, kicked off the session with an explanation of what the ADA is, who is protected under the ADA, and provided an AV and IT friendly overview of past legislation leading up to the current environment of accessibility at Duke. The conversation touched on a number of areas including accessible websites, software-based hearing assist, accessible parking, live event accommodations, video captioning and descriptive audio tracks (see below), hardware mounting guidelines, and fundamental classroom acoustics.

Four key takeaways from the session:

  • If you have ADA specific questions (hardware, software, legal, etc.), DMS is Duke’s go-to resource (Pssssst… they are also great to work with!)
  • DMS has a treasure trove of free and fee-based solutions for making your learning environment and live events more accessible.
  • Disabilities come in many shapes and sizes. We must think beyond basic wheelchair, visual, and auditory accessibility when accommodating our students, faculty, and staff.
  • Accessibility is a living goal that changes over time, not a checkbox or task to be completed (and forgotten).

Descriptive audio, as mentioned in the session and above, has started to appear in many films as an alternative soundtrack for individuals with a visual impairment. Take a listen to see the current state of accessibility in the film industry and how this will make its way to a college campus near you.

For those interested in the PowerPoint presentation: DDMC_ADA_SESSION

We Captioned the Announcement of Duke’s New President–Live!

Duke’s announcement of a new president last week gave us a fantastic opportunity to take a big stride forward in our use of and advocacy for video captions at Duke.
price-youtube
We’ve been making strides in encouraging the use of captions for on-demand video, and while much of our work has focused on the partnership between OIT Academic Services and the Student Disability Access Office (SDAO) around accommodations for students with disabilities, it’s important to note that captions add a lot of value not just for users with disabilities but for many others as well.

As most of you know, we have numerous digital signs deployed across campus through a system called Four Winds Interactive, and we pushed the live stream of the new president’s announcement to those signs so that people across campus could watch the event while it was happening. The captions provided value for everyone in this case, because many of those signs don’t have audio, or don’t have it turned on because it would be intrusive.

To create the captions, Media Technologies, part of Academic Services, worked with a company called ACS (Alternative Communications Services), which specializes in the delivery of live captions. This company has done quite a bit of work for SDAO in providing accommodations for students needing captions for classes at Duke, and SDAO has spoken highly of their experiences in working with ACS. Our experience with the president’s announcement lived up to the high praise we heard. ACS did quality work for a reasonable rate, and was very responsive in their communications around this project. They even did testing with our production team the day beforehand for free. They were able to talk through our setup and help us design a workflow that resulted in the best possible quality. A key in our case was being able to provide ACS a direct audio feed so that there wouldn’t be a delay of more than a few seconds between the captions and the video. To accomplish this, we needed to give ACS a direct feed of the audio, and our production team was able to do that via WebEx. ACS also was able to provide helpful instructions to our production team for enabling captions on the live broadcast, which we did through YouTube. There were some rather intricate menus that needed to be navigated in the YouTube interface to allow ACS to tie back in with our live broadcast.

Adding Subtitles with X-Title Importer and FCPX

For anyone looking to add subtitles or captions onto a video in Final Cut, X-Title Importer by Spherico makes pretty short work of the job.

Screen Shot 2016-08-03 at 4.40.03 PM

XTI Interface

To start, you’ll need a supported source format for your captions. We used a .SRT file, but XTI supports over a dozen formats including Web VTT, Spruce STL and YouTube SBV.

You’ll also want a template for your subtitles. X-TI includes a basic template, TextUp, which you can modify within the application. For captions with a dynamic background, like a box whose width changes depending on the width of the text, you’ll need to go a little further. Installing one of Spherico’s other applications, TitleExchange, will add a TextUp Box template that can achieve the effect. Additionally, one could create their own by modifying FCPX’s own Basic Title in Motion with a couple clone layers.

Once you have your template settled, its a simple matter of selecting your source file within the application and converting it to a .fcpxml file. Open that .fcpxml file with Final Cut and it will create a new project within the library of your choice. The project will include all of your captions added as titles, below which you can simply include your original video.

Help us Test a Captioning Vendor

Help us test Rev

For those of you who missed the talk Molly Bragg  and I gave at Tech Expo last month, we are seeing competition among captioning vendors continue to drive pricing down. The most recent vendor we’ve tested is Rev (www.rev.com), which offers captions for $1.00 per minute. I tested Rev using a 5 minute video that we also tried with two of our other vendors, and Rev performed very well. They have offered us free captioning for a 90 minute video, so I am searching for someone who has an actual use case that would allow us to take advantage of this so that we can better vet this vendor. Please let me know if you could benefit from this or know someone who could.

Thanks!

-Todd