Published on 14-Dec-2012
"The RamSan has just completely eliminated [bottlenecks]. Everything’s flowing smoothly." - Mike Saindon, Production Engineer, UFC
Media & Entertainment
Ultimate Fighting Championship® is the fastest growing sports organization in the world and is also one of the industry’s largest producers of video content. Along with its live pay-per-view shows that are available to 345 million homes worldwide, UFC® will produce 350 hours of original content this year for its broadcast and cable outlets on FOX broadcast networks. In addition, the UFC is in the process of digitizing its entire 10-year library of analog videotape content.
UFC had the following challenges: fast delivery of massive amounts of video in hundreds of derivatives; operations affected by heavy transcoding load; constraint on amount of content UFC can transcode and deliver to its partners; overall production output affected by disk slowdown.
UFC implemented the RamSan-710 which delivered a non-disruptive solution and was deployed into the production environment within a day.
Time for transcoding jobs dropped by up to 70 percent. Overall operations were improved 40% by implementing the RamSan due the the reduced load of the storage.
Ultimate Fighting Championship® is the fastest growing sports organization in the world and is also one of the industry’s largest producers of video content.
Along with its live pay-per-view shows that are available to 345 million homes worldwide, UFC® will produce 350 hours of original content this year for its broadcast and cable outlets on FOX broadcast networks. In addition, the UFC is in the process of digitizing its entire 10-year library of analog videotape content.
I/O intensive video transcoding
UFC operates a robust video production facility in Las Vegas, encompassing 1 Petabyte of hard disk capacity and a 2.5 Petabyte LTO-5 robotic tape library. The system is built on Quantum’s StorNext® platform, a scale-out file system designed for high-performance file sharing that is widely-used in the video production industry, with Telestream transcoding engines to deliver content across a spectrum of diverse platforms. Due to the massive scale of video production at UFC, Christy King, Vice President of Digital, Technology and R&D, said the company built an asset management system with state-of-the-art capabilities. The entire video workflow is managed by Level Beyond’s Reach Engine software, an advanced media management and integration platform that enables UFC to ingest, edit, manage, archive and deliver premium digital media anywhere with unprecedented efficiency.
UFC provides digital content for 130 platforms worldwide, including Hulu, iTunes, Xbox, PlayStation and Samsung TVs, in addition to its own web and mobile consumer-facing platforms. Service level agreements with some of those partners require UFC to edit content and deliver it, typically within hours of the live event.
“Quick turnaround is absolutely critical,” said Danny Gold, Vice President, Strategy and Solutions at Levels Beyond. “UFC has extreme requirements on how fast they have to deliver massive amounts of video in hundreds of derivatives as soon as it comes in the door.”
To make the video content compatible with the variety of platforms, every piece of content must be transcoded. Video transcoding allows content created in one format to be playable on a different format. The process is crucial to UFC, which provides content in over 100 unique final formats and resolutions, for everything from big-screen TVs to tablets and smart phones. Video transcoding is a write-intensive process that can choke even the most robust hard disk arrays.
Accelerate video production without rearchitecting the environment
UFC discovered that while the Reach Engine asset management system was capable of handling the huge transcoding load, its backend SAN storage was not. The impact of the heavy transcoding load was taking a toll on UFC’s operations. The massive volume of transcoding high-resolution video imposed an IOPS load that the UFC disk storage arrays could not support. Even high-end disk arrays can fall short of the demands of frame-by-frame transcoding workloads where the disk heads simply are not physically fast enough to keep up. This is an inherent bottleneck with all rotating hard disk drives that cannot be overcome even by adding more drives to the array. The fundamental performance limitations of HDD technology degraded the video production operations on two fronts: the company was constrained in the amount of content it could transcode and deliver to its partners, and the disk slowdown caused by transcoding demands in turn slowed down all other editing projects, limiting the overall production output.
“We were quickly hitting the roadblock with the spinning disk that the volume of transcodes was bringing down the storage,” said Gold. “Their editors weren’t happy because they were dropping frames as the disk would take such a huge hit when transcoding multiple jobs. UFC was having to limp long and really throttle the number of transcodes and throttle the number of deliveries which isn’t good for business.”
“We’re a sports promotion company, and what we do is sell the next live event,” said King.
“So if I can’t get this video out of here and deliver it to my television partners in a super timely manner then that just slows down the entire marketing process.”
The challenge in finding a solution to the disk performance problem was complicated by the need to come up with an alternative that would fit into the existing environment without disrupting the whole workflow rather than a rip-and-replace solution that required a re-architecture of the entire system.
High-performance RamSan-710 Flash Storage
A colleague referred Gold to Texas Memory Systems, an IBM® Company and UFC quickly installed a RamSan-710 unit to evaluate. The RamSan-710 is a 1U system with 5 TB of SLC Flash storage that delivers 400,000 IOPS read and write performance. To get equivalent performance would require more than 2,200 15-K disk drives with all of the associated power, cooling and floorspace costs.
The RamSan-710 met all of UFC’s initial requirements: a non-disruptive solution that offloaded the heavy lifting from the disk arrays and dropped right into the existing 8 Gb Fibre Channel SAN infrastructure with no change required to the workflow.
“Within a day we were able to get it running in a production environment without a huge change in the infrastructure,” said Gold. “There was nothing we had to change in code or workflow. We just pointed all the transcoders to the new StorNext volume on the RamSan instead of volumes on the spinning disk arrays. The fact that it played so nicely with StorNext was a huge benefit.”
Reduction in processing time by 70 percent
After the simple setup, UFC saw the time required for transcoding jobs immediately drop by up to 70 percent, with an across-the-board reduction of 40 percent for every one of the thousands of files processed for delivery to partners.
Just as important, said Gold, was that the improved performance with the RamSan also took the load off the disk storage that was impacting other operations.
“It’s not just that we needed to get these things through faster—which we did—but we needed to get them through without affecting the rest of the network. So even if the speed was the same, that would have been a huge benefit. With the RamSan we gained both benefits.”
“The biggest difference I see is no bottlenecks,” said Mike Saindon, UFC’s production engineer. “That’s the problem we were running into before. We were having a handful of editors kickoff encoding exports and it would slow the entire system down to the point of not moving at all. The RamSan has just completely eliminated that. Everything’s flowing smoothly.”
“Before we put the RamSan in we couldn’t use our asset management system to its maximum capacity and intent,” said King. “The RamSan makes it so we can push a couple thousand requests at a time for specific encoding jobs and handles it just fine. Everything gets in the queue and life is good.”
King said that the current system should sustain UFC’s growth for the next 12 - 18 months. At that point she expects to add six to eight more Telestream boxes to handle UFC’s increased production workload. “We’ll be processing orders of magnitude more content and it appears that RamSans will continue to be the best processing solution for speed and flexibility.”
Levels Beyond is now exploring other opportunities for the RamSan-enhanced Reach Engine production platform, presenting prospective customers with a drop-in solution that leverages the capabilities of solid state-based Tier0 storage for high-volume, IOPS-intensive video workflows. Gold says that state-of-the art Reach Engine content production and delivery platform combined with RamSan solid state storage to mitigate the storage-busting effects of high volume transcoding in a solution that seamlessly integrates into the industry-standard StorNext environment presents a compelling value proposition for enterprise video production users.
Products and services used
IBM products and services that were used in this case study.
Storage: IBM FlashSystem 710
© Copyright IBM Corporation 2012 IBM Corporation Systems and Technology Group Route 100 Somers, NY 10589 Produced in the United States of America December 2012 IBM, the IBM logo, ibm.com, Texas Memory Systems and RamSan are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries. This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates. The performance data discussed herein is presented as derived under specific operating conditions. Actual results may vary. THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements under which they are provided. Actual available storage capacity may be reported for both uncompressed and compressed data and will vary and may be less than stated.