Early on in the November 26 Stanford-Notre Dame football clash, Stanford quarterback Andrew Luck lobbed a ball toward the corner of the end zone. His receiver, tight end Levine Toilolo, leapt up, reached over the head of the Fighting Irish defender to nab the ball, and landed in the end zone for the first touchdown of the game.
Or so I assumed, judging by the crowd’s reaction. From my vantage point in the nosebleed seats at Stanford Stadium, I had no idea if Toilolo had come down with the ball, let alone if he landed in bounds.
Not a problem: With the same cool head and dexterity that has NFL scouts raving about Luck, I pulled out my iPhone, fired up Stanford’s Gameday Live website in Safari, and tapped on the link to watch the replay of Toilolo’s touchdown catch. In fact, I could choose between three different camera angles of the same play.
Near-instantaneous replays delivered directly to your mobile phone while the crowd around you goes wild sound like something out of a sports fan’s daydream. But it’s very much been a reality at Stanford University since this past fall. The school worked with Elemental Technologies, a video processing and delivery company, to install a system that gives fans in the stands access to replays and highlights of the game unfolding before them.
The system Elemental put into place at Stanford covers both the school’s football stadium and its gymnasium, Maples Pavilion. (In fact, the control booth for Stanford Stadium’s video replay system is tucked away in Maples, about a quarter-mile walk from the football field.) Using this instant replay system, fans at football, basketball, and volleyball games can get replay on any mobile device, including the iPhone and iPad, over a Wi-Fi connection.
That kind of on-demand replay has long been touted as a way of enhancing the in-game experience for fans at live sporting events. Even the best seat in the house doesn’t offer the best view on every play. Maybe the action is taking place on the opposite side of the stadium, or your view was blocked by the guy seated in front of you waving a giant foam finger. Or maybe you were off getting nachos during a critical score. Video replay on your phone means you’ll never miss a key moment, and Elemental’s approach means you’ll have multiple views of those key moments.
“I think, going forward, you’ll go into a stadium, download an app for that specific stadium, and be able to watch replays,” Elemental Technologies CEO Sam Blackman said. “It customizes the experience.”
Elemental knows a thing or two about delivering video content to different devices. Just last month, the company announced that it now serves more than 100 enterprise customers after launching its professional product line in 2010. Customers include Comcast, HBO, and ESPN, which uses Elemental’s products to create on-demand video for the sports network’s websites and mobile apps.
Stanford’s video project offered a different set of challenges. For one, there’s the not inconsequential matter of making video available in a stadium filled with 50,000 people, putting a strain on the network’s bandwidth. There’s also the fact that Stanford had its own system already in place—video replay screens on the scoreboards in Stanford Stadium show some highlights—so whatever Elemental could offer had to fit within the existing set-up.
On that first issue, Elemental got some help from Apple in the form of the HTTP Live Streaming technology the company developed. A key advantage of using HTTP Live Streaming, or HLS, to send out audio and video content to assorted devices is the technology’s ability to adjust playback on the fly to account for network traffic. In other words, if available bandwidth drops, HLS dynamically downgrades the video quality to keep the content coming. That’s made the technology a popular choice among companies that delvier sports action to mobile devices— Major League Baseball uses HLS for its offerings, for example—and Elemental is no exception. “Our goal is to eliminate buffering,” Blackman said.
Working Elemental’s video-replay offering into Stanford’s existing operation meant working with the university’s already existing EVS system, a video replay and production set-up that’s fairly common in athletic venues around the country. The EVS system constantly records action captured by multiple cameras; an operator logs the start and stop points and names the clip, pushing selected replays to the stadium’s video board.
“We basically leveraged that existing system,” said Adam Salkin, an engineering manager with systems integrator Diversified Systems, which worked with Elemental to set things up at Stanford.
Essentially, the same feeds going into the EVS system are simultaneously delivered to Elemental’s encoder. Each feed is converted into an HTTP file that can be pushed out to Stanford’s Gameday Live site.
The key to this process is speed. Salkin told me that a clip can be converted to the HTTP format on one of Element’s encoders in about 15 seconds—a fraction of the time it would take to convert something from the EVS system’s proprietary format to an MPEG-4 file that could be viewed on a smartphone.
“By the time the EVS operator says, ‘That’s the clip I want,’ the encoding’s already been done,” Salkin said.
It’s something to see in action. A producer back at the football stadium coordinates the action, communicating via headset with the half-a-dozen people in the control booth a quarter-mile away. The EVS operator who’s logging clips for video board replays (and, by extension, the ones that Elemental’s encoders will push to Stanford’s Gameday website) sits at a workstation with four monitors showing assorted camera angles from the stadium. Two monitors below the incoming feeds allow the operator to trim and log clips—each one is labeled with the name of the player involved, a one word description of the play (like “touchdown” or “sack”), the quarter it occurred, and the camera angle used. (This same information shows up on the Gameday Live mobile site when video highlights appear there.) All of this happens fairly quickly, with no extra manpower needed to manage the Elemental-produced clips other than the personnel already on hand in the control booth.
Clips prepared via Elemental’s system may be ready to view within 15 seconds or so, but at the game I attended to see the setup in action, they were appearing on Stanford’s mobile site a minute or two after the play in question. That’s by design, according to Stanford associate director of athletics Kevin Blue, who explained to me that the university is “still playing around with how to create the best experience for the fans.”
The system had been put in place in September, and by the last football home game of the season, Stanford officials were still managing some of the technical issues posed by trying to beam near-instantaneous replays to a stadium full of fans. For example, at a football game, the folks running the Gameday system had to deal with a lot more radio-frequency interference from everything including the communication systems coaches use to send plays from the press box to the sideline and the systems TV broadcasters employ to remotely control the skycam that floats on wires above the field.
Things are a little easier to manage at basketball and volleyball games, according to Blue, because of the smaller crowds (Maples Pavilion holds about an eighth of the crowd that crams into Stanford Stadium) and less cluttered airwaves. Blue estimates that between 10 and 30 percent of the people attending games at Maples Pavilion are accessing the Gameday Live site on their mobile devices, and while not all of those visitors are necessarily watching video highlights, it does illustrate the opportunity for augmenting the fan experience.
“There are technical issues to overcome precisely because it’s an unprecedented learning experience,” said Blue, adding that the university is happy with the way things are “out of the gate.”
Testing the Stanford-Elemental highlight system first-hand, it was easy for me to imagine how on-demand video highlights could evolve given a little time. By design, the team selecting highlights for Stanford’s mobile site were restricting the clips they picked to scoring plays and a few Stanford defensive highlights. But a wider selection of clips would certainly benefit smartphone-toting fans. For example, the opening play of the game featured a fumbled kickoff return that was subject to a video review by the game’s officials; I imagine that fans would love to fire up their smartphones to see a replay of the play in question as officials deliberate over whether to reverse the call. The game I attended featured Luck, Stanford’s quarterback, tying and then breaking the school record for career touchdown passes; the mobile video highlights showing the record-breaking pass could have included that important bit of context to further fans’ enjoyment of the game.
“The value is in making the [fans’] phones extensions of the video board,” Blue agreed.
Still, it’s obvious that Elemental and Stanford are on the right track with their mobile phone-based video highlights. And, given the wide adoption of smartphones in general and the iPhone in particular, it’s an experience likely to reach far beyond Stanford’s Palo Alto, Calif., campus as it comes to a stadium or arena near you. “There’s no question the future is what we’re doing here,” Blue said.
Elemental’s Blackman agrees: “In five years, every single tier-one stadium will have these kinds of capabilities.” And when that happens, your only excuse for missing a play will be that you’re a little too slow on the draw with your iPhone.
[Philip Michaels is the editor of Macworld.com.]