Rene Kaiser, Wolfgang Weiss, Gert Kienast and Georg Thallinger, Joanneum Research
We present research results from the FascinatE project which investigates several aspects of a hardware/software system for live event broadcast. A number of innovative features aim to enhance the immersiveness of viewer experience and to save production cost. While the paper gives an overview on the whole system, it focuses on the Production Scripting Engines, software components which aim to automate camera selection, e.g. where to position virtual cameras (picture-in-picture), how to frame static and dynamic shots, and when to cut, in order to select the most relevant action in the scene while respecting cinematographic rules.
The FascinatE system is format agnostic which means it produces live content streams for different playout devices in parallel, enabling new forms of content consumption beyond personalization. In contrast to classic TV, every viewer can watch different parts of the scene, framed in the style of her/his choice. On a fundamental level, this is made possible by using a 180° high-definition panoramic camera that captures the whole scene at all times, the HHI OmniCam. Audio and video analysis modules extract low-level cues about the action in real time, and a tool for live manual annotation, the EditorUI, is integrated to assist the task.
The Production Scripting Engines (PSE) process those cues further to derive domain-specific higher-level concepts, a sub-process named Semantic Lifting. Decision making on automatic camera selection works in parallel in order to take individual parameters into account. It follows a rule-based approach that reasons with both pragmatic and cinematographic principles. Rules are triggered by the occurrence of higher-level cues and states that are specific to each viewer group. The current prototype deals with content from both the domains of soccer and dance performances.
1 The research leading to this paper has been partially supported by the European Commission under the contract FP7-248138, “FascinatE – Format-Agnostic SCript-based INterAcTive Experience” (http://www.fascinate-project.eu/).
Link zum Vortrag: https://vimeo.com/58181241