Splinter Cell: Blacklist Virtual Camera Brings AR Magic To Gaming Cutscenes

Splinter Cell: Blacklist is giving developer Ubisoft the chance to test out a new way of filming and editing cutscenes, blending live actors, advanced motion capture, and a new virtual camera system for the human touch. The virtual camera, which Ubisoft Toronto demonstrated to Outside Xbox, mixes real-time video editing with augmented reality, allowing a human camera operator to navigate a virtual rig freely through previously filmed scenes, changing the perspective as they go.

At its core, the filming setup uses the same motion capture technology as has been used on many animated movies. Actors wear special suits which are tracked by multiple cameras at different angles, giving the controlling computer a full 360-degree view of the scene.

Meanwhile, head-mounted camera jigs do the same thing on a much smaller, more precise scale, capturing each actor's facial movements. That can then be overlaid with digital graphics later on. Audio is also recorded at the same time.

However, what's different is how the recorded scenes can then be manipulated. Ubisoft uses a proprietary camera – actually an LCD display mounted on a lightweight, portable rig, complete with motion sensors to track movement and direction – and some heavyweight processing to show a view of the scene from wherever the virtual camera is pointing.

That allows for different angles to be pieced together into the final cutscene, without requiring the actors to repeat things multiple times with filming from each angle. According to Ubisoft, the end result is more natural, with the feel of a physical operator rather than the artificial smoothness of a computer-generated perspective.

Ubisoft will use the same technology in the upcoming Assassin's Creed V and more.