On-set Virtual Production

Large, semicircle LED video walls are becoming the new choice for virtual backgrounds in the film & TV industry.

LED walls are effectively replacing Green Screen shooting, as they give actors something to react to in real-time, and create more realistic lighting on any reflective elements within the physical action. Very akin to historic rear-screen projection.

However, one of the biggest issues facing the use of LED video walls is the way the camera sees the colour emitted by the LED walls, compared to the live-action foreground elements.


On-Set Virtual Production

The main issues with Virtual Sets is enabling the camera to see the virtual images, displayed on the LED wall as being correctly matched to the to the live-action foreground elements.

Due to the different spectral response generated by the LED wall, compared to the lighting illuminating the foreground objects, the LED requires volumetric colour adjustment to generate am accurate perceived match in-camera.

Virtual Set Calibration Requirements

Many companies that both supply Virtual Sets, as well as those that utilise them within their productions, have adopted Colour Management tools from Light Illusion to ensure an accurate match between the live action foreground and virtual background, in-camera.

A combination of ColourSpace, and suitable probes(s) is used for initial LED video wall calibration, with MatchLight then used to auto-generate a driving LUT for the LED walls, to ensure the captured foreground and background images match.

Calibration

The primary requirement for Virtual Set calibration to ensure the captured foreground and background images match, within the camera image.

This requires an input LUT to be used on the video input to the LED Wall, to pre-distort the colours of the displayed images such that in-camera they match the foreground objects. In theory, this LUT can be generated and utilised without first performing calibration on the LED wall.

However, a better approach is to accurately pre-calibrate the LED wall, either to a known colour space standard, such as P3, or just to itself - the widest gamut and contrast range the LED wall can generate. This pre-calibration removes volumetric colour errors, making the generation of the foreground/background input matching LUT that much easier.

Calibration to a known colour space standard, or to the LED wall's native colour space, is performed as normal, using ColourSpace and suitable probe(s). Due to the spectral bandwidth of the LED wall primaries, a good Spectro is required to pre-match a Tristimulus probe, or the Perceptual Match approach can be used.

Generation of the foreground/background match LUT can be performed a number of different ways, but all require the camera in use to capture a range of colour patches, and the resulting captured colours matched to the correct colours, so enabling the generation of an LED wall input LUT to pre-correct the images to negate the spectral differences as seen by the camera, compared to the foreground objects.

MatchLight enables such LUT generation in a number of different ways, often in compilation with the Camera Options with ColourSpace.