LED based Virtual Production /with Green Screen

Announcement FAQ Editions 3D Scene Editing 3D Scene Export Virtual Camera Camera Tracking Augmented Reality External Control Unreal


Hello. We are doing Virtual Production using LEDwall. We succeeded in synthesizing extended CG on and outside the LEDwall.

But we have nothing on the floor. Therefore, I shoot so that my feet cannot be seen. I would like to project CG on the floor, but it takes time and money to prepare LEDs for the floor. So we thought of laying a green screen on the floor.

Is it possible to use LEDwall and green screen at the same time for Virtual Production? I am attaching a sample image.

Answer please.


Profile Image


There is no ready-made solution for this in Aximmetry. However, if you want to experiment with things, then you can quickly put this together in the Flow Editor with some limitations.

What you need to do is to place the green in a virtual world, then use that as a mask of the keyed input video. While the green in the virtual world is looked at from your tracked camera's position. Then put this video as an overlay of the Led Wall compound.
You can do this by connecting the following modules to the Overlays pin of the Led Wall compound:

The Modules in the highlighted yellow rectangle are Transmit modules (Transmit Transformation, Transmit Scalar, and Transmit module), they get information from the Led Wall compound, and you have to set their From Tunnel pin as their names are in the picture above. More on transmit modules here in the Transmit modules paragraph: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/scripting-in-aximmetry/flow-editor/pin/#transmit-modules

The Keyer__All is compound located at: [Common_Studio]:Compounds\Keyers\Keyer__All.xcomp

When placing the green's position in the virtual world, you can connect the [Common]:Shaders\Special\Measure.xshad shader instead of the [Common]:Shaders\Standard\Basic_Solid.xshad shader to better see where is your green. When placing it, you should look at the Preview Green Placement exposed output:

This green should be almost identical in dimensions and position to your real-world green, but a bit smaller, just like when you set up greens in the Tracked camera compounds.

Also, there should be no gap between your green and led wall, and you should place the green in the virtual world a bit above the led wall:

This is because the Digital Extension's Edge Softness will cut down a little bit from your led wall's edges, even when you set it to zero:

You want no such cut where the Led Wall is bordered by the green, so that is why the green in the virtual world should overlap with the virtual led wall to some degree. There is a more accurate way to do this, but that would need some editing in the Led Wall compound.

If you are using more than 1 camera, then you can use the following logic to automatically switch the green when switching the cameras:

SELECTED CAMERA is a Transmit Integer module. Animation delay is coming from the Picture Delay, if you use a multi-machine setup where the control machine renders the Digital Extension, then you will need a bit different logic for multiple camera inputs...

Here are a few things to keep in mind:

  • Depending on your keyer settings, the keyed-out picture can have a different color than the input. This can result in talent having different colors over the green than over the led walls.
  • You need to key out shadows on the green, as shadows on the Led Wall won't appear.
  • The light coming from the led walls will affect the green's color and so it will affect the keying.
  • For good keying, you want good diffuse lighting on the green, this lighting might interfere with the led wall's picture.

Even led floors can have their own limitations compared to led walls. For example, led floors may have different colors or brightness than led walls due to additional glass layers, low camera angles, or just dirt or scratches from extended use. Or for example, the led floor position may change from people walking on it, if it is not fixed well to the floor.

Warmest regards,