XR/AR/LED with real floor workflow

Hi, i'm interested in a particular workflow that were used in this case.

I understand the LED wall with digital extension workflow, but dont understand how to create a mask for the floor under the LED wall without soft edges (blurry edges) so that it fits in to the geometry of UE.

XR/AR/LED with real floor workflowXR/AR/LED with real floor workflow

   KNodlin

Comments

TwentyStudios
  -  

Looking at the screen shot, it looks like the foreground stage is simply an AR overlay. The entire set extension is rendered on top of the camera capture, so instead of masking out the floor it’s just covering by the virtual extension in the foreground. That way it doesn’t have to align precisely with the physical floor either. 

Eifert@Aximmetry
  -  

Hi,

You can use the LEDWallCam+AR_3-Cam_4-Wall.xcomp compound to render the AR overlay part as well. You can find more information about this here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/ar-production/aximmetry-de-scene-setup-ar/#ar-in-green-and-led-wall-projects
Alternatively, you could use a separate machine to render the AR overlay to save on performance. Details on this can be found here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/led-wall-production/combine-different-productions-in-separate-machines/

Warmest regards,

KNodlin
  -  

Thank you for the reply, we achieved the result by using led wall 2 as a mask, but we cannot manipulate LEDs wall 2 (floor) edge softness separately, this is a little bit confusing, so we suffer from serious ghosting because of that. Also we noticed if we place the LED 1 (screen) in right position with auto placer and manual tweaking - it will displace itself in a critical offsets when we approach/distance from the screenXR/AR/LED with real floor workflowXR/AR/LED with real floor workflowXR/AR/LED with real floor workflow

TwentyStudios
  -  

What tracking system are you using? If it shifts like this it means that either your calibration isn’t precise enough or that your tracking system itself isn’t precise enough.

Regarding the floor part, I still think it would be preferable to hide the transition with an AR overlay, but you can easily edit the camera compound and add a Blurrer module to the mask output to feather the edge a bit. 

KNodlin
  -  

We are using mo-sys, we tried to calibrate with aximmetry's calibrator, then measure offsets manually to verify that they are acceptable

TwentyStudios
  -  

Maybe use the Mosys calibration instead? I think that’s more accurate than the Aximmetry one. Super precise calibration and tracking is essential for LED wall set extension. If your calibration isn’t precise you might offset the error when aligning the LED wall offset, which will look good from one position but will cause drift when you move the camera. 

KNodlin
  -  

It would be so nice of you if you will drop down some more details about the AR overlay to do the floor part also. Thanks

TwentyStudios
  -  

The AR overlay wouldn’t do the floor. You would just use the AR elements to hide the transition between the set extension and the masked out floor. The AR elements are rendered on top of everything. 

KNodlin
  -  
That's what we did, screenshot above. So the only problem is the calibration. Thank you very much, we will calibrate on our MO-SYS manually and will send the results in the thread.