XR/AR/LED with real floor workflow

 

Hi, i'm interested in a particular workflow that were used in this case.

I understand the LED wall with digital extension workflow, but dont understand how to create a mask for the floor under the LED wall without soft edges (blurry edges) so that it fits in to the geometry of UE.

XR/AR/LED with real floor workflowXR/AR/LED with real floor workflow

   KNodlin

 
Profile Image
TwentyStudios
  -  

Looking at the screen shot, it looks like the foreground stage is simply an AR overlay. The entire set extension is rendered on top of the camera capture, so instead of masking out the floor it’s just covering by the virtual extension in the foreground. That way it doesn’t have to align precisely with the physical floor either. 

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

You can use the LEDWallCam+AR_3-Cam_4-Wall.xcomp compound to render the AR overlay part as well. You can find more information about this here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/ar-production/aximmetry-de-scene-setup-ar/#ar-in-green-and-led-wall-projects
Alternatively, you could use a separate machine to render the AR overlay to save on performance. Details on this can be found here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/led-wall-production/combine-different-productions-in-separate-machines/

Warmest regards,

 
Profile Image
KNodlin
  -  

Thank you for the reply, we achieved the result by using led wall 2 as a mask, but we cannot manipulate LEDs wall 2 (floor) edge softness separately, this is a little bit confusing, so we suffer from serious ghosting because of that. Also we noticed if we place the LED 1 (screen) in right position with auto placer and manual tweaking - it will displace itself in a critical offsets when we approach/distance from the screenXR/AR/LED with real floor workflowXR/AR/LED with real floor workflowXR/AR/LED with real floor workflow

 
Profile Image
TwentyStudios
  -  

What tracking system are you using? If it shifts like this it means that either your calibration isn’t precise enough or that your tracking system itself isn’t precise enough.

Regarding the floor part, I still think it would be preferable to hide the transition with an AR overlay, but you can easily edit the camera compound and add a Blurrer module to the mask output to feather the edge a bit. 

 
Profile Image
KNodlin
  -  

We are using mo-sys, we tried to calibrate with aximmetry's calibrator, then measure offsets manually to verify that they are acceptable

 
Profile Image
TwentyStudios
  -  

Maybe use the Mosys calibration instead? I think that’s more accurate than the Aximmetry one. Super precise calibration and tracking is essential for LED wall set extension. If your calibration isn’t precise you might offset the error when aligning the LED wall offset, which will look good from one position but will cause drift when you move the camera. 

 
Profile Image
KNodlin
  -  

It would be so nice of you if you will drop down some more details about the AR overlay to do the floor part also. Thanks

 
Profile Image
TwentyStudios
  -  

The AR overlay wouldn’t do the floor. You would just use the AR elements to hide the transition between the set extension and the masked out floor. The AR elements are rendered on top of everything. 

 
Profile Image
KNodlin
  -  
That's what we did, screenshot above. So the only problem is the calibration. Thank you very much, we will calibrate on our MO-SYS manually and will send the results in the thread.
;