Tracked camera billboard not rendering behind translucent materials

 

Hello, I'm new to Aximmetry and coming from a Unreal Engine game programmer background. I have a simple VP setup: I built a UE5 map that has a translucent object. I'm using Aximmetry Broadcast DE to receive a SDI input image from a camera. Moreover, I'm using VIVE Mars with 3 trackers: One for centering, one for camera tracking and one for object/talent tracking. The camera and the billboard tracking are working fine. However, when I step backwards on my green screen (to "walk trough" the translucent object and be behind it, just like some "portal" thing) the billboard is still rendering in front of the translucent object.  

Here are some screenshots:

(Vive tracker near center position)


(Vive tracker far away from center position and it should be behind the translucent object):



Here is how my Aximmetry compound is setup:


Here's the desired effect (the person should be behind the translucent object):

I manage to make it work on a simple Unreal Engine Virtual Camera with billboard. However, I can not make it work on a tracked camera setup.


Any ideas, please? 

Best regards,

   Saulo Soares

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

We got a little bit sidetracked in the other thread. Your problem might not related at all to focus and DoF.

What you should know is that when Allow Virtuals is turned off, Aximmetry renders the Billboard:
https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/green-screen-production/tracked-camera-workflow/scene-control-panel/#use-billboards-&-allow-virtuals 
In your case, you want to Allow Virtuals to be turned on, otherwise, Aximmetry will mask the rendered Billboard based on the B Mask pin:

And your translucent object is probably not included in this mask, that is why you see the billboard in front of it.


You can turn on Allow Virtuals in the virtual camera at the CAMERAS control board:

You can turn on Allow Virtuals in the tracked camera at the INPUTS control board:

Note, to turn on Allow Virtuals in tracked cameras, the Use Billboards must be also turned on.

If it still doesn't work with Allow Virtuals and Render to Depth turned on in the Billboard panel, then can you share the material of your translucent object? That could help a lot in figuring out where things go wrong.

Warmest regards,

 
Profile Image
Saulo Soares
  -  

I already have all this turned on but still not working. 

However, as you said on our last thread (https://my.aximmetry.com/post/3237-why-is-my-actors-billboard-not-behind-tr), changing the cam focus manually actually did the trick. As noted by you, I only want to change the camera focus, though. Thus, following the other link (https://my.aximmetry.com/post/1795-focus-control-with-tracking). I came with the solution:


Interesting enough, it only works if I set a "Cam Focus" in range of [1; 1,7] for the given scene (please note the chair behind the translucent material):


What I'm understanding now is that the camera focus should be at max on the translucent material distance from the tracked camera. If the camera focus goes beyond it, it won't render the talent behind the translucent object... I notice this as I can barely see the talent being out of focus when the tracked billboard is nearly passing the translucent object to be rendered behind it.

Anyway, here is my material on Unreal (although I have tried with different glass material and none of them worked):


 
Profile Image
TwentyStudios
  -  

Try r.Translucency.AutoBeforeDOF to 1.

You can use the Execute Console Command node in the level blueprint to automatically set r.Translucency.AutoBeforeDOF 1 in your project.

 
Profile Image
Saulo Soares
  -  

The problem still remains...

Today I moved my camera forward to calibrate and I noticed that now I have to change the camera focus so the talent is rendered behind the translucent object. Thus, indeed, the camera focus distance should not go further than my translucent object.

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

Thank you for sharing the material. I managed to recreate the exact problem you are having:

Note that this behavior can also affect other translucent materials, like the red rectangle in the screenshot.

The source of this problem is that your material's translucency is set to be calculated before the Billboard's translucency.
The simplest fix is to set your material's Translucency Pass to After DOF:



However in this case, at a high focus distance (DOF) value, the refraction of your material will work incorrectly on the other translucent materials:

You can increase that at which focus distance value this happens by setting r.Translucency.AutoBeforeDOF to 0.


You can fix the refraction issue by setting the Billboard's Translucency Pass to Before DOF. In this case, you can leave your refractive material's Translucency Pass at Before DOF.
You can find the Billboard's material in the materials folder of your camera's folder in Unreal:


You will have to do this for every translucent material in your scene that is behind your refractive material. Or they will have the original problem, just like the red rectangle in the above screenshot.

Warmest regards,

 
Profile Image
Saulo Soares
  -  

Hmmm, setting both materials to Before DOF and r.Translucency.AutoBeforeDOF to 0 indeed worked! Thanks :)