Why is my actor's billboard not behind translucent object?

 

Hello, I'm new to Aximmetry and coming from a Unreal Engine game programmer background. I have a simple VP setup: I built a UE5 map that has a translucent object. I cooked the map and on Aximmetry I play a recorded video (for now) of an actor on a green screen. However, when I position my camera's billboard behind the translucent object, it keeps rendering in front of it, and not behind - which would be nice to have some kind of blur:



Here is my simple Aximmetry compound setup:



However, with some kind of luck, I managed to make it work on a initial test (on a different compound with the same UE5 cooked content). But I can't seem to replicate it on this new compound, although I know it's possible.


What do I have to do so my billboard is rendered behind the translucent object? What am I missing?

   Saulo Soares

 
Profile Image
TwentyStudios
  -  

Did you set up the project with Global Clip Plane and Custom Depth Stencil Pass: Enable with Stencil? In the Cameras Panel, you should also enable Allow Virtuals.

 
Profile Image
Saulo Soares
  -  

Yes, all that is currently setup correctly. I still don't know how I manage to make it work on a compound and in another it does not work even with the same UE5 project. I mean, both compounds are really basic.

 
Profile Image
TwentyStudios
  -  

Did you try enabling Allow Virtuals like I suggested?

 
Profile Image
Saulo Soares
  -  

Yes, I did.

I actually discovered that when I use a "CameraSequencer.xcomp" and my Camera 1 Path in "SEQ" it just works somehow... But when I just leave my Camera 1 at path 1 it does not.

I'll just keep using CameraSequencer, I guess.

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

This is quite strange, the only things that change when you switch between SEQ or Camera Path are the Camera position, Zoom, and Focus.

I would say that the focus is causing it somehow.
When using SEQ, you could change the focus to 20 here:

And see if the problem returns.

Note that 20 is the default value of the focus in non SEQ paths. But you might have changed it, in that case, you can see the current focus by peeking here:

You can change the focus in Camera Paths with Alt + Space + Left Mouse: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/starting-with-aximmetry/aximmetry-composer/camera-mover-mouse-control/

Warmest regards,

 
Profile Image
Saulo Soares
  -  

Hmmm, indeed. That's what the problem was. If I set the Cam Focus Distance to 20 using SEQ the problem returns.

Good to know, I will explore more about the camera focus distance.

Thank you!

 
Profile Image
TwentyStudios
  -  

@Eifert: This seems to be a bug, right? 

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

I don't think it is a bug. The focus is being transmitted to Unreal correctly.
I don't know why the billboard gets blurred like that from the focus. Probably it has to do something with the DoF settings in the translucent material in front of the billboard.
Also, turning off or on the Render To Depth might have fixed the issue even at the original focus values:

Warmest regards,

 
Profile Image
TwentyStudios
  -  

@Eifert: Well, the Billboard disappearing at certain focus distances isn’t an expected or desired outcome, so I think it’s something Aximmetry should address. Rendering issues with the Billboard have become rampant since Unreal 5 (especially with Lumen), forcing users to constantly come up with workarounds and compromises to make it work. Since you’re running a custom build of Unreal, I suggest doing a deep dive into the Rendering pipeline to figure out a better way to render the Billboard inside Unreal without causing these issues. Other similar software doesn’t have these issues, so we know it can be done. 

 
Profile Image
Eifert@Aximmetry
  -  

Hi TwentyStudios,

I don't think this issue is actually related to Aximmetry. It could be how the vanilla Unreal works with the DoF related updates in Unreal 5.2.

Nevertheless, quite a lot has changed in the recent versions of Unreal. The changes in DoF in 5.2 are just one at the end of a long list. We will try to find all the new issues.
Some of these issues are actually caused by the fact that Aximmetry deep dives into the Rendering pipeline and makes changes in it for better quality billboards. It can make things much more complicated than just having a simple plugin that works within the intended limits of Unreal.
Note, that we will never have this many problems with Aximmetry's own render engine, as it is solely made for virtual production.

Warmest regards,

 
Profile Image
TwentyStudios
  -  

@Eifert: I understand that the issue happens in Unreal, but it’s still the Aximmetry integration and pipeline modifications in Unreal that are causing issues. What I’m suggesting is that you try to find a more problem free integration with Billbords and in-engine compositing in Unreal, or at least put more effort into finding and fixing rendering issues with new UE5 updates. Right now it feels a bit like a house of cards, with the constant workarounds we have to do. We simply expect to have the same quality of UE5 integration as your competitors do. Switching to Aximmetrys own rendering engine is not an option for us.

 
Profile Image
Saulo Soares
  -  

Hey, guys! I don't want to step into the discussion, but I started having the same issue for a tracked cam.

I have three Vive Trackers setup: One for centering, one for camera tracking and one for object/talent tracking. The camera and billboard tracking are working fine. However, when I step backwards on my green screen (to "walk through" the translucent object) the billboard is still rendering in front of the translucent object. 

Here are some screenshots:

(Vive tracker near center position)


(Vive tracker far away from center position)


Here is how my Aximmetry compound is setup:

I tried to find the "Cam Focus Distance" and didn't find it. At the same time, I can't use sequencer here as the camera is being tracked.

Moreover, I tried turning on/off "RenderToDepth" on Billboard 1A but nothing changed. 


Any ideas, please? 


Best regards,

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

You can change the focus in the tracked camera compounds by turning on the Manual Lens and changing the Manual Focus:

However, this will also make the Zoom to be manual. If you want only control Focus from Aximmetry you can do what is written here: https://my.aximmetry.com/post/1795-focus-control-with-tracking

We answered your underlying problem in the other thread here: https://my.aximmetry.com/post/3274-tracked-camera-billboard-not-rendering-b

Warmest regards,