UE5 Beta: Double Distortion on video with Billboard turned on

 


This post is more of a bug report than an issue that needs fixing. While testing with the beta, I got this weird result when testing with the new beta:



This is the video image without billboards turned on, with Stype lens correction applied. Everything seems fine without billboard on, but when I turn It on, the camera image gets distorted (take a closer look on the tip of the whiteboard, you can notice the distortion there):



Was this supposted to happend? I've never encounter this when using billboards with version 4.27, so this might be a bug.

   tiagosilvadiretor

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

It could be that you have a Virtual path with zoom. Virtual paths are only active when Use Billboards are turned On.
So make sure you reset your Virtual path:

If the issue still happens, does turning on and off Lens Distortion has a similar effect? Or does turning on and off Allow Virtuals change the distortion?:

Warmest regards,

 
Profile Image
JNeumann
  -  

Hello, I am encountering the same issue. It happened in version 2023.3.2. Reverting to version 2023.2.2 didn't help either. Also the same issue. It happens when I turn on lens distortion (lens distortion file made with camera calibrator), use billboards and allow virtuals at the same time. Of course I checked that virtual camera paths are turned off. Has there been a fix for this that I am not aware of?

Thanks,

Joern


UE5 Beta: Double Distortion on video with Billboard turned onUE5 Beta: Double Distortion on video with Billboard turned onUE5 Beta: Double Distortion on video with Billboard turned on

 
Profile Image
Eifert@Aximmetry
  -  

Hi Joern,

At this point, I am not sure if what you and Tiagosilvadiretor are encountering is related to Aximmetry applying lens distortion or not.

Instead, it seems something within your Unreal project is affecting the Billboard or the camera, producing an effect similar to lens distortion. Could you test if the same issue persists when using a new Aximmetry Blank project?

If the problem still occurs with the Aximmetry Blank project, it is possible there may be an incorrect connection between the camera compound and the Unreal module in Aximmetry.

Additionally, do you observe similar distortion of the Billboard in the B Texture video pin when using both the Allow Virtuals and Lens Distortion?:

Warmest regards,

 
Profile Image
JNeumann
  -  

Hello Eifert,

the project I was using already was a Aximmetry Blank project. But I created a new one just to be sure. I don't think there may be an incorrect connection between the camera compound and the Unreal module in Aximmetry. I connected the pins as usual (Out Size, Control Data, B Texture, B Refl Texture, B Shadow Texture)  of the tracked cam compound with the unreal project as shown in the tutorial.

When I connect the B Texture video pin to Preview I get the same distortion of the Billboard. But only if Lens Distortion is turned on. "Use Billboards" and / or "Allow Virtuals" have no influence on that.

Any ideas?

Thanks,

Joern

UE5 Beta: Double Distortion on video with Billboard turned onUE5 Beta: Double Distortion on video with Billboard turned on

 
Profile Image
Eifert@Aximmetry
  -  

Hi Joern,

Unfortunately, my previous suggestion did not make us fully understand the source of the issue. We are still trying to identify the root cause.
If possible, sharing the calibration file with us might provide some insight.

Additionally, try conducting the previous test with the keying turned off:

It would also be beneficial if you could provide an image of just the input:
UE5 Beta: Double Distortion on video with Billboard turned on

Without seeing the input, it's challenging to understand how the distortions manifest. The input should ideally mirror the final output when keying is deactivated. Moreover, if the Edge Expand setting is adjusted correctly, there should be no cropping visible.

If you want to be very professional about it, you can put the Aximmetry calibration board in front of the camera instead of the mannequin doll (talent). If the lens calibration is correct, the calibration board should have straight lines when looking at the video image of the B Texture pin. As it will undistort your camera's lens's distortion.

Warmest regards,

 
Profile Image
JNeumann
  -  

Hello Eifert,

it seems that if I turn on LensDistortion and Billboards together that the lens distortion is doubled instead of undistorted. 

Here are the two screenshots of the input (screenshot 1) and the B texture pin when connected to Preview  (screenshot 2). I will send the lens calibration profile via e-mail.

Thanks,

Joern

UE5 Beta: Double Distortion on video with Billboard turned onUE5 Beta: Double Distortion on video with Billboard turned on

 
Profile Image
Eifert@Aximmetry
  -  

Hi Joern,

It looks more like quadrupled at the B texture pin.
We examined your calibration file and it seems to be incorrect, applying a distortion different from what your camera actually requires. This likely explains the significant distortion in the wrong direction observed in the video of the B texture pin.

However, there seems to be another issue that probably causes this distortion to be not correctly reverted once Unreal renders the image. The reason for this we are still not exactly sure. It would help a lot if you could test if the Lens Distortion is bad in every zoom level (focal length) of your camera and if you could provide a screenshot of your INPUT settings like this:

Warmest regards,

 
Profile Image
JNeumann
  -  

UE5 Beta: Double Distortion on video with Billboard turned on

 
Profile Image
JNeumann
  -  

Hello Eifert,

I just checked, this happens at every zoom level.

 
Profile Image
Eifert@Aximmetry
  -  

Hi Joern,

Upon further review, it appears there was an oversight on my part, and there is no problem with your calibration file.
I’m currently unable to pinpoint your issue with Lens Distortion when using Allow Virtuals.
We will send you an mail requesting some further information.

Warmest regards,

 
Profile Image
chinzillafilms
  -  

Hi Eifert and JNeumann,

we also experience this issue and are not able to use the Billboards with lens distortion activated any more. We habe some projects in the pipeline and need this to work soon. Do you have any updates?


Best, Jan

 
Profile Image
Eifert@Aximmetry
  -  

Hi Jan,

This is caused by the Video Input and the Rendering Frame size not being at the same resolution. We are currently working on a fix. Until then, you can resolve this by setting the resolution to the same. Or by editing the camera compound, which I can guide you through if needed.

Warmest regards,

 
Profile Image
chinzillafilms
  -  

Hi Eifert,

I just found out that in the newest Axi-Version the issue has not been fixed. Could you please guide me through editing the camera compound? We have a very importing project next week.

Thank you, best, Jan

 
Profile Image
Eifert@Aximmetry
  -  

Hi Jan,

Yes, unfortunately, this didn't make it into the last release.

To fix it on your own, you have to edit the camera compound:
UE5 Beta: Double Distortion on video with Billboard turned on

You can find more information about editing compounds here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/scripting-in-aximmetry/flow-editor/compound/#edited-compound

Then, double-click on the camera compound to enter it. Find the INPUT sub-compounds you are using, and edit them as follows:
UE5 Beta: Double Distortion on video with Billboard turned on

Inside the INPUT compounds, enter the Line compound:

Finally, add a Measurer module here and connect it like this:

Warmest regards,

 
Profile Image
Eifert@Aximmetry
  -  
Hi,

This issue has been resolved in the latest version of Aximmetry (2024.3.0 BETA).
It fixes the problem where, if both Lens Distortion and Allow Virtuals were enabled and the camera input resolution differed from the Rendering Frame Size, the lens distortion was incorrectly applied to the billboard.

Warmest regards,
;