MAJOR ISSUES WITH SHADOWS AND REFLECTION TRACKED SCENE

 

Dear Team,

I am currently conducting tests on billboards and reflections in Aximmetry. I am encountering two significant issues while using the trial version of Aximmetry Broadcast Edition, with Ncam serving as my tracking device.

I have set up a scene where the Talent and UE Mannequin are positioned similarly within the UE scene. Ideally, they should cast shadows and reflections from their respective positions. However, there appears to be an offset between the billboard’s location and the shadow/reflection. This issue persists in both scenarios. Generally, tweaking the shadow offset value would rectify this, but the difference is so substantial that the shadow’s scale relative to the mannequin is extremely small. I have tried all three auto-position options, and the Optical option seems to be the most effective.

In Image A, another issue arises where the shadow/reflection is cast from a single point. If the talents are even two meters apart, their shadows are cast at the same point, which is inaccurate. This issue is more prevalent when it comes to reflections, rendering it unusable.

In Image B, the problem mentioned in Image A persists. The shadow/reflection of the talent sitting on the chair and the one standing in front are cast at the same point. This makes the person on the chair appear to be floating and not grounded. Additionally, there seems to be an offset between the billboard and their reflection/shadow. The Z mirror offset doesn’t seem to be working for me, and the Shadow offset drastically alters the scale of my shadows. It’s not feasible to keep tweaking shadows during the shooting phase. I am curious about what causes this offset and whether this issue is specific to my setup.

I would greatly appreciate your assistance in resolving these issues.

Best Regards,

MAJOR ISSUES WITH SHADOWS AND REFLECTION TRACKED SCENE

MAJOR ISSUES WITH SHADOWS AND REFLECTION TRACKED SCENE

   kssvp.workspace

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

Unreal Engine has four different reflection methods. They are documented here, including their settings in Unreal and Aximmetry:
https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/green-screen-production/aximmetry-de-scene-setup-green-screen/#reflections-in-unreal-engine

We recommend using Planar Reflections to achieve the highest quality with billboards. Also, Billboard's Mirror Offsets only works with Planar Reflections. When using Planar Reflections, ensure that Render To Depth is turned off in the Billboard panel of the INPUTS control board. For other reflection methods, Render To Depth must be turned on. Correcting the Render To Depth setting should fix issues where reflections do not start from the bottom of the billboard.


The issue of two talents being on one billboard with shadows or reflections starting at the same point is a fundamental limitation of tracked green screen productions. To overcome this, use more than one billboard per input (one billboard per talent). More information on this can be found here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/green-screen-production/tracked-camera-workflow/tracked-camera-billboards-placement/#the-case-of-multiple-billboards
However, this solution works when the talents are not overlapping, unlike in your images.
For overlapping images of talents, you would need to cut the two talents' images apart, otherwise, the rendering engine can not tell where the talents start. Even in post-production, this would only be possible by time-consuming rotoscoping, frame by frame.




Note, that Unreal Engine offers several methods to render shadows, each with its own limitations. More information on this can be found here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/green-screen-production/aximmetry-de-scene-setup-green-screen/#shadows

Warmest regards,

 
Profile Image
TwentyStudios
  -  

@ kssvp.workspace: It should be noted that this is not a limitation specific to Aximmetry. Your footage is naturally in 2D, even if you’re using a tracked camera, so how could you possibly be casting shadows and reflections in 3D space? Without volumetric data (which no existing camera can provide) there’s no way to determine from where the shadow and reflection would be cast in 3D space. The best you can do is to track the position of a single person which would automatically place the plane at the correct position in the scene. 

Again, there’s nothing Aximmetry can do about this fundamental limitation of green screen compositing. You should light your talent to cast real shadows instead. If you want reflections, place a plexi glass surface under the talent on the floor. The Aximmetry keyer can handle this if you have a properly lit green screen. 

;