AR pop-out effect

 

Hello Aximmetry.

I was wondering what would be the approach to achieve the following effect:

https://www.youtube.com/watch?v=ZtsySq1ZGIQ (0:12)

Mainly I am concerned about two questions:

1The picture frame as a 3d object shows an entire world behind it, yet it is masked in some way outside the borders of the frame. How would you achieve masking the interior of a certain element, such as this, or a hole in the ground or etc.

(I have tried using a simple worldPosition mask material, as suggested here

https://my.aximmetry.com/post/1331-augmented-reality-template---reveal-ar-o

but it masks out the entire object from a certain point in space).

2. In the case that i would like to use a 2D video on a physical screen, instead of the 3d element such as the frame, and make the air-plane pop out as an AR element right at the moment it has reached the surface of the screen in the video. What could be the best approach for doing so?


Thanks alot.

Andrei.


  

   Blackfish Studio

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

There are several ways to do this. The methods listed here are the simplest I could think of. If they don't work with your setup, then there are probably other solutions that will better fit your needs.


Scenes rendered by Aximmetry (SE) in AR:

You create a window by hiding the rest of the scene. You do this by rendering a mask with the Camera module's mask feature.

To render the mask use [Common_Studio]:Shaders\VCam_Billboard.xshad shader and 4 big rectangles:

Place the 4 rectangles around the window:

Make sure that the rectangles are big enough that the camera will never see behind them. They don't have to end like the markings in the above screenshot. Or you can use any other model with the shader to function as a mask.

And connect the Camera module's Mask and Out output pins the following way to the AR camera compound:

The airplane model has to be in front of the rectangles in the world position when leaving the window.


Scenes rendered by Unreal (DE) in AR:

You create a window by hiding the rest of the scene. You do this with a material that is set to AlphaHoldout Blend Mode and set to Unlit Shading Model:

Then you apply the material around the window on meshes. The simplest way is to use 4 basic plane meshes around the window:

And place the 4 planes like this:

Make sure that the planes are big enough that the camera will never see behind them. They don't have to end like the markings in the above screenshot. Or you can use any other mesh with the material.

The airplane model has to be in front of the planes in the world position when leaving the window.

In Aximmetry you set up the Flow Editor as with any other Unreal AR scene:

Warmest regards,

 
Profile Image
Blackfish Studio
  -  

Thank you so much for the detailed explanation. 

You absolutely nailed it.

The UE solution works like a charm.

 
Profile Image
Blackfish Studio
  -  

Hi again. 

Iv tried doing the first version (where the Aximmetry is the renderer), and it works just fine, however, the mask plane leaves dark visble borders.

Is there anyway to make it completly trabsparant?

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

It is likely because you used the masker after the Camera compound.
You should send the already masked video to the Camera compound like here:


If it still does not work or you need this to happen after the Camera compound, then can you send a screenshot of your Flow Editor connections so I could better understand your situation.

Warmest regards,

 
Profile Image
solovai
  -  

@Blackfish Studio


How to do this now? Can share a tutorial?

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

We made a little documentation based on the things that were said here.
You can find the documentation here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/ar-production/ar-mask/ 

Warmest regards,

 
Profile Image
WillDylan
  -  

Hi Eifert,

We notice that the ar-mask doesn't work if the mask is set on a movable object. Do you know how we can set this to work on movable object?

Thank you

Best regards

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

Are you sure it is caused by the moveable setting?
I checked it in Unreal 5.3 and it is working with the moveable setting too. You can use Unreal 5.3 with Aximmetry 2023.3.0 Beta.

Also, you have to set Translucency Pass to Before DOF in the ar mask material in the new version of Unreal:

Warmest regards,

;