Hi :) I am about to test the implementation of video footage as 3D assets in AR. I use Unreal for rendering my background and Aximmetry for the AR elements.
I‘d like to use video assets I generated with Beeble AI. It gives me diffuse, alpha, normal, depth, etc. of, for example, a portrait video loop of a soccer player.
I can not find out how to setup a shader in Aximmetry that works with diffuse and alpha (so it creates a mask for the footage) and also extrudes the footage with help of the depth or normal map.
My goal is that the footage does not look flat and I can hover around some degrees.
Can you help me out?
Thanks, best, Jan
Hi Jan,
Beeble AI provides an Unreal Engine plugin that you can easily add to the Unreal Editor for Aximmetry. You can find more about this plugin here:
https://docs.beeble.ai/integration/unreal-engine-plugin
For instructions on installing third-party plugins in Unreal Editor for Aximmetry, refer to:
https://aximmetry.com/learn/virtual-production-workflow/obtaining-graphics-and-virtual-assets/creating-content-in-unreal-editor/advanced-information-and-features/how-to-install-third-party-code-plugins-for-unreal-editor-for-aximmetry/
If you want to use the texture video generated by Beeble AI within Aximmetry's Flow Editor, that’s possible as well. Beeble AI generates PBR video textures, which should be compatible with Aximmetry’s PBR shaders found in the [Common]:Shaders\PBR folder. More details about these PBR video textures can be found here: https://docs.beeble.ai/beeble/video-to-vfx#video
To use these textures, follow the official Aximmetry PBR materials documentation: https://aximmetry.com/learn/virtual-production-workflow/obtaining-graphics-and-virtual-assets/creating-content-in-native-engine/pbr-materials/
Note: There are no open standards for PBR materials that everyone would use. So you will need to manually define properties such as roughness in the shader. In the Beeble Unreal plugin, these material values are already set by the provided shaders.
When working with depth maps from Beeble AI, you have two options:
Apply the depth map with *Disp* shaders to give your talent a more 3D appearance.
Combine the depth map with Aximmetry-rendered video using the Depth Mixer module. Note that this method only works with objects rendered by Aximmetry (not Unreal) and would support various occlusions by other rendered objects.
For both methods, you may need to process the depth map from Beeble AI to ensure it's the correct depth.
If you need further assistance or clarification, let me know.
Warmest regards,