Beeble AI for Aximmetry Render - Forum

Beeble AI for Aximmetry Renderer

Back

Hi :) I am about to test the implementation of video footage as 3D assets in AR. I use Unreal for rendering my background and Aximmetry for the AR elements.

I‘d like to use video assets I generated with Beeble AI. It gives me diffuse, alpha, normal, depth, etc. of, for example, a portrait video loop of a soccer player.

I can not find out how to setup a shader in Aximmetry that works with diffuse and alpha (so it creates a mask for the footage) and also extrudes the footage with help of the depth or normal map.

My goal is that the footage does not look flat and I can hover around some degrees.

Can you help me out?

Thanks, best, Jan

chinzillafilms

Comments

Eifert@Aximmetry

Hi Jan,

Beeble AI provides an Unreal Engine plugin that you can easily add to the Unreal Editor for Aximmetry. You can find more about this plugin here:
https://docs.beeble.ai/integration/unreal-engine-plugin
For instructions on installing third-party plugins in Unreal Editor for Aximmetry, refer to:
https://aximmetry.com/learn/virtual-production-workflow/obtaining-graphics-and-virtual-assets/creating-content-in-unreal-editor/advanced-information-and-features/how-to-install-third-party-code-plugins-for-unreal-editor-for-aximmetry/

If you want to use the texture video generated by Beeble AI within Aximmetry's Flow Editor, that’s possible as well. Beeble AI generates PBR video textures, which should be compatible with Aximmetry’s PBR shaders found in the [Common]:Shaders\PBR folder. More details about these PBR video textures can be found here: https://docs.beeble.ai/beeble/video-to-vfx#video
To use these textures, follow the official Aximmetry PBR materials documentation: https://aximmetry.com/learn/virtual-production-workflow/obtaining-graphics-and-virtual-assets/creating-content-in-native-engine/pbr-materials/
Note: There are no open standards for PBR materials that everyone would use. So you will need to manually define properties such as roughness in the shader. In the Beeble Unreal plugin, these material values are already set by the provided shaders.

When working with depth maps from Beeble AI, you have two options:

Use as Displacement Maps:
Apply the depth map with *Disp* shaders to give your talent a more 3D appearance.
Mix with Rendered Video:
Combine the depth map with Aximmetry-rendered video using the Depth Mixer module. Note that this method only works with objects rendered by Aximmetry (not Unreal) and would support various occlusions by other rendered objects.

For both methods, you may need to process the depth map from Beeble AI to ensure it's the correct depth.

If you need further assistance or clarification, let me know.

Warmest regards,

david libraty

Hi Eifert, Is it possible to explain more clearly how to get a video texture to connect to the video test button of the AXIMMETRY virtual camera in the Flow Editor?

Eifert@Aximmetry

Hi David,

Could you explain in a bit more detail what you are trying to achieve:
Are you looking to use Beeble AI as well? If so, do you want to incorporate it as a billboard in your green screen production (virtual or tracked?), or as a separate object within your scene? Additionally, is your project rendered in Unreal or Aximmetry?

Note: The virtual green screen camera compound’s test input requires a standard video with the talent in front of a green screen. If you want to use the various textures generated by Beeble AI, you will need to make significant edits to the camera compound (or even modify the camera blueprint if using Unreal).
If you simply display it as an object separately, rather than as a billboard within the camera compound, far fewer modifications are needed, and you won’t need to edit the camera compound at all. You can still achieve billboard-like behavior for the separate object, but you will need to set up a separate control board panel for it. Otherwise, I don’t see much reason to use Beeble AI-generated textures as billboards within the camera compound, rather than applying them to a separate object that behaves like a billboard.

Warmest regards,

Beeble AI for Aximmetry Renderer

Comments

Cookie consent