Render the talent and scene separately

 

Hi, brother. Can I render the billboard and the 3d scene separately?

   Fullman

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

Yes, you can.
You have several options, depending on what you want to achieve with which camera compound.

For example, you could record the unmodified camera image and the tracking with tracked camera compounds. And then with these recordings, you can extensively modify and re-composite your final rendering in post-process: https://aximmetry.com/learn/tutorials/for-studio-operators/recording-camera-tracking-data/

Also, for example, you can just record the unmodified camera image and key it out in the post process using the [Common_Studio]:Compounds\Keyers\Keyer__All.xcomp compound:

And then you can use this when rendering the 3D Scene.
Note that above I use a fixed Frame Rate in the Video Recorder instead of realtime. This allows Aximmetry to render frames as fast or slow as it can. Ensuring a constant smooth frame rate in post-process renderings.

And for example, if you use an Unreal tracked camera. You can turn off Allow Virtuals and turn on Use Billboards in the INPUTS control board's SCENE panel. Then disconnect all the video going into the Unreal node in Aximmetry. And record B Texture, Unreal node's Out, and Unreal node's B Mask videos. This way you will record the current billboard without the Unreal scene (B Texture), record the Unreal Scene without the billboard (Out), and record a mask, that shows how much the billboard is obscured in Unreal (B Mask).

Note, you can do the same with scenes rendered by Aximmetry, but you will need to use bit different settings. If wanted, I can detail that too.

Warmest regards,

 
Profile Image
Fullman
  -  

Thanks for your detail explanation. I just use Aximmetry's virtual camera. Can I render the talent with alpha and the scene separately while record a video so that I can do some post processing in Adobe AE or Pr.

By the way, is there a Node or a Compound or something else that allows me to track the talent's hand movement like what I do in AE. For example, I want to track the talent's hand path then let a fire or a glow follow it without a hardware tracker.

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

Still, even with the virtual camera, you have many ways to do this.

In the case of the talent, are you trying to record it for live production or are you already recording the talent in postproduction?
If you are recording live production. You should probably record the input as it is, before keying and any cropping. This way you can change for example the keying's parameters and quality even in postproduction. For this, you can just use a Video Input module and a Video Recorder module:

After recording you can add the keying as mentioned in the previous post:

Or forexample connect it back to the Virtual Camera compound as a Test Input:

If this doesn't fit your needs for some reason, you can still just record the billboard coming out from the Virtual Camera for each camera:


In the case of the scene, what many of the users do and I forgot to mention in the previous post, is that they record the tracking data into an FBX and play it back in Unreal (or in a different render engine) for further changes. This way you can even change the Scene in post-production. For this, you can use the [Common_Studio]:Compounds\Record\Record_3-Audio compound:

Sadly currently it is not easy to use that FBX file in Aximmetry and you need to convert it to DAE. While in Unreal you can use the FBX in a sequence.

Another way is to record what you did in Aximmetry and play it back. For that, we don't really have a ready-made solution. You can try using the View->Macro, but it has very limited capabilities. Or you can save relevant camera transitions into a txt file, but that needs significant editing in the Flow Editor.

You can also just record the Out pin of the Unreal node in Aximmetry in live production, instead of trying to save the camera movements:

But this way you won't be able to re-render it in the post-production.
Even just re-rendering it for example in 8k instead of HD and then downsampling it from 8k to HD can significantly increase the quality of your video.
Note, if you re-render using Aximmetry, you will want to save this Out pin for your scene video in Adobe.

Also, if something will occlude your talent in the virtual world. Then you will want to save the B Mask pin too. As that contains the occluded pixels.


There is no ready-made solution in Aximmetry for hand tracking or motion tracking based on video image. Note that After effects is not real-time software, so it can do all these at non-real-time speeds.
Never the less, there a several compounds that do various kind of simple image detections at: [Common]:Compounds\Detectors\
Or there is also a Bounds Finder module, that can help you.

If you considering hardware tracker. Note that you don't need as accurate tracking system as you would want in the case of camera tracking.

Warmest regards,

 
Profile Image
Fullman
  -  

I appreciate you greatly for your  patience and detailed explanation. Wish you happy and lucky every day. Thanks again.

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

To add to this, you can preprogram changes between cameras and camera paths using the sequencer:
When the playlist is selected here:

This way you can render the whole video in non real-time.

Warmest regards,

 
Profile Image
Fullman
  -  

You are so warm hearted. Thanks again for your help.