Seeking Advice on Optimizing Keying in HTC Tracker Camera Tracking Setup

 

Hi,

We are a small team using the HTC Tracker system for camera tracking in the Studio version. One of the main issues we face is that our green screen often gets wrinkled and has uneven lighting(I have tried my best to handle it). The most challenging part when using a tracking camera is dealing with keying as the camera moves. Unlike with a fixed camera, where we can rely on a clean plate to handle everything.

But the studio version cannot use 3D clean plates. Could you please provide suggestions on how to better optimize keying in this situation?

I want to use multiple keying in passes,Is it feasible to chain multiple keying processes to make multiple adjustments for green screen keying? Would this significantly impact real-time performance?

If this method is feasible,Can you provide guidance on the steps for multiple keystrokes?

 Do you have any other recommendations?


Thank you for your help!

----alan

   alan.shi

 
Profile Image
dysoncompany
  -  
It sounds like a good solution, In the case of poor hardware environment such as uneven lighting on a green screen, expect a solution.
 
Profile Image
Eifert@Aximmetry
  -  

Hi,

First, you should try to address keying issues by following the documentation on keying: https://aximmetry.com/learn/virtual-production-workflow/green-screen-production/keying/chroma-studio-backgrounds/https://aximmetry.com/learn/virtual-production-workflow/green-screen-production/keying/chroma-studio-backgrounds/


If you find that you cannot achieve satisfactory keying results using the tips and available solutions in the documentation, you can consider modifying the camera compound for multiple keying passes.

Rendering additional masks imposes minimal performance impact. The additional keying processes have a greater impact, but they should not cause a significant performance decrease, even when handling 4K inputs.

To proceed with this modification, open the camera compound by selecting Edit Compound In-Place:
Seeking Advice on Optimizing Keying in HTC Tracker Camera Tracking Setup

Then double-click on it to enter it. Within, open the specific input you are using, such as INPUT %N=1. Then, navigate to Line %N. Here, you'll find the Keyer and Studio Mask compounds. While the appearance may vary slightly in the upcoming Aximmetry version (2025.1.0), the fundamental structure is similar to previous versions:

The Studio Mask compound renders the mask and you should enter it. What you find inside is that there are two cameras, one is rendering the mask and the other is rendering the preview. You can see this preview when you are in the Studio mode:
Seeking Advice on Optimizing Keying in HTC Tracker Camera Tracking Setup
The trick here is that one model can have multiple shader indexes. And the Camera modules can be set to only render a specific shader index.
This way you can render two different videos of the same model. For example, one renders a solid white model for the mask and one camera renders a helper texture on the model, making it easier to place the mask. Cameras have a Shader Index pin that specifies which shader to render:

And there is a Shader Array module that specifies the different shaders:

In the above picture, I just copied the Studio Mask compound that was already used by the camera's STUDIO. You can do it by alt-clicking on a node, this way the copied node will also copy the connections. And added a Rectangle module, Shader Array module, Basic_Solid shader, and the Measure shader.
After this, you can add many things. For example, add a Rectangle module as ground and only have it visible in shader 2 (monitor camera). This way you get a better sense of where is your mask in the real world.

You can then use this Mask with a different keyer in the Mask Mixers that follow the STUDIO MASK compound's out.


In the next version of Aximmetry (2025.1.0), probably the best approach is to leverage the Holdout feature. To do so, you should use a differently keyed input for the Mask Mixers that uses the Holdout mask:

Warmest regards,

;