Have you calibrated the focus? What tracking system are you using? The idea with calibrating is to match your real camera focus to the virtual camera. If your real camera is out of focus on the green wall when you zoom in, the virtual camera would be out of focus to. If you want something in your virtual scene to be in focus, just adjust the focus on your real camera. 

Yes I calibrated the cameras, the question is: If there is nothing in front of the camera except the green and I zoom on it, the virtual background gets blurry because of the calibration, but I would like that focus is on a mesh behind the green for example.
So I should have either an infinite focus, or have the virtual camera focus on a virtual mesh that is far behind the green.

I don’t understand the question. If the camera focus is properly calibrated and the virtual background is blurry, it’s blurry because it’s supposed to be with the focus distance you have set on your physical camera. Of there is nothing in front of the camera and you want the background to be in focus, just set the focus on the camera where you want the focus to be. Or am I missing something? 

Oh, I am sorry, we are working with PTZ cameras and autofocus, so if there is nothing in front of the camera, the PTZ cam has the focus on the green.

Hi,

To detect when auto focus is not on the talent, you have several options, for example:

• Manual: You could press a button to switch from the tracked focus value to a virtual focus value.
  
• Keyed Input: Another approach is to create logic that detects when there is nothing in front of the green screen in your camera's picture. This is relatively straightforward: you only need to check if the keyed input contains many pixels that are not transparent. 
• Tracked Focus Distance: In the Flow Editor, you could create logic to detect when the tracked focus distance approaches the value corresponding to your green screen, or larger. This could be especially effective if your PTZ camera stays stationary and is not placed close to the green screen.
• Talent Tracking Device: If you use a talent tracking device, you can create logic in the Flow Editor to determine whether the talent (tracking device) is within the camera’s field of view.
  

I am happy to guide you through implementing any of these solutions, depending on which best fits your workflow.

After that, if you want virtual auto focus instead of infinite focus, note that Unreal Engine does not have this functionality by default. However, you can create one, similar to what is discussed here: https://www.youtube.com/watch?v=TvXJdDSHzro 
Alternatively, if you only want your virtual auto focus to focus on a single point, you can calculate the focus distance based on that point and the camera’s location. I can show you how to set this up.

Warmest regards,