@Andrew: I agree that the Camera Calibrator app is far too finicky and the results are often not acceptable. Doing the exact same calibration process for the exact same camera multiple times yields widely different results, which tells us that something isn’t right in the calibration process. I sincerely hope that Aximmetry can find a better, more robust calibration method. It shouldn’t be this hard to get right, especially for a paid product.

That being said, do make sure that you’re not adding an additional source of errors by making sure that the ReTracker bliss has an optimal environment for tracking, especially when doing the calibration. Check the Viewer app to confirm that it’s picking up plenty of feature points, also when the camera is pointed to the checkerboard on the floor. Place out additional markers on the floor and around the environment when doing the calibration and check that your exposure settings are optimal for picking up solid tracking points and not noise. A featureless green screen is not a good environment for tracking, so if you’re not pointing the tracker up or down and haven’t put out many extra markers, chances are that many of the tracking points are very noisy and unreliable. The ReTracker will work with what you give it, but it will definitely help to give it a good environment for accurate tracking. 

Another factor to at is often missed is to check if your floor plane in the UE5 scene you’re working with actually is at world zero. If the floor height is offset, tracking will not work as expected. 

Again, none of this should affect the results of the lens calibration. This is something the Aximmetry team needs to improve on their end. 

Hello Andrew,

so how was this eventually resolved? We're having exactly the same problem (objects on the floor visibly shifting in relation to the UE world when panning) with our lighthouse system here. We also found the automated camera calibrator useless (it introduces way too much distortion) and the basic calibrator very cumbersome...

Hi Stefan,

In your other post here: https://my.aximmetry.com/post/4338-lighthouse-tracking-ezttrack-jittering , you posted this video: https://www.dropbox.com/scl/fi/j303g415pn1hogtv7i88x/Tracking_Warble.mp4?rlkey=f4foyb9ooi300pi4gsg09st0r&e=1&st=vuh7jjwk&dl=0
I could barely detect any shifting in that video.
If the shifting is absent in the Aximmetry virtual floor (like in the video) but appears in your Unreal scene, it indicates that the floor in your Unreal scene is not set to zero height.
Note, that in the newest version of Aximmetry, you can turn on that virtual floor in any tracked camera compound's studio panel. If that virtual floor is shifting compared to the floor in the Unreal scene, that means the floor in the Unreal scene is not at zero height. You could correct the height even with the SCENE panel's Base Cam Transf parameter.

Note, even if the lens distortion wasn't calculated correctly in the Camera Calibrator—perhaps due to high focal length lenses—you may still use it for calibrating only the tracking (Delta Head Transf).

Warmest regards,

It's not that noticeable in the video showing the origin, but you can clearly see it here: https://www.dropbox.com/scl/fi/qkyb3yxob4u67luerbepy/CAM3_SWAN_40mm_unclamped.mp4?rlkey=u8z7wsxul5ceh66qaimes6lhh&st=kja4bdrx&dl=0

https://www.dropbox.com/scl/fi/p5ks0bcym6zlxjziouad7/Pan_Tilt_Jitter_Unreal_VIVE.mp4?rlkey=uq68iwpztw61gl37mc734yet3&st=69gd3n69&dl=0

(Sorry for the mismatched frame rate of the captured video, the monitor was running at 75 Hz vs a 30p render. fixed that now)

There are two issues showing here at the same time:

- real objects are shifting in relation to the virtual background even with only 40mm of focal length and moderate pan movements. I surmise that this is due to imprecise lens calibration, and we are actually looking at having the lenses calibrated externally in EZTrack now. Seems like the Lighthouse system we use is simply not precise enough for the basic calibrator. We can't use the automatic calibrator with our lenses either; it introduces a lot of distortion and the results are very inconsistent.

- There is an excessive amount of jitter that shows up as vertical "warble" while the camera is only rotated in the horizontal plane.

Hi Stefan. Please share the results after EZTrack calibration if you can. Curious to see if there's any difference. 

Hi Stefan,

At one point, there was a jump in the tracking data, which likely wasn't caused by the fluid head but rather by inaccurate tracking data.

In the second video, it is hard to tell what is causing the left-right shifting. We suspect the camera tilting could obscure the camera's tracker from one base station or make it visible to another, potentially leading to the issue. You should try to place the base stations at a high position, ensuring they are always visible to the tracker, or follow any guidelines provided by EZTrack for optimal placement.

I strongly recommend recording with Aximmetry. This way, the monitor's frame rate won't affect your recording. You can even capture EZTrack's interface using the Screen Capture module, for example:

Also, do not change the frame rate in the Video Recorder; keep it set to Realtime.

By investing time in mastering the Basic Calibrator and how to set each value, you can achieve near-perfect calibration that is possible with any particular tracking system. There is no inherent limitation in the Basic Calibrator that would cause significant tracking inaccuracies which other automatic tracking calibrator systems could somehow overcome.

Warmest regards,

@Eifert: There’s a lot of functionality missing in the Basic Calibrator that won’t allow perfect calibration. For example, I don’t think lens distortion is handled in the basic calibrator? Focus breathing isn’t accounted for either. The Advanced Calibrator fails more often than it succeeds, especially with zoom lenses, where it will wobble wildly when you zoom in/out. I suspect the center shift isn’t calculated accurately and/or there’s an issue with how the values are interpolated between the different calibration points. It seems each zoom calibration point is calibrated individually, without any regard for the previous calibration values, leading to further inaccuracies. I’ve used the Advanced Calibrator many times and I’ve talked to many other users that have experienced the same thing I have, which is that something with the Advanced Calibrator is fundamentally broken. Start with taking a look at the EZTrack calibration software and what they do to achieve accuracy lens distortion, sensor offset and absolute camera positioning. There is no way the Advanced (or Basic) calibrator comes even close currently. 

Hi TwentyStudios,

You can set center shift and radial distortion for lens distortion in the Basic Calibrator. These four values typically offer a good representation of lens distortion in most scenarios. Additionally, you can set calibration points for different focus levels which can mitigate focus breathing.
However, the issues Stefan is experiencing in the videos are not due to lens distortion or focus breathing. He likely faces more underlying issues, so it's not helpful to suggest that calibration problems are the cause here. But yes, I could have said that in a better way in my previous post.

We are aware that the Advanced Calibrator struggles with certain setups and we have received various feedback on this issue. Be assured, that we are working on improving it and supporting a wider range of setups.

Warmest regards,