Mo-Sys camera and lens tracking does not align graphic and real objects

Hello,

I am setting Mo-sys camera and lens tracking. If I use lens tweaker of Mo-sys to adjust in Unreal 5 then graphic and real object are align  position correctly. But when I get data tracking from Mo-sys and use External Lens data from Mo-sys then the graphics are bigger than real object. How can I custom you camera tracking like the camera from Mo-sys in unreal ( I use VP of Mo-sys to check).

Regards,

Thach

   ThachNN6

Comments

Eifert@Aximmetry
  -  

Hi Thach,

If graphics in Aximmetry appear bigger than the real objects, it likely indicates differing zoom levels in Aximmetry as compared to the real world. This should in theory be managed by Mo-Sys's tracking data, eliminating the need for manual adjustments.

You should make sure that you didn't set a Zoom Device or Manual Lens turned off when using External Lens Data as these settings can override the zoom level coming from the external lens data:

More on these settings here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/green-screen-production/tracked-camera-workflow/inputs-tracked-camera/#external-lens-data


If the Lens Tweaker changes apply only in Mo-Sys's Unreal, then there isn’t much you can do to bring them into Aximmetry unless they can be exported somehow. Alternatively, you might be able to recreate your use case for Lens Tweaker within Aximmetry, though I am not familiar with what Lens Tweaker can do.

Note that you can use Aximmetry's Camera Calibrator instead of Mo-Sys's Lens data. For more information, visit the Camera Calibrator's page here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/tracking/camera-calibrator/

Warmest regards,

ThachNN6
  -  

Hello,

In Zoom device I must choose Map 1# that the same Mo-sys tracking device when I set tracking  Device Mapping the it can control Zoom in graphic if not choose then It can not control although I have set turn on External Len data.

I also use camera and basic calibrate but it still does not correct.

I am trying to use Mosys lens tweaker and tweak with output Aximmetry but it is not easy beacause it need to update and deploy to Mo-sys device to check.

Could you please check if Camera in Unreal of mo-sys and Camer  of Axymmetry have the same parameter or not?

Regards,

Thach

Regards,

Thach

Eifert@Aximmetry
  -  

Hi Thach,

If you set a Zoom Device, the INPUT will use the calibration profile's zoom calibration (lens data) or fixed values. These will not be the same as those used by Unreal of Mo-Sys, as they will rely on their own Mo-Sys calibration (lens data).
Note: The diagrams at the following link illustrate when the tracking's lens data is being used and when it is not, as the External Lens Data switch does not have absolute control over this: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/green-screen-production/tracked-camera-workflow/inputs-tracked-camera/#external-lens-data

It seems you are not getting lens data from Mo-Sys at all and you should first figure out why is that happening. You can check the data being received in Aximmetry by following this guide: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/tracking/how-to-set-up-tracking-systems-in-aximmetry/


To my knowledge, all Mo-Sys systems send their tracking, zoom, and other lens data as a single package. Therefore, you do not need to set up a Zoom Device with Mo-Sys. However, if a different system is tracking your zoom, this could change. It would be very helpful if you could specify which Mo-Sys system you are using and the tracking protocol involved.

Warmest regards,



ThachNN6
  -  

Hi Eifert,

I use Mo-sys star tracker and currently I use Mo-Sys F4 data protocol so It send PTZR , Zoom, Focus from lens.

When I using Device Tracking Map I map it to Map 1 without Camera calibration file ( I do not choose Camera calibration file, it is blank). CAM1 Input I choose tracking device : Map 1, External Lens data: OFF then It can not use zoom, focus. Then I choose Tracking device Map 1,   External Lens data: ON then I can use Zoom, Focus.

I have Lens file of Mo-sys I will send you If you need to debug.

Regards,

Thach

ThachNN6
  -  

Hello,

If I don't use the MAP for Tracking device and choose Mo-Sys tracking , Zoom device : NONE , External Lens data is ON then Zoom, FOCUS and camera tracking is working. But graphic still  bigger than real object.

Can you help me to send data tracking directly to Aximmetry Camera in Unreal and does not processing before send to it?

Regards,

Thach

Eifert@Aximmetry
  -  

Hi Thach,

Then maybe the graphic being bigger than the real-world object might not be related to Zoom.

I suggest resetting all the settings in the camera compound. The easiest way to do this is to delete the camera compound and add it again to your project. Don't forget to make a copy of your xcomp file before doing so to save your previous settings.

After setting up your INPUTS and the relevant settings, try turning off Lens Distortion. This could be causing some of the discrepancies.
Mo-Sys camera and lens tracking does not align graphic and real objects


Once you have turned off Lens Distortion, Aximmetry Unreal should receive the same Zoom (FOV) values as the Unreal of Mo-Sys.

You can obtain this Zoom value from the Control Data pin using a Collection Scalar module by setting the Key pin to "Cam FOV". Then, convert this FOV value to Focal Length using the FOV Calculator module, and set its Param Mode pin to Horizontal FOV.

For example, this will allow you to easily compare the Focal Length to the value on your studio camera's lens.


You can send data tracking directly to the Aximmetry Camera in Unreal in a similar way by overwriting the Cam FOV key with a Set Collection Scalar module. However, this shouldn't be necessary unless you want to perform some custom Zoom interpolation or create a custom Zoom effect.

Warmest regards,

ThachNN6
  -  

Hi Eifert,

Thank you for your support. Mo-sys Star tracker send the data lens with lens distort. It is more accurate when use Lens Distort: ON I have tested with this parameter.

I also test with Basic Lens Caleb and Camera lens Calib then it still does not correct when using zoom and focus function

 I have tuned some parameter of Lens offset of Mo-sys then it is better.

Could you contact to Mo-sys to check Mo-sys star tracker to get correct tracking like Unreal VP Pro of Mo-sys.

Regards,

Thach 

Eifert@Aximmetry
  -  

Hi Thach,

I forgot to mention that when using the FOV Calculator module to show the Focal Length, you also need to set its Sensor Width pin to get the correct value.

We cannot connect a third party (Mo-Sys) without understanding the exact issue at hand. In this state, I am unsure where your underlying issue lies. You could consider connecting with Mo-Sys directly; they can probably provide general tips on how to map or tune their tracking's F4 data in their software.

I am also unclear on how you tested things with the Basic and Camera Calibrator. Note that if you perform a calibration with these tools, it will overwrite the data (Zoom and Focus interpolation and Lens distortion) from Mo-Sys's calibration.

Warmest regards,


ThachNN6
  -  

Hi Eifert,

Thank you for your support. I seems Aximmetry lens distort and Mo-sys Lens distort is different.

I have tried manual lens tweak then it is OK know.

regards,

Thach

TwentyStudios
  -  

ThachNN6: It shouldn’t be different and you shouldn’t have to tweak it manually. 

Sergio Gómez
  -  

Hi TwentyStudios,

I'm having similar issues with Mosys and they told me I have to override my sensor width.

Could you show me how to do it?

I'm using TrackedCam_Unreal_8-cam

Thanks in advance and best regards,

Eifert@Aximmetry
  -  

Hi Sergio,

Overriding the sensor width in Aximmetry does not affect the zoom level. It only changes how the aperture affects the depth of field (focus).
Note that adjusting the sensor width will also alter the displayed focal length (mm) value in the ON SCREEN Lens information of camera compounds, since the mm value is calculated based on the sensor width and the zoom (field of view).

It is possible that Mo-Sys intended for the sensor width to be overridden in their software or hardware. This would be necessary if they are calculating the zoom level in reverse, starting from the focal length (mm) values.

Warmest regards,

Sergio Gómez
  -  
Thank you Eifert, I will tell them what you answered me.
They say that when they work with other engines like vizrt, this parameter use to be changed
Anyway I would appreciate if you tell me how to override any lens value because looking at the documentation I can't find any example
Eifert@Aximmetry
  -  

Hi Sergio,

The Input panel includes the Override Sensor Width and Override Zoom parameters:
https://aximmetry.com/learn/virtual-production-workflow/green-screen-production/tracked-camera-workflow/inputs-tracked-camera/#override-sensor-width 
Note that Override Zoom was previously called Manual Zoom before Aximmetry version 2025.2.0, and there was no override option for Sensor Width in earlier versions; you could only set the Sensor Width in the calibration file.

Warmest regards,

Sergio Gómez
  -  

Hi, can you show me a screenshot about how I can connect a vector of 9 pins like documentation says
Mo-Sys camera and lens tracking does not align graphic and real objects

Do I have to do something like this?

Mo-Sys camera and lens tracking does not align graphic and real objects

The values I don't want to override. Must I use zeros?

Best regards,

Eifert@Aximmetry
  -  

Hi Sergio,

CX, CY, K1, K2, K3 with 0 values mean that they will change nothing. The other values must be accurately defined.

You can use the Copy module, but you should modify its In pin instead of the Default pin, or use a Vector Concat Multi module. (The Default pin sets the value for the In pin if you reset the In pin using the right-click context menu and Use Default pin is turned on.)
Alternatively, you can simply install the latest version of Aximmetry, which allows you to overwrite the Sensor Width within the INPUT panel.

Warmest regards,

Sergio Gómez
  -  

Hi Eifert, 

Is it possible to use Mo-Sys F4 data but overwrite the sensor width using the calibration file?

If I create a xml file like this, could it work or I have to change any parameter like TrackingCalibrated = False?

Mo-Sys camera and lens tracking does not align graphic and real objects

Thanks in advance and best regards,

Eifert@Aximmetry
  -  

Hi,

Since Aximmetry 2025.2.0, you can easily override the sensor width in the Camera compound’s INPUT panel. For more information, see:  https://aximmetry.com/learn/virtual-production-workflow/green-screen-production/tracked-camera-workflow/inputs-tracked-camera/#override-sensor-width 

Note that the sensor width only affects the aperture for focus calculations and the lens focal length (FoV in mm) displayed in the ONSCREEN lens information. It does not change the underlying field of view (FoV) value; it only alters how much that value is in millimeters.


You should be able to change the calibration text file without issues, as long as the changes don’t conflict with other settings/values. Setting TrackingCalibrated = False should work, but without knowing your specific workflow, I cannot guarantee it will work in your exact use case.

Note, in Aximmetry Camera Calibrator, lens calibration and tracking calibration values are not strictly dependent on each other. You can repeat the tracking calibration without redoing the lens calibration, as long as you are using the same camera and lens setup. For instance, if only the tracking device has been repositioned on the camera, you only need to recalibrate the tracking. The key requirement is that lens calibration must be completed before you perform tracking calibration.

Warmest regards,

Sergio Gómez
  -  

Hi Eifert, 

My client (university) didn't pay the maintenance plan, so the highest license they can use is 2023.3.2

The installation and cameras calibration was done by another external company. They told us their callibration was working well and the problem was on Aximmetry's side because the sensor width was incorrect.

Knowing that sensor width can't be overwritten directly in previous versions I was trying to put a calibration file so Aximmetry can only get Sensor Width = 13.2 and ignore the rest of parameters.

In the file created with camera calibrator I see other attributes like MInZoom = 0, MaxZoom = 0 and I'm afraid these can be a problem.

Perhaps setting TrackingCalibrated = False ignore those values, that was my question.

Does this make sense to you? 

Best regards,

Eifert@Aximmetry
  -  

Hi Sergio,

This doesn't make sense, because as I mentioned earlier, the sensor size does not affect lens or tracking calibration in Aximmetry. It only affects focus, but even that is not applicable in your Aximmetry version 2023.3.2, since aperture handling was not available at that time. I suspect the Mosys calibration was created with the wrong sensor size, which would explain why the calibration itself is incorrect.

You should set the Min and Max Zoom to match the minimum and maximum values used in your calibration:

Also, setting TrackingCalibrated = False likely means that tracking calibration was not performed, so the DeltaHead values are not reliable.


I suggest we take a step back and identify what exactly is causing the issue. Are you experiencing problems only with lens distortion, or is the tracking also not working as expected (real-world and virtual graphics fall apart when moving the camera)? If you are using Zoom tracking, is it functioning correctly?...

I recommend reviewing our recently updated documentation on tracking, which now includes several new pages on debugging and testing your tracking setup, including lens calibration. You can find these resources here: https://aximmetry.com/learn/virtual-production-workflow/tracking/testing-of-the-calibration/scene-setup/ 

Warmest regards,