Virtual Tracking Stutters During Playback

 
  • Using a ​​laptop + external BMD (Blackmagic) capture card​​ to receive SDI feed from the camera.
  • Aximmetry occasionally throws the error: ​​"Can't keep latency..."​
  • Testing in an ​​AR scene​​:
    • ​RECORD mode​​: Tracking works fine, both video and tracking data are stable.
    • ​PLAYBACK mode​​: Video plays smoothly, but ​​virtual elements stutter in a rhythmic pattern​​ (not smooth).
  • ​Sync is connected​​, but the issue persists.

​Question:​
Could this be a ​​sync signal issue​​, even though I’m using genlock? Or is there another underlying cause

   CUC_STU

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

If you receive the "Cannot keep latency" message, you need to increase the In-to-out latency. You can find more information on how to adjust this here: https://aximmetry.com/learn/virtual-production-workflow/inner-workings-of-aximmetry/in-to-out-latency/ This issue likely arises because the default latency settings are not tailored for external capture cards.

However, when using Playback, In-to-out latency generally shouldn't affect it unless the problem also occurred during recording. If this is not the case, it's probably a performance issue. To fix it, ensure that nothing in the Processor Load Display panel is red.
Additionally, during playback, using Genlock is usually unnecessary.
When conducting post-production work in Playback, you can record in non-realtime mode, which allows you to record frames slowly without being limited by your hardware. You can find more about this process here: https://aximmetry.com/learn/virtual-production-workflow/setting-up-inputs-outputs-for-virtual-production/video/recording/how-to-record-camera-tracking-data/#offline-rendering 

Warmest regards,

 
Profile Image
JohanF
  -  

CUC_STU: By “Sync Signal” do you mean timecode or genlock? When you say that Sync is connected, where exactly is it connected? Genlock and timecode are completely different things. Unlike timecode, genlock isn’t a signal that gets embedded with the video signal. Genlock simply makes all equipment run with the same clock pulse reference, so when you (for example) genlock several cameras to an external clock source (not timecode), they will capture a frame at the exact same time. The genlock isn’t embedded with the signal itself, it just locks the camera on a hardware level. When using (most) camera tracking systems, genlock is important since it ensures that the tracking data is captured at exactly the same time as the camera captures a frame. With a 24 fps frame rate, not having genlocked tracking data would mean the tracking could be misaligned to the camera by up to 1/24 of a second. While this doesn’t seem like a lot, it will be quite noticeable in practice. 

Syncing the Decklink card doesn’t help with this at all, since it just genlocks the output of Aximmetry, after the camera signal and 3D environment have already been combined. 

 
Profile Image
CUC_STU
  -  

Additional test details:​
The test was conducted using a jib arm in a complete studio system where all video equipment was locked to an external clock source (tri-level or BB sync). A standalone laptop was connected to the system, receiving:

  • Video feed via SDI output from the jib camera head
  • Tracking data via Ethernet from the jib's tracking box
    The test used Aximmetry's built-in AR calibration chart scene, with no red alerts in the performance monitor.

Initially without sync connection, Aximmetry on the laptop showed proper virtual tracking alignment during live operation. However, recorded playback exhibited regular tracking stutter. Connecting a sync signal from another device to the external capture card didn't improve the issue.

Since local recording (tracking+video) doesn't require sync connection, what exactly does Aximmetry's sync option reference when outputting to external devices? Enabling it impacts performance - what other methods can I try to resolve the playback stuttering issue?

 
Profile Image
JohanF
  -  
The Sync checkbox on the output ensures that Aximmetry is rendering at the same frame rate as that output is set to. It shouldn’t affect performance. If it does, probably means that Aximmetry is set to render at a different frame rate than the output is set to. You can safely disregard genlock when troubleshooting this. I would start with making sure that Aximmetry is set to render at the same frame rate as the input signal. That means both the output and the Render settings. 
You haven’t mentioned where timecode is generated. Are you getting embedded timecode from the camera via the SDI input? Have you made sure you’ve selected this timecode as the TV source for the recording? 
What format/codec are you recording in? It might be that the laptop can’t handle playback without dropping frames. These frame drops won’t necessarily show up on the CPU/GPU or frame rate meters in Aximmetry. 
 
Profile Image
CUC_STU
  -  

Thank you for your response.

First, I have confirmed that the camera format is set to 1080i, and my laptop with Aximmetry is configured to render and output at 50i. There have been no errors regarding input and render mismatches.

Regarding the timecode you mentioned, I haven’t specifically set a timecode on the system, and my laptop does not have a timecode card installed. In previous tests where I used the laptop with Axi Eye and a phone for single-unit tracking data recording, I simply checked the "timecode sync" option in the input module and did not encounter any issues. Therefore, I didn’t realize the need for an external timecode source. Based on my understanding, this time, I only swapped the phone with the egripment, and the setup is still running as a single unit. Do I need to configure a timecode source for this setup?

Lastly, I tested the recorded data and video playback on a 4090 workstation, and the issue persists. Please refer to the video link for more details:
https://vimeo.com/1077545727?share=copy


 
Profile Image
Eifert@Aximmetry
  -  

Hi,

The video suggests that the issue may be due to bad tracking data.
Does this problem only occur when you are playing back tracking recordings with the PLAYBACK panel?
Virtual Tracking Stutters During Playback

If so, make sure that everything is set to the same frame rate as the recording.

While troubleshooting this, I recommend using the simplest setup possible, minimizing any additional systems or settings. Gradually reintroduce each system to identify which might be causing the problem. So, first, avoid using interlaced settings and genlock while troubleshooting.

Also, make sure the Tracking Dejitter option is turned off:
Virtual Tracking Stutters During Playback
This is the only setting in Aximmetry that makes corrections to the tracking data and hence, could cause such jumps as in your video.


Regarding timecode, it is not something that you must have, especially if you have already configured a genlock system. The primary advantage of timecode sync over genlock is that it eliminates the need to set a tracking delay value in Aximmetry. However, most hardware does not support timecode sync. Note that with genlock, you only need to set the tracking delay once, after your hardware is turned on and genlocked; after that, the delay will remain the same throughout the production.
Aximmetry Eye uses timecode because the same device (the phone) generates both video and tracking data. This allows tracking and video data to share the correct timecode without requiring an external device for synchronization.

Warmest regards,

;