Hi,

Do you have Tracking Dejitter turned on?

If it is turned on, you could try playback with it turned off. And see if that fixes the issue. You might have set values for Dejitter that cause problems in the playback.

Also, do you have in Edit->Preferences the same Frame rate when playing back as when you recorded the tracking?

And you could share the .xdataaux and .xdata files. These files are saved next to the recorded video and contain the tracking information which could help us figure out the issue's source.

Warmest regards,

Hi Eifert,

Thanks for your reply. All framerates are set to 25p whichis the same as the raw footage. I also checked the sequencer framerate and it is set to use the default project framerate.

Tracking Dejitter is always off as movement with the cameras can be wonky with it turned on. For elimination purposes, I have checked the rendering with Tracking Dejitter on and with it off and the results are the same. 

All the files I am using are in the below link:

https://drive.google.com/drive/folders/1Vr7lAnWaBwrONyaH6WlwMXNBhwqBwOjk?usp=drive_link

What is also curious is that any changes done to the cameras in via the Origin Pin Button that I use during the live recording do not translate correctly when the file is played back with the tracking info. The camera is not aligned to the background the same way it is during the show and I'd have to do corrections to move it in its proper place on the virtual set.

Thanks,

Joseph

Hi Joseph,

I found nothing wrong in the tracking data. I matched up the Wicked Game cam1 video to your video with the issue and went through the frames when your virtual background moved a lot.
You can actually easily check this, just go with the Sequence Editor to the frame where you experienced the sudden movement. But since I think it is not in the recorded tracking, you won't find such a frame and it will probably happen at random frames (time).

It could be that you accidentally set a virtual camera movement, go to CAMERAS and stop virtual camera paths:

The tracking recorder in the camera compound records the raw tracking data, so it doesn't record anything that is set in your project or in the camera compound.
The tracking recorder in the camera compound records the raw tracking data plus the calibration profile or the external lens data, more on it here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/setting-up-inputs-outputs-for-virtual-production/video/recording/how-to-record-camera-tracking-data/#tracking-data
The tracking recorder doesn't record anything that is set in your project or for example in the camera compound's ORIGIN,
This way you can easily change anything you did under the production. But it also means that it doesn't record any interactions with the ORIGIN panel.
Also, instead of changing the camera position by hand in the ORIGIN panel, you could use the virtual camera paths (as in the above screenshot).

There is no ready-made solution to record interactions with the Control Board during a show.
But you could easily save the INPUT settings before and after the show:

So it would be easier for you to recreate what happened under the show.

Note that the Sequencer's frame per second only affects the displayed time. It doesn't affect the playback or any logic in the camera compound or in the Sequencer. More on it here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/scripting-in-aximmetry/sequencing/sequencer-and-sequence-editor/#12-set-time-format  

Warmest regards,

Hi Eifert, 

Thanks for following up.

I have checked the virtual camera paths and they are on Stop. I don't think it should matter though, because I am using the TRK Inputs cameras to track the BG with the cameras. I followed the steps outlined here:

https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/setting-up-inputs-outputs-for-virtual-production/video/recording/how-to-record-camera-tracking-data/

Did the tracking glitches happen in your testing, or did the tracking work without the jumps? It is very peculiar that the tracking data is error free as from the screen grab below it seems that the tracking was not recorded for the frames in questions (ex 00:04:42:17). Infact, the BG jumps to another part of the unreal scene, outside of the virtual set we use and even the ProcessorLoad meters change drastically.

Screen grab here:

https://drive.google.com/drive/folders/1d9hr5g5GNsISA8i_G6zHdOaGLBctVDhK?usp=drive_link

Thanks,

Joseph

Hi Joseph,

That is quite strange, I tried the same frame (00:04:42:17) and didn't see anything going wrong.

You could write out the camera transformation that the Unreal receives from the TRK Input of the compound like this:

With the Collection Transformation module's Key pin is set to "Cam Transform".

You could try disconnecting these connections, and only leave the connection going from the Sequencer to the Playback Seq 9 pin:

It shouldn't matter, but I am running out of ideas.

If that doesn't work, I would suggest doing a new compound, adding the mixed camera compound and the unreal project module again. Then try to playback the tracked video and see if the issue arises again. This can tell us if things go wrong in some compound setting or not.

By the way, TRK Inputs can also have virtual camera paths. They are related like this:
Both types have the same panels for editing and playing back paths, so they can be quite confusing.

Warmest regards,

Hi Eifert,

Nifty trick writing out the camera transformations, thanks!

I have found the problem and it laid in the session that I use for the live show. As per your testing I created a new module with the same 8+8 compound and the tracking worked as intended. I therefore went back to my usual session and began reverting settings to their default one by one. The problem laid in the input menu of the tracked cameras, specifically the Tracking Device submenu. Since I used this session for the show, the device selected was still on the Antilatency tracker. 

In my case, the tracker is still connected to the pc (albeit with the floor tracking setup switched off) so I am guessing that when Aximmetry was playing back the scene using the recorded tracking data, it would occasionally switch to the physically connected tracker. The solution was to switch the Tracking Device to Mapped: #1. This also solved the issue of the camera framing, so I had to reset the changes done in the Origin pin and the camera snapped back to the framing of the show.

I have made a screen grab for anyone else who may encounters this problem.

https://drive.google.com/file/d/16tz5GviZEp2bdTw2JEsNL0FisfXxeNMy/view?usp=drive_link

Maybe you could include a solution in a future update whereby the actual physical sensors are excluded when rendering a sequence, or make the user choose?

Final question is whether or not you can record audio (or ideally the orginal timecode of the raw clip) via the sequencer as this would help immensely with syncing back to the other clips.

Thanks,

Joseph

Hi Joseph,

This is indeed not supposed to happen. It is going to be fixed in the next version of Aximmetry.
We apologize for any inconvenience this issue may have caused. Also, thank you for patiently going through all the possibilities, and finding the actual source of the issue.

I was wrong in saying that the recorded tracking data contains the raw data, that is only true when you turn off the External Lens Data: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/setting-up-inputs-outputs-for-virtual-production/video/recording/how-to-record-camera-tracking-data/#tracking-data

If you actually turn On External Lens Data during the playback, your issue won't happen.
Also, this means that your playback will include the calibration profile that you used during the show:

Instead of audio, you will have to save the timecode into a video file. Aximmetry can not handle audio timecodes, but we have the audio timecode on our request list.
You can set different Inputs as master timecode when recording and this way have all inputs recorded with the same timecode, you can read more about it in the Input Recording paragraph: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/setting-up-inputs-outputs-for-virtual-production/video/recording/how-to-record-camera-tracking-data/#timecode 
In playback, you can set any of the videos as master timecode, more about that in the Final composite recording paragraph:
https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/setting-up-inputs-outputs-for-virtual-production/video/recording/how-to-record-camera-tracking-data/#timecode-1
You can then save your final rendering with the master timecode by turning on Use Master TC in the Video Recorder module:

Note, that you can read or use up the Master Timecode using a System Params module:

Warmest regards,

Hi Joseph,

Thank you once again for bringing the issue regarding the jump in tracking playback to our attention. We are delighted to inform you that this has been resolved in the latest version of Aximmetry, 2023.3.1.

For your reference, the relevant changelog is as follows:

Fixed jumping/gliding when playing back recorded tracking data with External Lens Data = OFF and a frame is missing from the record.

Warmest regards,