Hi,

Have you tried the new playlist module?

It's very easy to use and you can program your camera switches once and recall them as many times as you want.

I've been using it for something very similar to what you suggests.

Best,

Eric

Hi Eric, 

Thanks for your quick response, like I mentioned in my post, I know of the new playlist module. But I am/was looking for a way to replay a recorded live scene with the exact same camera changes/switches, without having to re-create them using the playlist module (or start with that). As we switch live, we have no exact idea which camera/shot will be used etc.

@Aximmetry, I've come up with a solution that works, but would like to hear your thoughts. Also for other users that face the same issue.

This is what I've done:

1. Captured the input recording tracking data (.xdata) per Camera source and the FBX tracking file during the live recording
2. Captured the HQ green-screen camera source with time code on SSD via Hyperdeck 12G recorders
3. Opened the FBX file in Blender, changed the out-point to include the last key frame
4. Discovered that on a camera change, the Y-position of the camera in the key frame changed
5. Exported FBX from Blender to DAE (Collada) file, but included the key frames in the anim-section
6. Imported the DAE file with 'import options' to include camera
7. Entered the DAE file compound and exposed the Play/Stop/Restart input pins from the Sequencer
8. Also exposed the 'World Pos' to the right (output)
9. Hooked up a 'Vector Split' module because I only needed the Y value
10. Added a greater / less logic + Switch Integer module, basically saying: If Y-pos = here, then Cam 4, otherwise cam 5
11. Hooked that up to 'Playlist Select Cam'
12. Added some triggers that could trigger both the normal and this sequencer + a delay option
13. Set Camera's to Playlist mode on 'Camera Page'
14. Make some initial adjustments on the sequencer to match the recordings with the final output recording (so start is the same)
15. And bingo… When I play back the HQ recording, the camera now also switches on the exact correct moment

@Aximmetry... Although this is working in our case (fixed camera's), and you only have to set up the initial sync, I'm wondering if this can be done better?

Still learning Aximmetry, and happy that it's working, but the process is quite frustrating because of the lack of documentation and the endless possibilities. 

Hi,

Sadly, there is no ready-made solution for this. However, we have similar items on our request list, so we will consider adding it in future releases.

Alternatively, you could make your own save and load logic in the Flow Editor. Like, you could use the JSON Exporter and JSON File modules to load and save. And the System Params module to get the master timecode for timing.

Also, I am not exactly sure why the Y-position change happens, so note that your solution might not work in the long run.

Warmest regards,

Hi Eifert,

Thanks for your reply. 
The Y-position is changing because we are using 2 fixed PTZ's. So I think it is expected behavior. Please see the attached images.

Also, could you explain more about the JSON part and the system Param Timecode part, and how I would set this up?

Hi,

Ahh, now I understand, the PTZ cameras explain why your Y-position is constant.

To save camera switches, at first, you should set the Timecode Master to be one of your input videos:

This timecode will be read by the System Params' Master Timecode pin. And it will be used both for recording camera switches and playing them back:

Note, in the next major Aximmetry release, you will be able to generate a timecode master within the Flow Editor using one of the "... To Timecode" modules.

Then you can use the following logic to save and replay. It will save the timecode as a key at every camera change, and the ID of the camera will be the key's value.

This way of saving things is far from an ideal solution, but it should work for simple things.

Note, the replay uses the External Control Mode, which you need to turn on for the replay to work: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/green-screen-production/virtual-camera-workflow/cameras-control-board/#external-control-mode 

Warmest regards,

Thanks, for the extensive example, appreciated! I'm going to try to implement this today.

Hi Eifert,

I understand the logic for 95% here, as you are combining the time code with the currently selected cam, and on-change, writing that to a JSON. The saving part works perfectly, as it adds a line to the JSON every time I switch camera's.

But on replaying via the sequencer, it still displays the Master Time code of the incoming live signal, not from the recorded video. That's why I think that part isn't working yet.

Some extra info:

- PTZ 1 (cam 4) is sending time code and is set to Time code Master
- I've set the External Control Mode ofc (had to do the same with my FBX workaround)
- I've also tried using the restart trigger for the sequencer, but no effect here.
- There is a default? cam0 line in the JSON, I think when loading the project?

Any idea why the playback part (loading logic) is failing?

Hi,

What you are experiencing is actually a bug in the compound. Which will be fixed in the next major version.
You can easily work around this bug by making sure no other input is set to Master Timecode. And add a Sequence Video module to the Sequencer. Then set this Sequence Video's video file to the recorded input video you used as the master timecode while recording. And set the Sequence Video to Timecode Master:

There are going to be actually several Timecode improvements in the next major Aximmetry version.

And you should press the reset right after pressing recording. Otherwise, the initial state of the Selected Camera won't be recorded.

Sadly, there is no Transmit module that sends out that the recording started. You could use a playlist to start recording and triggering reset in sequential order: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/scripting-in-aximmetry/sequencing/playlists/

Warmest regards,

Yes, that trick indeed fixed that I am now seeing the time code of the source file. And when I play/pause in the sequencer, I see the correct time code.

However, still no switching of camera's. Looking at the JSON I should have passed a switch moment. Not sure why nothing is happening though. When I peek on the Collection of the JSON file (or JSON out) I don't see any info, feels like that is the issue.

Hi,

Yes, the problem will be with the JSON file module. Note, you have to give the file name with its extension, like Y:\test.json. Cause without an extension like this Y:\test, it will only work at exporting with the JSON Exporter module, but it won't work at importing with the JSON file module as that module requires file extension.

Warmest regards,

Ah,

You are totally correct, as I saw the output file I didn't think about this again. I'll try this and report back.

Thanks.

Hi Eifert,

It's indeed working now. However, it has one interesting behavior in the current setup.
That is, that every time you play back the video from sequencer, it's re-writing to the original JSON file and changing the timings. As the current logic is "write on change".

I think I'll have to implement some logic like"If in Playback mode, don't write to the JSON".

But apart from that and the time code bugs (with working workaround), it's working as intended.

Hi,

Ahh yes, I forgot to say that. The easiest way to separate import and export is to have an if module before the JSON Exporter module's Export pin.
Note, you can connect the Trigger data type pin to the If module (vector), and the trigger data will be automatically converted to vector and back to trigger without loss of information.

Warmest regards,

Hi Eifert,

We've installed the new 2023.2.1 version, are there any specific updates that would improve the above 'replay' workflow?

Thanks

Hi,

There wasn't anything directly related to this added.
Note, we have on our request list a complete solution for this as multiple people asked this before. And we are considering adding it in a future release.

Warmest regards,

Hi,

Ahh, sorry, I forgot about the Timecode issue.
Yes, it was fixed. You no longer need this Sequence Video module:

You just need to have the same INPUT for Timecode Master as the one you had as Timecode Master when you did the recording:

Warmest regards,

Hi Eifert,

We've installed the newest version (2023.2.2 build 25803), but I think there are new bugs related to time code, preventing proper working.
I'm trying to get the above working in this version (saving the camera switches), but I am experiencing strange things, which is resulting in the fact that the cameras are never switched.

My logic worked in the previous versions!
- When I peek the values of System Params Timecode, I get 2 different values from 2 of the same modules at the same time? But this shouldn't be the case and this seems to be the cause that when I play back, it's never 'hitting' the frames to switch the camera. This wasn't the case in previous version of Aximmetry. It's seems to be a bug in the timecode which is causing an offset in the timecode.

- I can confirm I have only 1 PTZ configured as 'Time code Master'
- I can peek the Master Timecode value both in Live and Playback mode correctly on 1 System Params Timecode module. (Not on the other one).
- It's not solved by keeping 1 instance of the System Params Timecode module
- Reloading the project corrects the timecode, but then goes bad again.
- Saving and loading of the JSON files with the selected camera is working correctly

I''ll send my project file to support and link to this post. As I can't share it here

Hi,

There is indeed something going wrong.
You need to go back to the old solution and use a Sequence Video module connected to the Sequencer as the one generating the Timecode Master:

This way both System Params should give the same Timecodes.

Thank you for reporting it, we are looking into the source of the issue.

Warmest regards,

Hi Eifert,

If I disable the PTZ 1 to be Timecode Master, and enable the Sequence Video to be the only Timecode Master, then indeed the both System Params have the same timecode. Apparantly the timecode of when Aximmetry launched. So far, so good. But when recording and then entering Playback mode, the System Param Master Timecodes will just keep on increasing. So it's not looking at the recorded Timecode and camera's will never change.

When I do the same but define PTZ 1 as Timecode Master, then when I enter Playback mode (and don't play the sequence), the Timeframe is stopped as you would expect. But then the other bug arises.

Hi,

You need to set the Sequence Video's video file to the recorded input video you used as the master timecode while recording.

Warmest regards,

Hi.

Just tried it. This is what I did:

- I set the PTZ 1 to be the Master Timecode, then record, switch camera's and set to Playback
- Now I set the sequence video to be the Master Timecode and point to the recording of PTZ 1
- I disable PTZ 1 to be the Master Timecode
- The Master Timecode of the 2 System Params now correctly show to a Timecode matching the recording
- I press Play on the Sequence and voilà, I see the all correct camera changes.
- I restart the Sequencer, and it stops working. Sometimes I see a camera change, but way less than the first time or not at all.

It seems to me there is a bug somewhere. Also, when playing back the sequencer, when I peek the values of the System Param Master Timecode, it seems to update slowly (like periodically), but when I view the live Timecode it's a fluent change of values. I'm not sure if it's expected behaviour? But I expect this to be the cause to miss most of the frames/changes.

Hi,

I should have said this. You need to reset the track when you change Sequence Video's video file. As it doesn't change the track in the Sequencer as in most use cases you don't want to lose any changes you made to the track. And since the track determines the length of the playback, it could be that this is why you experience it stop working. You can easily reset the track with this pin once you changed the video file:

What Timecode displays are not sequential counting of frames. You can make more sense of it if you put Timecode To Text module after it:

The resulting text will have the following format: Hour:Minute:Seconds:Frame (00:00:00:00) where the Frame counting resets at every second. More on it here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/scripting-in-aximmetry/sequencing/sequencer-and-sequence-editor/#15-timeline-timestamps 

The underlying problem is not a bug in the System Params or Timecode. The problem is in which order the modules are executed. The incorrect System Params get executed before the Master Timecode is overwritten by the playback because it is connected to a playback's input pin.

Warmest regards,

Hi Eifert,

No, that's not the issue. I understand how to interpret the timecode.

I've found the cause, perhaps you can now reproduce:
If you record the Green Screen in Aximmetry with Timecode to the DNxHR format, and use that as a playback source, when you peak the Timecode in the System Params will be updated infrequently and seem to skip timecodes. If I then playback the sequence it won't change correctly, as it seems to miss most of the frames.

When I record to for example H264/MOV, the peeking works as expected and the camera changes work everytime.
Also when recording to an external recording (our Blackmagic Hyperdeck 12G) in the DNxHR format works perfectly. So my logic was working, but there is an issue in combination with the DNxHR format recorded in Aximmetry.

Attached a sample video where you can see how the peaking is 'wrong'. The values aren't passing fluently as they do with another video format.

https://drive.google.com/file/d/14ugkcFxrsUD8n1nVT0-4GJ6_Y4jMZPXo/view?usp=sharing

Here is with another format and the timecode passes correctly:

https://drive.google.com/file/d/1YYmF9-ZE3xUBGsql9c-q72okh4NJSZUq/view?usp=sharing

Hi,

That is quite strange. I couldn't reproduce it.
Would be good to figure out if things go wrong at recording or at playback. Do you get back the same bad timecode If you playback the same DNxHR video in a new compound using a Video Player module?
Also, does the video of DNxHR lag too? And the input video is 10bit and recording is done in DNxHR 10-bit?

One more thing to keep in mind is that Aximmetry has to run at the same frame rate as the video you are recording, otherwise, there won't be a timecode for each frame. However, you have too big of lags in the timecode for this to be the source of the issue.

Warmest regards,

Hi Eifert,

Found the issue. The issue is with recording the file, apparently Aximmetry on this computer can't record 1 single cam in 2160p25 to DNxHR, it completely chokes. Although SSD has 3000 Mb/sec r/w speed and specs are beefy enough (AMD Ryzen Threadripper 3960X 24-Core Processor  (48 CPUs), ~3.8GHz / 128 GB Ram / Nvidia Geforce RTX 3090. So I guess that's why timecode is lagging. Doesn't make sense. Although it's solved by capture via the before mentioned Hyperdeck.

I do however have encountered another Timecode 'bug'. Perhaps you can reproduce this one:
As soon as I place a recorder and connect 'Rendering' to it or record in anyway, it will never use the Master Timecode from the file loaded in Sequence Video (our workaround), but default back to Aximmetry's built in Timecode. Which makes sure our camera switching logic won't fire.
(selecting 'Use Master TC' in the Video Recorder Module doesn't do anything in this case).

Simply playing back the sequencer works perfectly and timecode from the file loaded in the Sequence Video is read and used without issues in the System Params. But as soon as I record... 

cm-aximmetry: Having many cores doesn’t help if a process doesn’t scale well between multiple threads (like real time rendering on a game engine). That’s why a system with the highest single core clock speed is recommended for Aximmetry. A much less expensive 13th Gen Intel i9 will have better performance than a high end thread ripper.

Using an external recorder for real-time rendering is the correct way to do it, but if you’re just rendering using pre-recorded files you could just set a specific frame rate in the Recorder module instead of Real-time, which will make Aximmetry render all frames as fast as it can without being limited by real-time performance. 

Finally, for internal recording/playback we’ve had much better results with the CineForm codec option in Aximmetry. Much more performant and better image quality than DNxHR.

Hi,

This Timecode 'bug' is sadly the same issue as before. By connecting the Video Recorder video input pin to the camera compound, you set the camera compound (including the playback modules) to be executed before the Sequencer Video module.
I think the best thing to do at this point, is to connect directly to the Sequence Video's timecode output pin:
If you want your post-production recording to have the same timecode, you could also connect the Video Recorder's timecode input pin to the Sequence Video's timecode pin.
Also, this way you don't need to use Master Timecode at all in the post-production, so you could turn off Timecode Master in the Sequence Video module.

When Aximmetry doesn't have enough resources to record the DNxHR video, you should get warning messages like this in the log:
Video Recorder: 26 frame(s) dropped. Cannot keep a realtime encoding rate with the current settings.

Warmest regards,

Thanks Eifert,

This workaround solves it perfectly! Appreciated.

@TwentyStudios, thanks for the Cineform tip. For other productions we are already converting all the recordings from the BM Hyperdeck (DNxHR) to Cineform already, saves space and retains quality well.

Indeed, for real-time, using an external recorder is the way to go, which we also do for each camera source. It only came up because I wanted to record a short clip for testing internally. I'm aware of the nature of Aximmetry that it's not a multi-threaded optimized program, but a single core in this system can boost from base clock at 3.8GHz, to max speed at 4.5GHz which should be sufficient in my eyes. But hey, it's not an issue as we use the external recorders anyway. We're indeed rendering the sequencer in a non-realtime way, which now works perfectly with Eiferts workaround.