I think I know what is going on.   It looks like it forcing me to use 60fps output (synced monitor output over HDMI) instead of 30fps causing my GPU to overload.  I am going to see if I can override it back down to 30fps

To help solve some of the problem, I had to uncheck the "SYNC" box on the monitor output which got me back to 30fps and the GPU usage was reduced, but I keep getting errors with my Blackmagic Decklink Quad HDMI recorder :  Cannot keep latency

Hello,

The update is causing an issue with my Blackmagic Decklink Quad HDMI capture card.  Please see the logs above in my previous post.  This card has 4 HDMI inputs and NO outputs.   Why is the logs showing six?

Arr] Opening device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[RenderQueue 0] Thread started

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Setting mode "AUTO" for device "Video: Blackmagic: DeckLink Quad HDMI Recorder (1)"

[Arr] Video: Blackmagic: DeckLink Quad HDMI Recorder (1): Cannot keep latency

[Arr] Video: Blackmagic: DeckLink Quad HDMI Recorder (1): Cannot keep latency

[Arr] Video: Blackmagic: DeckLink Quad HDMI Recorder (1): OK

[Arr] Video: Blackmagic: DeckLink Quad HDMI Recorder (1): OK

UPDATE:  I seemed to have solved this issue by setting the RENDERING frame rate inside Aximmetry DE Studio to match the capture card camera frame rate.  (In this case it was 29.97fps).  I never had to do this in previous versions, but this is what solved it.   

Hi guys, same problem over here, after the update nothing seems to work correctly, please can you help us?, we have an important presentation soon and obviously at this moment it seems to be impossible to do it.

Wow, it looks like it might be causing issues with all Blackmagic cards!   Hopefully they can help us!   I resolved it by matching my frame rate setting to my camera input frame rate and then matching the render frame rate to that same number.

@LiveGuyUSA, Gaston Alvarez:

It’s just a matter of setting the frame rate in the project to match the frame rate of the camera exactly. This is and always has been what you need to do to get professional results without stuttering. Also, the behavior for rendering outputs on the GPU has changed. Now, if you have the Sync checked for a GPU output, it will try to render the project at the monitor refresh rate. Either set the monitor to a refresh rate matching the camera input frame rate, or disable Sync. Better yet, don’t use the GPU for program out. A Decklink or similar will be better. 

It's even mentioned in the release notes :-)

• Removed special behavior of Sync in the case of PC monitor outputs. Now Sync always sets the system rendering framerate equal to the monitor frequency.
  
• When using Sync on a video output, Preferences / Rendering / Frame rate always shows the actual value, and cannot be edited.

Hi,

All 6 log is about the same port (1) of the device:

Probably the log is repeated 6 times because the same port is used 6 times in the compound.

If you get the "Cannot keep latency" error then you should increase the In to out latency:

Note that the latest version completely revamped the input and output timing system. And the default latency value has been a little bit decreased. In most productions, you want to keep this value as low as possible. You can read more about it here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/setting-up-inputs-outputs-for-virtual-production/latency-and-delay-in-aximmetry

Also, it might be that you set your input to a different frame rate than its actual frame rate. So you could first try putting your video input's Mode to AUTO. And see if that fixes the "Cannot keep latency" error before you change the in to out latency:

Output

If you are just editing and building up your project before production, then you don't need to use the sync. And you can run at the Frame rate that your production will be by setting the frame rate in Preferences->Rendering:

Just to reiterate, try to always use sync when you are in live production and you send video to your audience on that output device.

Warmest regards,

Hello Guys,

One bit of information on something I just discovered. The same latency issue happened to us after the update. I found that I had an extra input mapped on the decklink card. It was not even connected to anything physically. After removing the mapping, my CPU load went from 50% to 25%. It may be worth checking.

Best,

Ulf

Hi everyone, 

New aximmetry 2023.2.0 , SportStudio commun template ( out of the box , no change into it)
RTX A5000
Blackmagic 8K Pro 
PTZ AWUE 150 
All the setup is genlock @50.

Aximmetry Processor Load : report 98% load on GPU
Windows report : 43% on GPU 
Why is there a difference between the GPU load report ?

in Video Outputs panel : sync radio button on or off :  same GPU report.

What is the real GPU load ? 

Thanks.

@ MatMatim: The GPU load report in the Taskmanager is notoriously inaccurate. You can trust the Aximmetry GPU load meter to accurately show the actual usage (although I haven’t tested this in the latest update). The most important indicator is of course if you start dropping frames. Btw, 99% seems very high for this scene. Are you dropping frames? 

@TwentyStudios: thanks for the reply. 

Where can i see the dropping frame report ? In the console ?

Should i keep watching the FPS bar on the proccessor load to see the frame drop  ? It is not fix @50. Sometime it goes very quickly on 49,7 or 49,3 and come back to 50.

In preference : If i swith the rendering to 25p : i get 32% for the GPU load in aximmetry and 18% in windows.

but i need 50p.  

Best regards

Hi Ulf,

That sounds strange. We will need a bit more information to locate the source of this issue.
Does it happen again if you map the input again in the capture card and you use an unedited scene, for example the [Studio]:News Room\News Room - VirtualCam_3-Cam.xcomp scene? If so, can you tell us the Decklink card's model?

Warmest regards,

Hi MatMatim,

It could be that Windows power or performance settings are limiting your GPU when it is on high load.

Otherwise, do you experience the same behavior if you change the Frame rate in Edit->Preferences?:

And you have the sync radio button off with both frame rates.

Warmest regards,

Hi Eifert, Hi TwentyStudios,
Thank you for the reply.

@Eifert : changing the framerate in the preference from 50p to 25p , divide the problem by half.

Check the first image  :  GPU load is report 97% in Aximmetry and 4% in windows.
Are you sure that windows is not telling the truth.
Because , I got no input and no output in this project , no sync radio button, the Unreal Project in the default Aximmetry blank project.

the second image : GPU and CPU becomes crazy in Aximmetry , FPS drop to 19
and after 10 mn GPU and CPU at 700% , still 9% and 3% in windows.

with the same computer Aximmetry 2023.1.0 don't report these crazy GPU and CPU values.

I do not change anything ( no new program installation : except Aximmetry 2023.2 and Unreal 5.2 for aximmetry ) , 
no hardware change. 
What's wrong with my computer and this 2023.2 version ?

@Eifert:  Please can you help me because i plan to buy Aximmetry DE brocast and i only have 15 days left to test the software.
Thank you.

Guys, i have tried everything and in the end nothing works, so my fix was uninstall Aximmetry 2023.2.0 and install again the previous version and BIG SURPRISE everything works well again, so in simple words, something in the update DOESN'T WORK AT ALL.

I really hope that the Aximmetry Team you can fix this error.

MatMatim dont waste your time and comeback to the stable version of Axim.

Hi Eifert,

Just tested with unedited studio compound. Same problem.

Decklink 8K

BMD 12.5.1 driver

When adding mapped input but no signal. Processorload doubles. Remove mapping problem gone. As a matter of fact the 2.0 version of Aximmetry uses less resources!

Best

Ulf

I want to add that I still can't use SYNC anymore without the GPU skyrocketing even with all the settings that were recommended in this post.  60fps at 1080p output kills my RTX 3090.   Turning off SYNC lets me do 30fps and helps, but I still feel that there is something wrong with this update as far as Blackmagic cards are concerned.   The previous version didn't do this and this post has multiple complaints from other Blackmagic capture card owners.   If I am supposed to use SYNC output for production, I simply can't without causing the latency errors on the Blackmagic card.

@LiveGuyUSA: When you enable Sync for an output, Aximmetry will render at the frame rate that output is set to. Previously it only did this for capture card outputs, but in the latest version it also does this for GPU outputs. If you want to render in 30fps, you also need to set the monitor refresh rate of that output to 30fps. If your camera captures at 29.97 fps, you need to set the monitor refresh rate to 29.97, or you’ll have issues with the inputs. If you disable Sync, on the output, you can again set the project frame rate independently of the GPU output frame rate. This is how Aximmetry behaved before the update, so you’d just be going back to how it worked previously. Rendering at 60 fps requires twice the performance of doing it at 30 fps, so if your projects previously ran at 30 fps it’s not surprising that the GPU load is much higher if it has to render twice the amount of frames per second. 

Hello,

Complete fresh install of both BMD driver and Aximmetry. No longer having the problem. This version of Aximmetry is performing way better on our system. UHD 10bit SDI and only using 25% of the CPU. RTX3090 is running at a cool 60%. Framerate is 59.94. We are implementing an ultimatte 4K. Seems to bring the GPU down to 50%  Really happy!

Thanks,

Ulf

@TwentyStudios.   

I am using the Blackmagic Decklink Quad HDMI capture card which is recommended and supported by Aximmetry, but does not have any output capability, so I am forced to use the GPU output for program.  Do you recommend an additional output card?   I use HDMI  (Not SDI).  

@LiveGuyUSA: If you just do what I suggested you can keep using the GPU, just like before. If you have a spare PCIE slot you could get the Decklink Mini Monitor 4K. That has HDMI out and is very inexpensive. 

Hi,

There is indeed an issue with some of the Black Magic (BMD) cards in version 2023.2.0 of Aximmetry.

If you have a Black Magic card and you are going to use it for production in the next weeks. Then we strongly recommend using Aximmetry 2023.1.0 while Aximmetry is working on a fix. And don't use version 2023.2.0 for any production.
If you don't have production in the next weeks and want to develop using version 2023.2.0, then make sure you have at least the latest Black Magic driver installed, version 12.5.1. or higher.

Thank you very much for your assistance, the reporting of the issue, and your patience.

Warmest regards,

@Eifert: Does this issue apply to the Decklink 8K Pro as well? 

@TwentyStudios: We use Decklink 8K Pro and Bluefish Kronos K8 with no issues. As I said in an earlier post. Performance is better in this version of Aximmetry.

@Malibuswede: That’s good to hear, since we use the Decklink 8K Pro and also love performance ;-)

Jag måste passa på att fråga om du också är svensk? :-)

This might be a long shot, but for those having issues, I just saw a post about someone having issues with Decklink in standard Unreal and they solved it by updating their Intel Ethernet drivers. Don’t know if it’s related to this, but it’s a good example of how seemingly unrelated things can interact and cause issues. 

@TwentyStudios: Vilken gissning. Visst är jag Svensk! Lever och arbetar sedan många år i USA. Här är min privata email adress. Jag ser att du arbetar på Twenty Studios i Stockholm. Ni har mycket bra rykte i branschen!

malibuswede@gmail.com

Ulf Soderqvist
Director/Director of Photography
imdb.me/ulf

@Gaston Alvarez:  we've fixed the flood of "frames missed" messages in the log. The fix is available as a new version 2023.2.1.
The "Cannot keep latency" might or might not be related to this bug. Nevertheless please test with the new fix, and let us know the result.

@MatMatim: the above issue also could cause performance problems.
Please try the new fix.
If it does not solve your issue, please try turning on the Low Latency Mode on your Unreal module:

Hi Aximmetry team, 
test Aximmetry 2023.2.1
my setup : Desktop ( RTX A5000 (536.25), Proc AMD Ryzen 5950x, Decklink 8K pro (12.5.1)
project : News room - TrackedCam_3-cam.xcomp ( not edited ).

1) 
Composer : Processor load windows :  GPU at 99% at project startup , open video output windows toggle off and then on the sync on decklink 8K : GPU at 41 %.
After that i try to open video output windows and the composer freeze then windows tell me to shut down the app. ( i try 20 times , it always crash the composer). 

2)
Multi machine setup : all 3desktop PCs are the same . ( RTX A5000 (536.25), Proc AMD Ryzen 5950x, Decklink 8K pro (12.5.1)
1 desktop PC File server.
Composer GPU report 98% load at project start's up.
Composer crash if you try to open video output windows.
I got a dump file 6.7Go to send to you.

Best Regards.

Mat.

Hi Mat,

You could check if you have a performance issue with the default settings.
You can do so, by selecting Manage Settings on the startup window. Then click on backup, and save your settings. And then click Reset.

After checking the performance you can restore your settings by using the Restore option in the Manage Settings.

Do you experience the same performance issue on the other two PCs?

If you have the output genlocked, the A5000 or the Decklink 8k Pro, then can you try it without genlock and see if it fixes the performance issue?

Warmest regards,

Hi Eifert, 

i reset to default settings
I try multi machine setup with 3 machines ( news room tracked cam compound ) 

the first problem with GPU load came when i enable the channel Matrix : unified multi machine setup radio button.
GPU load goes to 99% on renderer 2.

@Eifert : 

A) it seems that GPU load at 99% is related to genlock on quadro sync because : no genlock @50 plugged -> no gpu overload. 

B) Is decklink VBlank overslep in log tell you something ? Maybe it is nothing.

C) composer freeze and windows shows popup to close the program if : 

1- try to set gpu output to #1 with the video ouput windows within the composer. 

2- the calibration file is locate on the file server witch is a separate PC.

D) Another thing, on a multi machine setup  with 3 machines: only the second rendered machine  is GPU overload at 99%. If i kill the overload renderer then i got 2 machines and the new second renderer become GPU overload.

i hope this could help you.

best regards

Mat.

Hi Mat,

Quadro sync is not yet supported. But it is being worked on and it is likely that it will be implemented in a future release. Until that, you are probably better off not syncing the video cards when using Aximmetry.

Warmest regards,

Hi,

Hi,

Aximmetry Version 2023.2.2 has been released today. The new version contains fixes for various monitor output issues and performance load.

You can find the change log here: https://aximmetry.com/learn/software-version-history/#%E2%96%BA-latest-2023-2-2

Hello, I just updated to 2023.2.2 and am running into the same issues as above. Previously I had 2023.2.0 and everything was working fine. Now keep receiving the cannot keep latency message, and continues to drop frames from inputs (all 4). Have there been any fixes to this?

Hi HoopsMD,

If you are getting "Cannot keep latency" errors, then you should increase the In-to-out Latency setting in Aximmetry. You can read more about it here.

Warmest regards,