UE5 Temporal Super Resolution and Aximmetry

 

Hi there,

Now that Aximmetry supports UE5 which has a native upscaling tech inside, I was wondering if it would make sense and how to use it for virtual production, since the resolution of the project is controlled from Composer.

Temporal super resolution (TSR) promises quality similar to native 4K at the cost of standard HD, or HD at the cost of 720p for instance. Unlike DLSS it's also supposed to work straight out of the box on both AMD and NVIDIA cards, with no plugin headache.

Being that UE 5 is also by default more demanding on the machines than UE4 (with lumen global illumination, virtual shadows, nanite etc... unless you specifically turn them off) it sounds tempting, but my early tests aren't that conclusive, so I'm wondering if I'm doing it wrong and if there are specific caveats to be aware of (eg: with billboards or whatnot).

Has anyone got to make it work and has a good workflow?

Thanks


   ericmarodon

 
Profile Image
Eifert@Aximmetry
  -  

Hi Eric,

Note that:
In Unreal Editor for Aximmetry, the default antialiasing method is TAA because we found that TSR can cause rendering errors on the billboard if it is occluded by an opaque object.
If this is not an issue in your use case, you can switch to TSR.

https://aximmetry.com/learn/tutorials/for-aximmetry-de-users-unreal-engine-5/ue5-how-to-install-and-work-with-the-unreal-engine-based-de-edition/#setting-up-objects-in-front-of-the-billboard

Warmest regards,

 
Profile Image
ericmarodon
  -  

Thanks for your input.

I'll test the occluded billboard part to see if it's an issue, but from your experience is the following worklow the correct one:

For "Fake 4K"

-In Composer : Set composer to 4K rendering

-In Unreal : Set antialiasing to TSR and in the level blueprint add a console command r.ScreenPercentage 75 (or 50)

For "Performance" :

-In Composer : Set composer to Full HD rendering

-In Unreal : Set antialiasing to TSR and in the level blueprint add a console command r.ScreenPercentage 75


At last, in your experience, is there a quality or performance advantage to use FSR 2 plugin or DLSS 2 plugin over native TSR with Aximmetry?

Thanks

Eric

 
Profile Image
Eifert@Aximmetry
  -  

Hi Eric,

Since we have errors with TSR we don't really have any workflow with it yet.

I don't really have experience with FSR. But it has probably similar issues as DLSS, which has many, especially in relation to virtual billboards.

However, in all these cases, you might be able to fix all the billboard related problems by turning off Allow Virtuals:

This will make Aximmetry render the billboard instead of Unreal.

Warmest regards,

 
Profile Image
ericmarodon
  -  

Well, just so you know I tried it, so other people can maybe get something from it:

- Setting Aximmetry Composer output resolution to Full HD and TSR + Screen percentage below 100 (75% and below) gives terrible results with the billboard. There are jagged edges and artifacts all around the talent silhouette. It's Ok with the rest of the scene, so maybe it's still an option with tracked camera and "Allow Virtuals Off"

- Setting Aximmetry Composer output to 4K and Screen percentage to 75% or even 50% on the other hand actually seems much better at first glance (very simple test scene). So it could still be an option in certain cases.

 

 
Profile Image
Eifert@Aximmetry
  -  

Hi Eric,

Unreal Editor for Aximmetry has been upgraded to Unreal 5.1 in Aximmetry 2023.1.0: https://aximmetry.com/learn/software-version-history/#%E2%96%BA-latest-2023-1-0

TSR should work without a problem with the new version. If you still experiencing problems with it, please report them here.

Warmest regards,

 
Profile Image
buffos
  -  

I switch to TSR and add in the level blueprint an "execute console command" with "r.ScreenPercentage 50" . There is no difference in frame rate whatsoever.

When I change it in the editor, the editor's frame rate jumps significantly.

Any ideas why this is happening and how to force the screen percentage change inside Aximmetry?
I tried both cooked and live sync modes. There is no difference.


P.S. I tried "r.Editor.Viewport.OverridePIEScreenPercentage 0" and indeed, when I run the scene inside Unreal, I do see the screen percentage affecting the scene. 

 
Profile Image
buffos
  -  

I did some more research.

I used the Basic Unreal project, provided by Aximmetry.

  • I opened the Unreal project, changed project settings to enable TSR, and then
  • In the level blueprint, I created the following blueprint so I can control ScreenPercentage from Aximmetry. 
  • To avoid running the command continuously, the command is run only when the screen percentage changes inside Aximmetry.

    https://blueprintue.com/blueprint/4b95lz3o


What I noticed. (with the frame rate set to unlimited)

When the analysis is set to Full HD, lowering the percentage (even to 10%) has no effect on performance, but it does change the quality of the image.
On the other hand, when the analysis is set to 4K, low screen percentages show an increase in frame rate.

I was expecting at least some frame rate gains in FHD too, but probably my intuition is wrong and the bottleneck is not in the resolution.

P.S. (Suggestion). I think controlling screen percentage inside Aximmetry, without the hassle of blueprints, is a welcomed feature, instead of adding such a mechanism in every Unreal project that needs performance fine-tuning.


 
Profile Image
Eifert@Aximmetry
  -  

Hi Buffos,

Thanks for sharing the blueprint.

Unreal actually has a quite nice page on ScreenPercentage here: https://docs.unrealengine.com/5.1/en-US/screen-percentage-with-temporal-upscale-in-unreal-engine/

Instead of testing it on the Basic Unreal project, you should find a scene with complex geometry and materials. That is where this optimization should really have an effect. Cause in the Basic Unreal project having Lumen enabled consumes most of the resources, Lumen has a high baseline resource need.

Warmest regards,