[Need advise] Improve quality of Input video with Virtual Cam from Unreal Project

Licensing Studio Operators Content Sharing Editions Composer Performer 3D Model Import 3D Scene Editing Post Effects Video Playback Virtual Camera Camera Tracking Augmented Reality Unreal

 

Hello everyone,

I'm new here and I think that everyone has this problem with Aximmetry.

I use 4k capture with 1080p60 input source with chromakey.

But the output is so poor, the color is so blended, brightness & contrast are so terrible and the talent is not detailed as input.

I know that the problem is from the connection between Aximmetry & Unreal.

So, how do you guys improve the output quality?

Please share and grow together.

Peace to all.

   caohoangphucan

 
Profile Image
TwentyStudios
11/28/2021 11:13:30 AM
  -  

Per default, the  live video is rendered inside the UE4 scene. This means it’s affected by the whole UE4 rendering pipeline, where things like fog and post processing effects like Bloom, chromatic aberration and color correction will affect the video. If you’re scene is correctly set up (custom depth stencil pass) it shouldn’t be affected by TAA and there is an Invere Tonemapper option to counter the effects of the UE4 Tonemapper, but you will often have to tweak both the scene and the incoming video to get a good quality.

Because of this there is an option in the Aximmetry Cameras panel to turn off Allow Virtuals. This will instead composite the live video inside Aximmetry, like a traditional post composite. This means the colors and image quality will be unaffected by UE4 and will get you a much sharper image, but you will have to work a bit harder with the Lightwrap and color correction to integrate the live foootage realistically in the scene.

 
Profile Image
DeataProduction
12/2/2021 9:58:36 AM
  -  

You can also get proper lighting to your actor without rendering your footage trough unreal but its a bit of a hack.

You would need to basically send unreal a masked white plane and then in Aximmetry using multiply module comp the irradiance received by the white plane over your actor.

You would also obviously lose reflections of your actor as they would appear white with this method.



 
Profile Image
TwentyStudios
12/2/2021 6:44:10 PM
  -  

@Deata: I don’t think that would work since the white plane would affect bloom in the PostProcessVolume. Better to use a black mask and add blend the footage over that. 

In any case, I would love for Aximmetry to comment on this. The video quality with AllowVirtuals on is definitely much worse than it should be. Maybe they could implement the mask+Blend as an alternative? Or dig deeper in the UE4 source to figure out a better way to render the video.

 
Profile Image
DeataProduction
12/2/2021 7:08:04 PM
  -  

@twentystudios ahh yeah I completely forgot about bloom, but couldn't you exclude the plane with a custom stencil

 
Profile Image
TwentyStudios
12/2/2021 10:54:27 PM
  -  

@DeataProduction: I think you would loose a lot of the benefits of compositing in 3D space without the billboard interaction with bloom. Theoretically, a black alpha mask hole rendered in UE4 could still receive bloom, fog and other effects in 3D space and then you would multiply the (unmultiplied) RGB billboard on top of that inside Aximmetry. The problem is that the soft edges wouldn’t align perfectly due to scaling and motion blur from the alpha mask would also be problematic, unless exactly the same motion blur was generated for the billboard in Aximmetry. 

 
Profile Image
DeataProduction
12/3/2021 2:28:59 PM
  -  

@twentystudios yeah currently it seems there is no real way to get all the benefits without drawbacks, as with a black mask you would lose all lighting information 

 
Profile Image
TwentyStudios
12/3/2021 9:31:48 PM
  -  

@DeataProduction: You actually wouldn’t loose all lighting information, since the black mask itself could be lit. When multiplied with the the clean keyed billboard in Aximmetry it would still keep some of the light interaction. Don’t know if the results would actually be useable, but might be worth a try. 

 
Profile Image
Eifert@Aximmetry
12/13/2021 5:46:19 PM
  -  

Hi all,

Some example screenshots would be nice to get into more details.
Note, if you don't want to show your own talents/actors in your screenshot, Aximmetry comes with some green screen videos located at Common\Videos\Green.

I think most of the problems mentioned here are coming from lighting.
Sadly, we don't have much documentation on Unreal lights and on real-world light set up for virtual reality.

About the Mask+Blend idea, you can achieve a similar effect by changing the shader of the billboard to use emissive more than the base color. Of course, this will have its own problems. But if you want to experiment: