Hi everyone,
we run a pretty standard 8x8x3 meters green box here, 1 VCAM (BMD Micro 4K) and 2 TCAM (BMD URSA Broadcast G2). Sets in Unreal, Single Broadcast DE workstation, currently on 2025.1 but will probably have to upgrade to 2025.3 for this as explained later.
The studio works great as an empty green box with just a photorealistic background room and the odd virtual object like a wall with a TV on a stand or a mobile LED wall. But now a big client wants to incorporate an actual virtual furniture piece into it for a talkshow-style production. The general idea was something like this:
The background isn't really an issue, the Unreal designers we work with are pretty good at making stuff like this look realistic. Plus it's obvious that the glass surface won't work either on the table or the floor, this will have to be some opaque material with maybe some reflective pieces in carefully selected areas. Kind of like this:
Now, the hard part is putting it into practice. We aren't Unreal Designers, and while the Unreal Designers we work with are really good at making photorealistic rooms they have little actual broadcast experience. So we did some experiments:
The keying is deliberately way off so the contours of the green table stay visible. With a clean plate it vanishes completely while objects on top of it are keyed in well, so that part works.
There are three things here where we are a bit stuck now:
First of all it seems to be really difficult to get the dimensions of the virtual "table" and the green placeholder to match up. They are designed to be the same diameter and the position can be adjusted with a blueprint transformation, but it seems impossible to get more than one edge at a time to match well enough. What are some tricks to do this better in Unreal, and how do you go about hiding the inevitable imperfections?
Secondly Billboard placement. It seems like with a round table the best way of doing this is having the billboard effectively halve the table so at least from the standard +-45 degree camera angles onto talent sitting or standing in a half circle around the table the table is always in front of any visible body parts. On the tracked cameras this will obviously require using a billboard with a horizontal section, hence the upgrade to 2025.03.
But what about the VCAM? There is no horizontal section on the Billboard there (yet...).
Third, what rendering settings should we use in Aximmetry to make this look realistic? Currently we do not allow virtuals (Keying in Aximmetry only) and also do not render shadows of real objects as it looks better if we just go with the real shadows on the green box floor.
Any Ideas? Are we missing something completely here?
Hi Stefan,
1.
Achieving perfect alignment is impossible, but you can get a closer match than what is shown in your video. (From watching your video, it looks like something in your virtual or real-world setup might be slightly rotated or oval-shaped, though I could be mistaken.)
What you are trying to do is very difficult to execute well.
See their final result here: https://my.aximmetry.com/post/2163-how-to-calibrate-the-floor-of-real-scene
A useful tip from that example is that they used a printed AruCo marker, with the Marker Detector module in the Flow Editor, to precisely position the table.
The main challenge is that the virtual desk may appear larger or smaller than the real one (the green-screened table in the studio). If it’s larger, part of your talent’s torso will overlap the virtual table. If it’s smaller, there will be a visible gap between the talent’s torso and the virtual table. This misalignment is why many productions avoid showing the tabletop in the shot.
One workaround in Unreal Engine is to make the table edge nearest the talent slightly blurred. You can achieve this by adding a material that transitions from fully opaque to fully transparent, making any gap less noticeable.
Alternatively, you could avoid having the desk be entirely virtual. For example, placing a table pad or topper on the real green table near the talent will make the edge no longer an issue.
2.
If the talent will be moving around (i.e., in front of or behind the table), you will need to use talent tracking with a tracking device to properly position the billboard. More information can be found here: https://aximmetry.com/learn/virtual-production-workflow/green-screen-production/tracked-camera-workflow/tracked-camera-billboards-placement/#billboard-tracking-auto-position. Optical tracking may not work in this scenario, as the green table will likely obstruct the talent abruptly.
If the talent will remain behind the table and be hidden by it, you might be able to use the Horizontal Section part over the table.
To be honest, I had not previously considered this use case and am not certain what the specific benefits would be of Horizontal Section in this situation.
3.
If the reflection is not near the actual talent (or their hands), as seen in your screenshot, you should not have any issues once it is correctly positioned. Since you are using a 3D Cleanplate, I recommend configuring the keyer to include real-world shadows. You can read more about this here: https://aximmetry.com/learn/virtual-production-workflow/green-screen-production/keying/keying/#keying-with-shadows
Warmest regards,