Setup to use LED wall with fro | Forum

Setup to use LED wall with frostrum

Hello,

i am trying to setup the following:
machine 1:
- connected via HDMI to a single curved LED wall (there's additional LED walls, but we're not using them till main one works)
- accepts 3 cameras sending data via NDI for video, Free-D for control data

machine 2:
- sets up one of the cameras and creates NDI sent to machine 1, the other 2 cameras are creating NDI via their own web controller
- the camera feeds already work in aximmetry, so this part is not an issue

what i'm having trouble with is conceptually understanding how to setup the LED wall rendering (fill) and frostrum(s):
- LED frostrum for machine 2 (camera 1)
- LED frostrum for camera 2 and 3

what i have for now (machine 1 is not a concern for now and i think just clearing up machine 2 will clear up anything not clear for machine 1):
Machine 2: unreal project, content is cooked:
Setup to use LED wall with frostrum

The aximmetry compound (using version 2023.2.2, last ue5.2 version)
Setup to use LED wall with frostrum
i'm trying to get this:

from https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/led-wall-production/using-led-walls-and-ar-together/

along with:
Setup to use LED wall with frostrum

and

Setup to use LED wall with frostrum

from https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/led-wall-production/using-led-walls-for-virtual-production/

and i'm having trouble understanding what this "YOUR SCENE" node is as i'm unable to find it anywhere (not by manipulating the unreal engine node or anything else) - i simply cannot get this parent node to show up

Mazimer

TwentyStudios

I’m a bit confused about what exactly you’re trying to accomplish, but I’ll try my best. Sorry if I misunderstood the question.

There seems to be several questions going on at once here. First thing is that I don’t think you can render the frustum for several camera simultaneously. Only one camera can be rendered at any time, so you would have to set up Aximmetry accordingly to allow camera switching. NDI probably won’t be a good option for this since you can’t genlock it and this not guarantee that the frustum switches in perfect sync with the camera switching.

Secondly, I think you’re mixing up different concepts here. You want to do AR/Set extension with your Unreal Engine scene, right? The documentation you’re referring to is both for Unreal and for the native Aximmetry 3D engine. These are separate workflows and wouldn’t be used simultaneously. One segment of the documentation mentions how to combine AR elements rendered using the native Aximmetry 3D engine with the main environment rendered in Unreal, but I’m guessing that’s not what you want to accomplish?

I haven’t checked, but it might be that the documentation for doing AR/Set extension and rendering to the LED wall from a single machine just isn’t updated yet, since this feature was just released a couple of days ago.

I suggest going back to reading the documentation from the beginning, taking care not to mix up the workflow for the native Aximmetry engine and the Unreal workflow.

Mazimer

Yeah, i suspect the documentation isn't completely up to date and therefore a little vague, hence the confusion.. not being primarily from the field of virtual production doesn't help matters either :D

to clarify:
- the final goal is set extension, yes. 3 cameras (2 AW-UE160s and one blackmagic, not sure which one exactly), 1 curved LED wall (blackmagic), all tracked with MARS via Free-D
- 3 camera feeds (NDI), with tracking data (Free-D), we'll need HDMI/SDI for the camera feed and frostrum rendering, but Free-D is ok?
- i tried both workflows because the unreal one just wasn't producing a frostrum, im guessing i wrongly set an input or output somewhere, the unreal one is the one we want to be using

you're saying:
1) machine 1 would need logical switching in the compound based on what input/camera is desired (basically switch/if statement?)
2) NDI doesn't work for this because it can't be genlocked and there will be delays between LED rendering switching and camera switching?
3) machine 1 would be for switching between cameras and outputing the final shot
4) "your scene" node is specific to aximmetry's native engine workflow?

so my questions would be:
- is there a way to do camera switching via external program or does it need to be via aximmetry's controls?
- can a single machine (4090 TI) handle 3 frostrums? having setup this with nDisplay, it didn't have an issue, does aximmetry have some issue with such a setup? is there anything that doesn't require physical connection to the computer that would work instead of NDI or is a physical connection to the render machine the only way? i recall there being a remote renderer option for LED walls but is this possible for frostrums? it was mentioned only as a LED wall render, but not in the context of frostrums
- would it be possible to setup nDisplay (which we already know how to setup frostrums for), but just use aximmetry for AR extension or is this fully incompatible? i've tried to add nDisplay to the scene and attach the nDisplay cine cameras to the aximmetry camera but aximmetry just complained about incompatible materials in the cooked content, im wondering if this is possible or not. this would be to bridge the gap while im learning how to setup the whole thing in just aximmetry and i realize it's probably blasphemy :D
- what would be the best split of workload between the 2 machines for this setup? or perhaps better worded: if i connect the 3 cameras to one machine, the LED wall to the other, what would the aximmetry setup look like (and is this even workload distribution)? is it even possible to separate workload between 2 machines or are more required?

TwentyStudios

There are lots of different options for switching cameras via external triggers in Aximmetry. Which one is most suitable depends on your video switcher. But, if you’re planning on using set extension for all 3 cameras, external switching probably wouldn’t be practical anyway.

I’m not sure what you mean with rendering 3 frustums? Aximmetry would only render one at a time and switch between them depending on what camera is selected in the video switcher. There are technologies like GhostFrame which allows you to render several overlapping frustums simultaneously, it I don’t know how well integrated that is with Aximmetry.

I think the easiest solution here would be to have one machine with a Decklink 8K Pro capture card that receives the 3 cameras and renders the one final, switched output from Aximmetry. This machine controls everything in addition to rendering the set extension. The second machine would be the one rendering to the LED wall using the HDMI or DisplayPort output of the GPU. Depending on what camera you have selected in Aximmetry, it would render the appropriate frustum for that camera. For a larger LED volume you could add more workstations, but keep in mind that performance won’t scale linearly, since the inner frustum would still be rendered fully in each workstation (this principle applies to nDisplay as well).

All of this should be covered in the documentation. I don’t know if the latest version of Aximmetry allows both set extension and AR overlays (using Unreal) to be rendered on a single workstation.

https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/led-wall-production/using-led-walls-for-virtual-production/

Mazimer

Tyvm, it will take several days to go through all this :)

Setup to use LED wall with frostrum

Cookie consent