Hello there.
I am sorry in advance for the strange questions, but iv been wondering how could it be possible to achieve a synchronization between a 3d element projected upon a LED screen, and a AR element which appears as it is coming out from the LED screen into the real world?
The timing and perspective of the appearence of the AR element, and the disappearence of the virtual element on the LED screen, must be perfect.
This would be very interesting since then i could have people in front of the screen and have the element pop out beside them.
Thank you in advance!
Andrei.
Hi Andrei,
If you are using Unreal then you will have to do it in a multimachine setup: https://aximmetry.com/learn/tutorials/for-studio-operators/using-led-walls-and-ar-together/ In this case, you can use OSC to send triggers between the machines. (OSC Out and OSC Method modules)
If the poping out geometry is not moving/changing when it pops out, then you can probably get away with rendering it to the Led Wall and in AR at the same time for a limited time.
If it moves then you could do some kind of animation/effect that will hide any incorrect timing.
Otherwise, you will need to set a delay and start the animations with that delay included.
If you are using a single machine, you can even measure the delay quite easily with a setup similar to this:
This will render the frame numbers on the led wall. Zoom on it with the camera. Set it in the Video Recorder to Video Format PNG and start recording. You can stop recording after about a second.
The recording should have two numbers. Try to change the Delayer's Frames until the two numbers are equal like this:
Note, if your Led Wall, the Camera, or Aximmetry runs on different frame rates, that will be quite visible from the recorded pictures, as they will show the same frames multiple times or skip frames.
Once you have found the right Frame delay, you can use that value as the delay between the animation on the Led Wall and AR.
A similar measuring setup is possible in multi-machine, but it will be more complicated.
Note, depending on your hardware, this delay might change over time.
Warmest regards,