Sequencer Help with Unreal and Studio DE

 

Greetings All,

Looking for some pointers to either a reference project or other self-learning material for how to use a sequencer to "script" post-production Axy camera moves with an Unreal project in Studio DE.

We are using Studio DE with Unreal in a post-production workflow to assemble our greenscreen footage after the fact in Aximmetry. With the help of the video tutorials, I have also figured out how to use the sequencer and I have figured out how to use the video recorder and Record_3-Audio nodes to allow for rendered productions. However, I am stuck with trying to figure out how to use the sequencer to fully script all the camera moves for a given series of shots AND get them to both play back in the sequencer and render. 


The closest I've been able to get is with a sequence and camera mover between the VirtualCam_Unreal_3-Cam_A-B-C xcomp and my Unreal xcomp. It'll play just fine in the sequencer but won't play or render when I use the video record node. I've also tried substituting my sequence and camera mover for the Camera Sequencer xcomp with no luck. 

Any assistance or just pointers to a good example of what I'm trying to do would be enormously appreciated!



   Gadget

 
Profile Image
Gadget
  -  

Never mind. I think I figured this part out. Doesn't completely work like the documentation, but close. Also can't seem to find any reference to camera position.

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

We have documentation on the Camera Sequence compound here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/green-screen-production/virtual-camera-workflow/camera-sequencer/
And information about the Sequencer here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/scripting-in-aximmetry/sequencing/sequencer-and-sequence-editor/

Regarding the reference to camera position, I am not entirely sure what you mean. A picture of your Flow Editor logic would greatly help in understanding what you are trying to achieve.

We are continuously updating our documentation. If you find anything misleading, feel free to pinpoint the issues, and we will work on improving it.

Warmest regards,


 
Profile Image
Gadget
  -  

With respect to the reference of a camera position, I was hoping for visual feedback about where the camera transform was at any time. When setting camera moves from the sequencer, you can set a key and it will show you the location of that key, individually, for each of the X,Y, and Z axes. You advance down the timeline move the camera with the mouse and set another key. Now what if you want to see the camera transforms between those keys?? Maybe to more precisely position the camera? Or some other object?

Sequencer Help with Unreal and Studio DE

So, I thought by adding a transform node to the camera transform pin of the virtual camera, I would be able to see the transforms. And I can, with the peeker, but that doesn't give me the ability to copy the transforms for pasting:

Sequencer Help with Unreal and Studio DE

With the Transform node, I would think that the camera transform values would show up in the table on the right in the Pin Inspector where I could copy them. I tried all different kinds of combinations of nodes, including the different kinds of nodes and placement but nothing would show me the camera transforms in the pin inspector.

As far as the documentation is concerned, this page (https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/green-screen-production/virtual-camera-workflow/camera-sequencer/) shows the ability to set keys in the CAMERA sequencer of the Camera Sequence node. Maybe I was doing something wrong, but I could not set keys in there at all. I ended up adding a new sequencer and a new camera mover, and only then had any success with setting keys. 


Sequencer Help with Unreal and Studio DE

Hopefully I'm just doing stuff wrong and you can set me straight. 

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

The Transform module has different functionality compared to what you're attempting to use it for. Instead, the Transformation Decompose module is more appropriate if you need to deconstruct the transformation into position, rotation, and scale components.


To easily write out the transformation values, you should use the Transformation Peeker:
Sequencer Help with Unreal and Studio DE
This module functions similarly to peeking in the Flow Editor, and there is a module available for each data type. You can use any of the video mixing modules to write these values over an output, for example, use a Blender or Placer module.
If you prefer to write out something more customized, you can use the Text Image module with the Vector To Text and Transformation Decompose modules.

If you want to copy a transformation, most pin data types offer a Capture Input option when you right-click on a pin that is connected to something:
Sequencer Help with Unreal and Studio DE

By using the Capture Input, the current value of the transformation will be written to the pin, which can be edited or copied once you disconnect the pin:
Sequencer Help with Unreal and Studio DE

Additionally, there is a Set Transformation Pin module that can automatically write transformations into a pin of a module.

It can be sometimes challenging to find these modules. We are working on providing more detailed documentation both online and within the software.


Alternatively, you can use an object in Unreal Engine and position it using a Get Aximmetry Transformation blueprint node. This allows you to see the object move according to your camera when using the free camera mode:
Sequencer Help with Unreal and Studio DE



Regarding the Camera Sequencer compound, you likely didn't have the first camera selected in its panel after adding it to the Control Board:
Sequencer Help with Unreal and Studio DE

Also, the camera has to be stopped to record the keys:
Sequencer Help with Unreal and Studio DE

Sadly, this wasn't part of the Camera Sequence's online documentation.

Warmest regards,

 
Profile Image
Gadget
  -  

Hi,

Thanks for all the detailed info. I was able to get the transformation peeker to work as you described. This at least gives me real-time position info. While I can also see the value of the Capture Input option of the Transformation Copy, and the Transformation Decompose with the Vector to Text/Text Image nodes, unfortunately they only capture the input at exactly the time you right-click, and having to disconnect the pin to copy the values makes it less practical. I was hoping for something more dynamic and interactive. I may have to use a combination of your suggestion of using a Blender or Placer module with the Capture Input on the Transformation Copy once I get to a place I like. Clunky, but it will work.

I think I may also try the alternative you suggested with the Unreal object set to follow the camera. Not sure if I will have the same trouble collecting transform data of that as well, but will try. 

Regarding the Camera Sequencer, I'm able to set start and finish keyframes with no problem, but when I attempt to play it back, the camera doesn't move. I tried numerous combinations of Camera 1 per your instructions and the SEQ button. No joy. It only works with the newly added Sequencer, but that's fine. I probably shouldn't be using the Camera Sequencer comp since I've only got one camera in this project anyway. :)

Thanks again for your help!!

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

You can use the Transformation Decompose with the Vector to Text/Text Image modules as a more complex solution than Transformation Peeker. This is not meant to be used with the Capture Input feature; for that any transformation pin is sufficient.
For instance, you could display the position over the preview output with these modules:

Within the Text Concat module, you could add some text before the position coordinates. Additionally, the Text Format module can be used to stylize your text.


Regarding how to set an Unreal object to follow the camera, it's quite straightforward to expose the transformation of an Unreal object to Aximmetry. You can follow the examples provided here: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/obtaining-graphics-and-virtual-assets/creating-content/creating-content-for-aximmetry-de/additional-control-with-blueprints/#get-aximmetry-transformation
Once the transformation appears on the Unreal module in Aximmetry, you can directly connect the Camera Mover to it.


About the camera paths in the Camera Sequencer, once you've selected the SEQ button, everything will be controlled by the Camera Sequencer's panel:

The above numbers of 1, 2, 3... correspond to CAM 1, CAM 2, CAM 3... in the Sequencer Editor.

Warmest regards,

 
Profile Image
Gadget
  -  

Hi,

I think my challenge is less about how to see the transforms and more about how to get them into a format that I can copy and paste easily. All of the Transform node suggestions you have are great, and certainly do the trick. I love the ability to even concatenate them with additional text for an overlay, but ultimately they all go out to a preview, and I was needing a way to copy the transforms. 

With the Camera Sequencer, I may have the wrong one loaded. Not sure if it makes a difference, I was struggling with is the one associated with the VirtualCam_Unreal_3-Cam_A-B-C. The panel looks more like this:

Sequencer Help with Unreal and Studio DE

Ty/D

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

You don't need to copy the transformation. You can directly send it to Unreal: https://aximmetry.com/learn/virtual-production-workflow/preparation-of-the-production-environment-phase-i/obtaining-graphics-and-virtual-assets/creating-content/creating-content-for-aximmetry-de/additional-control-with-blueprints/#get-aximmetry-transformation This will also handle the conversion to the Unreal coordinate system.

This is a Control Board of the Camera compound (VirtualCam_Unreal_3-Cam_A-B-C):

What you need is the panel of the CAMERA SEQUENCER, which you can add to a Control Board by right-clicking on the node in the Flow Editor.

Then, it will appear as a panel like this on the Control Board:

Warmest regards,

;