Looking for a 3D gaussian Splatting plugin that works in Cooked mode

 

Hi all,

I'm looking for a 3D gaussian splatting plugin that works in cooked mode for Aximmetry.

If you don't know what that is, 3D gaussian splatting (or 3DGS) is a new scanning and rendering technique, recently published, similar to photogrammetry or NERFs, but much more efficient than Nerf, and excellent to capture environments for virtual production.

The ones I have tested so far (Luma ai, available on the marketplace for free, and Xverse, available on github) work in live mode, but when trying to open it directly in cooked mode, they both throw errors (runtime unavailable...).

Thanks a lot for your help, this would be a killer feature for Aximmetry to support any of them.

   ericmarodon

 
Profile Image
Nestruction Studios
  -  

Following this. Such crazy possibilities awaiting there! Haven´t got to experiment with these
my self yet but what I´ve seen so far has made me really curious about 3DGS.

Which would you say is better to start with, Luma AI or Xverse?

Hope there is a solution to get it to work in cooked mode!


Emil


 
Profile Image
ericmarodon
  -  

Hard to say so far. They're both free, they both work similarly and offer the same visible (good) performance from my early tests.

Luma AI is on the marketplace, so it's a bit easier to install, but Xverse included tools to create 3GDS from within unreal if you have a beefy machine. It's not so relevant if you use a platform like polycam or lumaAI to generate them though.

There are at least two other paid plugins but they're over 100 USD so I'm very wary to get one only to discover that maybe they don't work either in cooked mode.

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

We are working to ensure the efficient functioning of the Volinga solution, which is designed to handle nerf. It also has the capacity to visualize nerf using a 3D Gaussian method.

Note that any alternative plugin should work in Live Sync mode. And in cooked mode too, if they support the cooking process and their plugin is accompanied by source code.

Warmest regards,

 
Profile Image
ericmarodon
  -  

Thanks for answer. I'll look into Volinga, thanks for the tip.

All plugins I tested do work in Live Sync Mode, but I wanted to avoid having to open Unreal or have someone mess with it.

In the meantime, the xverse plugin is on github there :

GitHub - xverse-engine/XV3DGS-UEPlugin: A Unreal Engine 5 (UE5) based plugin aiming to provide real-time visulization, management, editing, and scalable hybrid rendering of Guassian Splatting model.

How can I check if source code is included? (I'm not really familiar with that).

Thanks.



 
Profile Image
Eifert@Aximmetry
  -  

Hi Eric,

I've successfully installed the XVERSE plugin and completed the cooking process, which usually means that all the necessary source code was included. However, I encountered a critical error when running it in cooked mode:
[LogPluginManager] Plugin 'XV3dGS' failed to load because module 'GSRuntime' could not be initialized successfully after it was loaded.

This suggests the plugin may not yet support content cooking. If you're interested, we can delve into more detail to understand why this error occurs. There might be a simple fix for the error.

Warmest regards,

 
Profile Image
ericmarodon
  -  

Yes, that's exactly the error message I had too. It "cooked" but then I got the "GSRuntime" error in composer.

Thanks a lot for investigating.

If this is a simple fix, that would be great, because it opens standard PLY splats (made from various sources and editable in supersplat) out of the box. You don't need to convert them to a proprietary NVOL format.

Best,

Eric

 
Profile Image
Eifert@Aximmetry
  -  

Hi Eric,

Upon further reviewing the plugin, we unfortunately discovered that it is missing the source code. It was purely by chance that the plugin installation managed to fully run without running into the missing code.
Nevertheless, in Live Sync mode you should be able to use the plugin without a problem.

Warmest regards,

 
Profile Image
ericmarodon
  -  

Hi, I've given a try to the Volinga plugin but when cooking, it's throwing me an error with the following:

UATHelper: Cooking (Windows): LogWindows: Error: appError called: Assertion failed: IsValid() [File:E:\Build2\UnrealEngine\Engine\Source\Runtime\Core\Public\Templates\SharedPointer.h] [Line: 1098]

Which is weird, because both aximmetry and regular unreal are on the C: drive on that computer.

Is that something you experienced?

Thanks

 
Profile Image
Eifert@Aximmetry
  -  

Hi Eric,

Yes, we have also experienced this issue.

Currently, the Volinga plugin only works in Live Sync mode.

Warmest regards,

 
Profile Image
ericmarodon
  -  

Hi,

Has there been any updates on that front: is there a 3DGS plugin that works in cooked mode with the 2024.1 version ?

Either Luma AI, Volinga or XV3GS (or else) ?

So far I haven't been lucky, but there could be human error as well on my part, so if someone cracked it, I'd be happy to know which one worked.

Thanks a lot.

Edit1: So it was human error after all, at least the Luma ai plugin is working in cooked mode when you follow all the steps.

Edit2: And so does the XV3GS plugin...