for what it does, its cool but claiming it's useful for games/vr/xr is a bit of a stretch, in it's current state it's only good for fixed position vr (which is a pretty rough experience) as the illusion falls apart when you move your head slightly. For games you could use it for environment maps or maybe a non-interactive background in menus or visual novel style.
But kinda think claiming a panoramic generator as a "World Generator" is intentionally hoping people expect it to be more than it delivers.
socalgal2 · 6h ago
Neat! But not quite what I was expecting. It doesn't really make a 3D "scene". It makes a 3D panorama (building a mesh based off the depth of the panorama). Move off the center and all the things that were occluded are missing.
I guess, because we're in the time of generative content, that I was expecting it would recognize what's in the scene and fill in the missing parts. Maybe in version 2
albumen · 4h ago
From GitHub page:
"We also support background inpainting for better scene generation, but it's currently an experimental feature, which may not work for all scenes.
It can be enabled by setting WorldGen(inpaint_bg=True)"
atrus · 10h ago
We really need models that produce intermediate files. A .blend, a whole godot project, an actual krita/psd file.
Things like this product such cool things....all the way up until you need to move a specific tree half a meter to the left, and then you're either stuck in some prompt/seed hell or have to open an abomination of a file in an editor.
ziyangxie · 9h ago
WorldGen supports exporting the generated scene as a vertices colored mesh in .ply format, which you can load into any 3D editor (like Blender). The output is also in metric scale, so you can easily align it with other assets
firtoz · 3h ago
I like that there are so many different approaches to generate worlds. One, few, or eventually all of them will stick.
We have been playing around with something similar at Greybox too, where we ask Claude 3.7 to create Lua scripts to define the scenes with primitives (cubes, spheres, etc) that you can then move around. It's not perfect but did better than we expected!
We saw that Meta, Krea are working on "compose the scene from 3d model generations or imports as individual components", we'll give that a try too, soon.
doshaa · 5h ago
I really hope there is like an "AI" flair on projects like these, i love what they aim to do but man the saturation levels mixed with "enough to make it work in theory" effort makes me want to filter out these posts.
I understand its all vibes and fun, but like at least some warnings that "this thing might lag your computer" and buttons that work.
pedalpete · 4h ago
Really nicely done, and I'm excited to see where this goes.
I'm the founder of 3d real-world scene generator https://ayvri.com - the tech was acquired a while back, but I still have people reaching out to me to ask me to bring it back, and I've told people that the way to do it today would be to generate the scenes with splats and AI.
jimmySixDOF · 7h ago
An opensource Blockade Labs Skybox.ai sounds good and thats a lot of tooling in your pipeline so congratulations putting it all together I would 100% clone this on a HuggingFace space !!
pdntspa · 8h ago
The splat outputs remind me of Cyberpunk 2077's braindances
shihabkhanbd · 4h ago
Just clicked link but Idk what I'm doing from there.. As I thought it was link to an Ai Image Generator.. Help?!
ziyangxie · 8h ago
Optimized the webpage layout for mobile users.
Ono-Sendai · 8h ago
Not really proper 3D, more like a kind of 2.5D.
ivape · 9h ago
Looks like only nvidia cards. Can anyone guess how much vram you need for this?
ziyangxie · 9h ago
It is currently 26GB, but I will reduce it to under 24GB so that you can at least run it on a 4090.
But kinda think claiming a panoramic generator as a "World Generator" is intentionally hoping people expect it to be more than it delivers.
I guess, because we're in the time of generative content, that I was expecting it would recognize what's in the scene and fill in the missing parts. Maybe in version 2
Things like this product such cool things....all the way up until you need to move a specific tree half a meter to the left, and then you're either stuck in some prompt/seed hell or have to open an abomination of a file in an editor.
We have been playing around with something similar at Greybox too, where we ask Claude 3.7 to create Lua scripts to define the scenes with primitives (cubes, spheres, etc) that you can then move around. It's not perfect but did better than we expected!
https://greybox.app/blog/articles/introducing-greybox-ai-cre... has a video at the top that shows what it looks like at the moment.
We saw that Meta, Krea are working on "compose the scene from 3d model generations or imports as individual components", we'll give that a try too, soon.
I understand its all vibes and fun, but like at least some warnings that "this thing might lag your computer" and buttons that work.
I'm the founder of 3d real-world scene generator https://ayvri.com - the tech was acquired a while back, but I still have people reaching out to me to ask me to bring it back, and I've told people that the way to do it today would be to generate the scenes with splats and AI.