pard68 parent
This seems neat but I don't understand the use of 4D. It's not four dimensional. It's 3D with the ability to have an arbitrary perspective.
If I’m understanding the paper correctly then the four dimensions are the position, density, radius, and color of the spheres in their volumetric model. So for any given viewing position and point in time, their model produces a 4D scene that is then rasterized to 2D.
When it comes to volumetric clouds, the D stands for how many dimensions you store the points in.
A 2d volume (similar to an image) has pixels stored in 2d coordinates
A 3d volume has points stored in 3d coordinates. Imagine an image for every vertical slice of a brain scan.
A 4d volume has points stored in 4d coordinates, where the newest dimension is time. Imagine a 3d volumetric capture for each frame in time.
It's volumetric video
So's a video game, and we call that "real-time 3D". Time is mentioned, but it isn't counted again as a dimension, perhaps because any given momentary view is a time slice, not a time range like it is an XYZ range.
I think the difference is that in a video game you are in one location only at any given moment and things travel only forward in time. We can view from any location at any time in volumetric video.
In a lot of racing simulators you can change the position of the "virtual camera". It can be in the cockpit, on the hood, behind the car and on some games in an arbitrary position. Usually replays allows you to see from other competitors and where TV cameras would seat in real world.
Some games absolutely let you do that. It's not a limitation of the medium.
I'm curious what are a few examples? I've missed the last decade of new games largely.
Unless it’s a cyberpunk 2077 brain dance in the editor
3D + video (motion)
It's not a 3D model that is animated using a skeleton and keyframes like traditional 3D. It's many consecutive 3D models that create the illusion of continuous motion (aka video).
4D is the name that has come to describe the jump from static 3D models (photogrammetry) to 3D "video" models.
Time is the forth dimension. The input data is a video, so the model learns the colors and the position of the elements (basically points). You can rende the scene from any angle at any time once the models is trained
It has a time dimension.
Downvoted at the time I see it, but actually correct. It's based on K-planes https://arxiv.org/pdf/2301.10241.pdf which effectively splits each space-time relationship off from the spatial relationship. It's just mathematics, guys. The original NeRF paper talked about a 5D coordinate. You know like a k-dimensional vector?
Yea it's probably to have a catchy name and get some attention. Although it's technically accurate to call it 4D since it includes time, I think 3D video recording would probably get the point across to more people in a less sensationalist way.
Is it technically accurate? Seems like its actually 6Dof view angles + time. The paper mentions 4D view, 4D point cloud, dynamic 3D scene and 4D feature grid.