PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics

By Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, Chenfanfu Jiang. Computer Vision and Pattern Recognition, 2023

444 citations recorded by the source corpus.

We introduce PhysGaussian, a new method that seamlessly integrates physically grounded Newtonian dynamics within 3D Gaussians to achieve high-quality novel motion synthe-sis. Employing a custom Material Point Method (MPM), our approach enriches 3D Gaussian kernels with physically meaningful kinematic deformation and mechanical stress attributes, all evolved in line with continuum mechanics principles. A defining characteristic of our method is the seamless integration between physical simulation and visual rendering: both components utilize the same 3D Gaussian kernels as their discrete representations. This negates the necessity for triangle/tetrahedron meshing, marching cubes, “cage meshes,” or any other geometry embedding, highlighting the principle of “what you see is what you simulate (WS2).” Our method demonstrates exceptional versatility across a wide variety of materials-including elastic entities, plastic metals, non-Newtonian fluids, and granular materials-showcasing its strong capabilities in creating diverse visual content with novel viewpoints and movements. Our project page is at: https://xpandora.github.io/PhysGaussian/

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a40e254265cbe20a746e72c3fa766e370398b740

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Bo Dai
2. Linning Xu
3. Tao Lu
4. Mulin Yu

In Computer Graphics and Computer-Aided Design

1. Bo Dai
2. Linning Xu
3. Lihan Jiang
4. Tao Lu
5. Mulin Yu
6. Kerui Ren
7. Zhangkai Ni

In Computer Vision and Pattern Recognition

1. Bo Dai
2. Linning Xu
3. Lihan Jiang
4. Tao Lu
5. Mulin Yu
6. Kerui Ren
7. Zhangkai Ni
8. Luc Van Gool

Citations in the corpus

• Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis (2023, background): “Recently, there are several work on this topic that are concurrent to ours [37, 45, 59, 64, 93, 95, 97, 99, 100].”
• 2D Gaussian Splatting for Geometrically Accurate Radiance Fields (2024, unknown): “Surfels approximate the object surface locally with shape and shade attributes, and can be derived from known geometry.”
• Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians (2024, background): “However, these methods often suffer from slow training and rendering speeds due to time-consuming stochastic sampling.”
• A Survey on 3D Gaussian Splatting (2024, unknown): “These results indicate the effectiveness of introducing additional properties or structured information to model the deformation of Gaussians so as to model the scene dynamics.”
• A Survey on Vision--Language--Action Models for Embodied AI (2024, unknown): “3D Gaussians can be deformed through physics-based simulation for generative dynamics, as proposed in PhysGaussian [58], or they can serve as 3D representations for VLMs, as enabled by UniGS [59].”
• Recent advances in 3D Gaussian splatting (2024, unknown): “9, PhysGaussian [19] employs discrete particle clouds from 3D GS for physically-based dynamics and photo-realistic rendering through continuum deformation [98] of Gaussian kernels.”