Authors: Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, Ying Shan

Published on: April 10, 2024

Impact Score: 7.6

Arxiv code: Arxiv:2404.07191

Summary

• What is new: InstantMesh introduces a faster, more scalable method for 3D mesh generation from single images, utilizing a novel combination of multiview diffusion and sparse-view reconstruction models.
• Why this is important: The challenge of quickly and efficiently generating high-quality 3D meshes from single images.
• What the research proposes: A new framework named InstantMesh that synergizes an off-the-shelf multiview diffusion model with a sparse-view reconstruction model, and integrates a differentiable iso-surface extraction module.
• Results: InstantMesh achieves state-of-the-art generation quality and training scalability, creating diverse 3D assets within 10 seconds and outperforming other image-to-3D baselines on public datasets.

Technical Details

Technological frameworks used: InstantMesh, incorporating multiview diffusion and LRM (sparse-view reconstruction) architecture.

Models used: Off-the-shelf multiview diffusion model, sparse-view reconstruction model based on LRM.

Data used: Public datasets for training and evaluation.

Potential Impact

3D content creation, gaming, virtual reality, film and entertainment, and CAD industries could be significantly disrupted or benefit from InstantMesh.

Want to implement this idea in a business?

We have generated a startup concept here: Meshify.