HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Fri, 28 Nov 2025 02:57:04 GMT access-control-allow-origin: * strict-transport-security: max-age=31556952 etag: W/"69290f80-407f" expires: Mon, 29 Dec 2025 07:42:32 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: E3F2:2F7ECD:87950D:984FBC:69522E8F accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 07:32:32 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210075-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766993552.995401,VS0,VE211 vary: Accept-Encoding x-fastly-request-id: 9bd979ec0d3ecc07cf56012a5bf92861e8e8462e content-length: 3706

Interactive3D: Create What You Want by Interactive 3D Generation

CVPR 2024

Shaocong Dong*¹, Lihe Ding*^2,4, Zhanpeng Huang³, Zibin Wang³, Tianfan Xue†², Dan Xu†¹

¹HKUST, ²CUHK MMLab, ³SenseTime Research, ⁴Shanghai AI Lab

arXiv Video Code

Interactive3D🪄 achieves controllable and high-quality 3D generation.

Method

The overall architecture: Interactive3D grants users precise control over the generative process through extensive 3D interaction capabilities. Interactive3D is constructed in two cascading stages, utilizing distinct 3D representations. The first stage employs Gaussian Splatting for direct user interaction, allowing modifications and guidance of the generative direction at any intermediate step through (i) Adding and Removing components, (ii) Deformable and Rigid Dragging, (iii) Geometric Transformations, and (iv) Semantic Editing. Subsequently, the Gaussian splats are transformed into InstantNGP. We introduce a novel (v) Interactive Hash Refinement module to further add details and extract the geometry in the second stage.

Rigid Dragging

Using rigid dragging to open a T-rex's mouth.
"A tyrannosaurus rex".

Part Combination

Using part combination to assemble your gundam.
"A gundam robot holding a sword with wings".

Hash Refinement

Using hash refinement to improve the quality.
"A gundam robot holding a sword with wings".

Semantic Editing

Using semantic editing to change your gundam.
"A gundam robot holding a ice sword with wings made of ice".

Deformable Dragging

Using deformable dragging to create banana whale.
"A whale that looks like a banana".

Geometric Transformation

Using geometric transformation to open a pokeball.
"A baby flying dragon in an open pokeball".

@article{dong2024interactive3d,
  title={Interactive3D: Create What You Want by Interactive 3D Generation},
  author={Dong, Shaocong and Ding, Lihe and Huang, Zhanpeng and Wang, Zibin and Xue, Tianfan and Xu, Dan},
  journal={arXiv preprint arXiv:2404.16510},
  year={2024}
}