Beginning with a concise text prompt, we employ a diffusion model to generate a 360$^\circ$ panoramic image. A self-refinement process is employed to produce the optimal 2D candidate panorama. Subsequently, a 3D geometric field is utilized to initialize the Panoramic 3D Gaussians. Throughout this process, both semantic and geometric correspondences are employed as guiding principles for the optimization of the Gaussians, aiming to address and fill the gaps resulting from the single-view input.

Unlike previous works that generate 3D scenes through a time-consuming score distillation process or progressive inpainting, our work uses Panoramic Gaussian Splatting to achieve ultra-fast “one-click” 3D scene generation. We integrate GPT-4V to facilitate iterative user input refinement and quality evaluation during the generation process, a feature that was challenging to implement in previous methods due to the absence of global 2D image representations.

In each row, from left to right, displays novel views as the camera undergoes clockwise rotation in yaw, accompanied by slight random rotations in pitch and random translations. LucidDreamer hallucinates novel views from a conditioned image (indicated by a red bounding box) but lacks global semantic, stylized, and geometric consistency. In contract, our method provides complete 360$^\circ$ coverage without any blind spots (black areas in baseline results), and show globally consistent semantics.