| CARVIEW |
Select Language
HTTP/2 301
server: GitHub.com
content-type: text/html
location: https://crockwell.github.io/pixelsynth/
access-control-allow-origin: *
expires: Mon, 29 Dec 2025 01:19:30 GMT
cache-control: max-age=600
x-proxy-cache: MISS
x-github-request-id: ABA3:21D6A4:8335D5:9344A2:6951D4C9
accept-ranges: bytes
age: 0
date: Mon, 29 Dec 2025 01:09:30 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210027-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1766970570.492921,VS0,VE198
vary: Accept-Encoding
x-fastly-request-id: c46e3b77e4a88d031d839f487b5956ce208a9460
content-length: 162
HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Mon, 22 Sep 2025 01:08:48 GMT
access-control-allow-origin: *
etag: W/"68d0a1a0-4143"
expires: Mon, 29 Dec 2025 01:19:30 GMT
cache-control: max-age=600
content-encoding: gzip
x-proxy-cache: MISS
x-github-request-id: 5117:3ABDEF:80FBB0:910BBE:6951D4CA
accept-ranges: bytes
age: 0
date: Mon, 29 Dec 2025 01:09:30 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210027-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1766970571.704903,VS0,VE207
vary: Accept-Encoding
x-fastly-request-id: 4178575f92a6ad5d00f5c5dde334630974d6abda
content-length: 4799
PixelSynth: Generating a 3D-Consistent Experience from a Single Image
PixelSynth: Generating a 3D-Consistent Experience from a Single Image
Recent advancements in differentiable rendering and 3D reasoning have driven exciting results in novel view synthesis from a single image. Despite realistic results, methods are limited to relatively small view change. In order to synthesize immersive scenes, models must also be able to extrapolate. We present an approach that fuses 3D reasoning with autoregressive modeling to outpaint large view changes in a 3D-consistent manner, enabling scene synthesis. We demonstrate considerable improvement in single image large-angle view synthesis results compared to a variety of methods and possible variants across simulated and real datasets. In addition, we show increased 3D consistency compared to alternative accumulation methods.
We display rendered scenes from PixelSynth and baselines on RealEstate10K and Matterport.
There has been a variety of exciting recent and concurrent work on single-image novel view synthesis. In addition to SynSin, here is a partial list:
Thanks to Angel Chang, Angela Dai, Richard Tucker and Noah Snavely for allowing us to share frames from their datasets. Thanks Olivia Wiles and Ajay Jain for polished model repositories which were so helpful in this work. Thanks to Shengyi Qian, Karan Desai, Mohamed El Banani, Linyi Jin, and Richard Higgins for the helpful discussions. Special thanks to the Michigan Help Desk (DCO) for after-hours help with machines. The webpage template originally came from some colorful folks.
|
|
|
|
|
|
|
|
|
|
|
|
Interactive Demo
Input
PixelSynth
SynSin (6x)
No 3D Accum.
Demo Instructions.
View in widescreen or zoom out until all images fit on one line.
Look around by clicking and dragging on an image above or using [W,A,S,D] keys.
Use the buttons above or the shift key to toggle between translation and rotation.
Click "Granular Scene Movement" to load more images for smoother movement.
This probably won't work on Internet Explorer or on mobile.
View in widescreen or zoom out until all images fit on one line.
Look around by clicking and dragging on an image above or using [W,A,S,D] keys.
Use the buttons above or the shift key to toggle between translation and rotation.
Click "Granular Scene Movement" to load more images for smoother movement.
This probably won't work on Internet Explorer or on mobile.
Abstract
Recent advancements in differentiable rendering and 3D reasoning have driven exciting results in novel view synthesis from a single image. Despite realistic results, methods are limited to relatively small view change. In order to synthesize immersive scenes, models must also be able to extrapolate. We present an approach that fuses 3D reasoning with autoregressive modeling to outpaint large view changes in a 3D-consistent manner, enabling scene synthesis. We demonstrate considerable improvement in single image large-angle view synthesis results compared to a variety of methods and possible variants across simulated and real datasets. In addition, we show increased 3D consistency compared to alternative accumulation methods.
Paper and Supplemental Material
Rockwell, Fouhey and Johnson.
PixelSynth: Generating a 3D-Consistent Experience from a Single Image.
In ICCV 2021. (Hosted on arXiv)
PixelSynth: Generating a 3D-Consistent Experience from a Single Image.
In ICCV 2021. (Hosted on arXiv)
Video Results
We display rendered scenes from PixelSynth and baselines on RealEstate10K and Matterport.
Recent & Concurrent Work
There has been a variety of exciting recent and concurrent work on single-image novel view synthesis. In addition to SynSin, here is a partial list:
- Jing Yu Koh, Honglak Lee, Yinfei Yang, Jason Baldridge and Peter Anderson. Pathdreamer: A World Model for Indoor Navigation [PDF]
- Robin Rombach*, Patrick Esser* and Bjorn Ommer. Geometry-Free View Synthesis: Transformers and no 3D Priors [PDF]
- Ronghang Hu, Nikhila Ravi, Alex Berg and Deepak Pathak. Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image [PDF]
- Andrew Liu*, Richard Tucker*, Varun Jampani, Ameesh Makadia, Noah Snavely and Angjoo Kanazawa. Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image [PDF]
- Meng-Li Shih, Shih-Yang Su, Johannes Kopf and Jia-Bin Huang. 3D Photography using Context-aware Layered Depth Inpainting [PDF]
- Richard Tucker and Noah Snavely. Single-View View Synthesis with Multiplane Images [PDF]
Acknowledgements
Thanks to Angel Chang, Angela Dai, Richard Tucker and Noah Snavely for allowing us to share frames from their datasets. Thanks Olivia Wiles and Ajay Jain for polished model repositories which were so helpful in this work. Thanks to Shengyi Qian, Karan Desai, Mohamed El Banani, Linyi Jin, and Richard Higgins for the helpful discussions. Special thanks to the Michigan Help Desk (DCO) for after-hours help with machines. The webpage template originally came from some colorful folks.