HTTP/2 301
server: GitHub.com
content-type: text/html
location: https://taesung.me/SwappingAutoencoder/
x-github-request-id: BA51:2916CC:919142:A32F6A:6952A410
accept-ranges: bytes
age: 0
date: Mon, 29 Dec 2025 15:53:53 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210055-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1767023633.792891,VS0,VE218
vary: Accept-Encoding
x-fastly-request-id: 9ff8e40e39627ee8a5c20eb5299e92548da0e234
content-length: 162
HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Wed, 27 Oct 2021 18:20:49 GMT
access-control-allow-origin: *
etag: W/"61799881-338b"
expires: Mon, 29 Dec 2025 16:03:53 GMT
cache-control: max-age=600
content-encoding: gzip
x-proxy-cache: MISS
x-github-request-id: 2AF6:3157C7:8F6C20:A10912:6952A410
accept-ranges: bytes
age: 0
date: Mon, 29 Dec 2025 15:53:53 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210055-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1767023633.031901,VS0,VE224
vary: Accept-Encoding
x-fastly-request-id: 64e1d94a0950282f030b2f4edbd63453c6d174ef
content-length: 3682
Swapping Autoencoder for Deep Image Manipulation
Swapping Autoencoder for Deep Image Manipulation
NeurIPS 2020
|
|
1UC Berkeley
|
2Adobe Research
|
3CMU
|
|
|
Abstract
Deep generative models have become increasingly effective at producing realistic images from randomly sampled seeds, but using such models for controllable manipulation of existing images remains challenging. We propose the Swapping Autoencoder, a deep model designed specifically for image manipulation, rather than random sampling. The key idea is to encode an image with two independent components and enforce that any swapped combination maps to a realistic image. In particular, we encourage the components to represent structure and texture, by enforcing one component to encode co-occurrent patch statistics across different parts of an image. As our method is trained with an encoder, finding the latent codes for a new input image becomes trivial, rather than cumbersome. As a result, it can be used to manipulate real input images in various ways, including texture swapping, local and global editing, and latent code vector arithmetic. Experiments on multiple datasets show that our model produces better results and is substantially more efficient compared to recent generative models.
Landscape Mixer in Photoshop
Swapping Autoencoder inspired the development of the Landscape Mixer feature in Photoshop, released Oct 26th, 2021. It was introduced at the Keynote Speech of Adobe MAX 2021.
3-Minute Video Intro of the NeurIPS 2020 Paper
Paper
 |
T. Park, J.Y. Zhu, O. Wang, J. Lu,
E. Shechtman, A. A. Efros, R. Zhang.
Swapping Autoencoder for
Deep Image Manipulation.
NeurIPS, 2020.
|
Acknowledgements
We thank Nicholas Kolkin for the helpful discussion on the automated content and style evaluation. We thank Jeongo Seo and Yoseob Kim for advice on the user interface. We thank Tongzhou Wang, William (Bill) Peebles, and Yu Sun for the discussion about disentanglement. Taesung Park is supported by a Samsung Scholarship and an Adobe Research Fellowship, and much of this work was done as an Adobe Research intern. This research was supported in part by an Adobe gift.
|
