This is the TensorFlow implementation of the IEEE CVPR 2020 paper "ARShadowGAN: Shadow Generative Adversarial Network for Augmented Reality in Single Light Scenes". The overview of ARShadowGAN is shown below.

• CUDA (9.0)
• cuDNN (7.4.1)
• tensorflow-gpu (1.12.0)
• opencv-python (4.1.1.26)
• numpy (1.16.5)
• python (3.5.4)

This code has been tested under Windows 10 and Ubuntu 16.04 successfully with all the requirements.

Our Shadow-AR dataset is partially available. Download the file Shadow-AR.zip and unzip it. Supervised data samples are shown below.

Shadow-AR contains five kinds of images and corresponding images in different directories have the same name:

The dataset configurations used for ARShadowGAN training and evaluation are comming soon.

Shadow-AR dataset is free for non-commercial research. You can use it for other tasks, merge or re-split it as desired.

We provide the code and pre-trained model for readers/researchers to reproduce our experimental results.

1. Run in terminal:

git clone https://github.com/ldq9526/ARShadowGAN.git
cd ARShadowGAN/

2. Download the pre-trained model model.pb and place it in directory "model/".

3. Prepare image data containing input AR images and virtual object masks. Such as samples in directory "data/":

4. Run in terminal:

python test.py

Generated attention maps and AR images with virtual object shadows will be saved in directory "output/".

To test with custom image data, modify test.py line 13 (data_root = ....) or replace images in directories "noshadow/" and "mask/".

Images input to ARShadowGAN should be resized to 256x256.

If you use the code or Shadow-AR dataset in your own research, please cite:

@InProceedings{liu2020,  
  title = {ARShadowGAN: Shadow Generative Adversarial Network for Augmented Reality in Single Light Scenes},
  author = {Liu, Daquan and Long, Chengjiang and Zhang, Hongpan and Yu, Hanning and Dong, Xinzhi and Xiao, Chunxia},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2020}
}