deep-photo-styletransfer

Code and data for paper "Deep Photo Style Transfer"

Setup

This code is based on torch. It has been tested on Ubuntu 14.04 LTS.

Dependencies:

• Torch (with matio-ffi and loadcaffe)
• Matlab

CUDA backend:

• CUDA
• cudnn

Download VGG-19:

sh models/download_models.sh

Compile cuda_utils.cu (Adjust PREFIX and NVCC_PREFIX in makefile for your machine):

make clean && make

Usage

Quick start

To generate all results (in examples/) using the provided scripts, simply run

run('gen_laplacian/gen_laplacian.m')

in Matlab and then

python gen_all.py

in Python. The final output will be in examples/final_results/.

Basic usage

1. Given input and style images with semantic segmentation masks, put them in examples/ respectively. They will have the following filename form: examples/input/in<id>.png, examples/style/tar<id>.png and examples/segmentation/in<id>.png, examples/segmentation/tar<id>.png;
2. Compute the matting Laplacian matrix using gen_laplacian/gen_laplacian.m in Matlab. The output matrix will have the following filename form: gen_laplacian/Input_Laplacian_3x3_1e-7_CSR<id>.mat;
3. Run the following script to generate segmented intermediate result:

th neuralstyle_seg.lua -content_image <input> -style_image <style> -content_seg <inputMask> -style_seg <styleMask> -index <id> -serial <intermediate_folder>

4. Run the following script to generate final result:

th deepmatting_seg.lua -content_image <input> -style_image <style> -content_seg <inputMask> -style_seg <styleMask> -index <id> -init_image <intermediate_folder/out<id>_t_1000.png> -serial <final_folder> -f_radius 15 -f_edge 0.01

Note: In the main paper we generate all comparison results using automatic scene segmenation algorithm modified from DilatedNet. Manual segmentation enables more diverse tasks hence we provide the masks in examples/segmentation/.

Examples

Here are some results from our algorithm (from left to right are input, style and our output):

Acknowledgement

• Our torch implementation is based on Justin Johnson's code;
• We use Anat Levin's Matlab code to compute the matting Laplacian matrix.

Contact

Feel free to contact me if there is any question (Fujun Luan [email protected]).