Skip to content

Latest commit

 

History

History
83 lines (61 loc) · 4.37 KB

README.md

File metadata and controls

83 lines (61 loc) · 4.37 KB

Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation

Pytorch implementation of our method for domain adaptation in semantic segmentation task.

Paper

Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation

Please cite our paper if you find it useful for your research.

@inproceedings{SEAN,
  title={Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation},
  author={Xu, Yonghao and Du, Bo and Zhang, Lefei and Zhang, Qian and Wang, Guoli and Zhang, Liangpei},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={33},
  pages={5581--5588},
  year={2019}
}

Installation

git clone https://github.com/YonghaoXu/SEANet

Dataset

Pretrained Model

Training

  • Training for GTA-5 to Cityscapes.
    • Change the default path of --data_dir_source in SEAN_GTA5.py with your GTA-5 dataset folder.
    • Change the default path of --data_dir_target in SEAN_GTA5.py with your Cityscapes dataset folder.
    • Change the default path of --restore_from in SEAN_GTA5.py with your pretrained VGG model path.
    • Refer to dataset/gta5_dataset.py and dataset/cityscapes_dataset.py for further guidance about how the images and ground-truth files are organized.
python SEAN_GTA5.py
  • Training for Synthia to Cityscapes.
    • Change the default path of --data_dir_source in SEAN_Synthia.py with your Synthia dataset folder.
    • Change the default path of --data_dir_target in SEAN_Synthia.py with your Cityscapes dataset folder.
    • Change the default path of --restore_from in SEAN_Synthia.py with your pretrained VGG model path.
    • Refer to dataset/synthia_dataset.py and dataset/cityscapes16_dataset.py for further guidance about how the images and ground-truth files are organized.
python SEAN_Synthia.py

Evaluation

  • Test for GTA-5 to Cityscapes.
    • Change the default path of --data_dir in evaluation.py with your Cityscapes dataset folder.
    • Change the default path of --restore_from in evaluation.py with your trained model path. You can also download our GTA-5 to Cityscapes model for a look.
python evaluation.py
  • Test for Synthia to Cityscapes.
    • For evaluation on Synthia to Cityscapes case, please replace the test_mIoU function in evaluation.py with the test_mIoU16 function. Since there are only 16 categories in common in this case, the code for writing the segmentation maps parts needs to be further modified. If you want to share your implementation for this issue, please pull a request.

Empirical Observations

  • Following the previous research setting in this task, we check the mIoU value on the target domain after every 500 iterations. A lower frequency for the checking would accelerate the network training, but may also miss the best performance.
  • A large --attention_threshold would be detrimental to the performance of the framework. Empirically, 0 to 0.3 is a suitable range for this parameter.
  • Best performance is usually obtained within 6 epochs. For the GTA-5 to Cityscapes case, the mIoU can reach about 34% to 35%. For the Synthia to Cityscapes case, the mIoU can reach about 36% to 37%.

Multi-GPU Training

  • This repo is tested with a batch size of 1 using a single GPU. For a larger batch size with multi-GPU training, the code may need to be modified. If you want to share your implementation for this issue, please pull a request.