KoBART-summarization (Abstractive)

KoBART abstractive summarization with pytorch

Data

• AI-Hub
  • "문서요약 텍스트" 데이터 활용
    • Train: 325,072
    • Validation: 40,134
  • "논문요약 텍스트" 데이터 활용
    • Train: 715,652
    • Validation: 73,061

Data unzip

python preprocess.py --data_root {directory/your/zipfile/is}

• 논문요약 텍스트/*.zip -> training.tsv, validation.tsv

Requirements

• transformers
• boto3
• rouge
• konlpy
• tqdm
• wandb

pip install -r requirements.txt

• install pytorch from https://pytorch.org/

Training

python train.py --log --data_root {dir/your/data/is}

References

• KoBART
• KoBART-summarization-pytorch