Vesuvius Grandprize Winning Solution

The Repository contains the First Place Vesuvius Grand Prize solution. This repository is part of the First Place Grand Prize Submission to the Vesuvius Challenge 2023 from Youssef Nader, Luke Farritor and Julian Schilliger.

Automatic Segmentation

Check out the automatic segmentation pipeline ThaumatoAnakalyptor of our winning Grand Prize submission by Julian Schilliger. ThaumatoAnakalyptor performs in full 3D and is also capable of segmenting in very mushy and twisted scroll regions.

Ink Detection Overview :

Our final canonical model was a timesformer small architecture with divided space-time attention. The dataset underwent expansion and cleaning rounds to increase accuracy of the labels and become as accurate as possible, approximately 15 rounds were performed between the first letters and final solution. Our solution also consisted of 2 other architectures, Resnet3D-101 with pretrained weights, I3D with non-local block and maxpooling.

Our implementation uses torch, torch-lightning,the timesformer-pytorch and 3D-ResNets-PyTorch.

🚀 Get Started

EASY: build the docker image:

docker build -t youssef_gp .
docker run --gpus all --shm-size=150g -it -v </your-path-to-train-scrolls>:/workspace/train_scrolls youssef_gp

Then to train:

python train_timesformer_og.py

Or to run inference with the already trained model:

python inference_timesformer.py --model_path timesformer_weights.ckpt --segment_path train_scrolls --segment_id 20231005123336

Important note: to install the ink labels and training data inside the docker image, run:

#to download the segments from the server
./download.sh
#propagates the inklabels into the respective segment folders for training
python prepare.py

You can find the weights of the canonical timesformer uploaded here

Example Inference

To run inference of timesformer:

python inference_timesformer.py --segment_id 20231210121321 20231221180251 --segment_path $(pwd)/train_scrolls --model_path timesformer_weights.ckpt

The optional parameter --out_path can be used to specify the output path of the predictions.