Skip to content

Latest commit

 

History

History
135 lines (101 loc) · 4.29 KB

trackers.md

File metadata and controls

135 lines (101 loc) · 4.29 KB
Raw

Running trackers on TAO

SORT

Here, we will reproduce a simpler variant of the SORT result presented in TAO. Specifically, we will reproduce the following row from Table 13 in our supplementary material.

NMS Thresh Det / image Det score max_age min_hits min_iou Track mAP
0.5 300 0.0005 100 1 0.1 11.3

Run detectors

  1. Download and decompress the detection model and config from here or here to $DETECTRON_MODEL.

    If you would like to re-train the detector, please see this doc.

  2. Setup and install detectron2

  3. Run the detector on TAO:

    python scripts/detectors/detectron2_infer.py \
    --gpus 0 1 2 3 \
    --root $TAO_ROOT/train \
    --output /path/to/detectron2/output/train \
    --config $DETECTRON_MODEL/config.yaml \
    --opts MODEL.WEIGHTS $DETECTRON_MODEL/model_final.pth

    On a machine with 4 2080TIs, the above took about 8 hours to run on the train set.

Run SORT

python scripts/trackers/sort/track.py \
    --detections-dir /path/to/detectron2/output/train \
    --annotations $TAO_ROOT/annotations/train.json \
    --output-dir /path/to/sort/output/train \
    --workers 8

On our machine, the above took about 11 hours to run on the train set.

Evaluate

python scripts/evaluation/evaluate.py \
    $TAO_ROOT/annotations/train.json \
    /path/to/sort/output/train/results.json

This should report an AP of 11.3.

Single-object trackers

Here we show how to run single-object trackers from the excellent PySOT tracking repository.

Setup

  1. Download and setup the PySOT repository. This code was tested with PySOT at commit 052b96. Please follow instructions from the PySOT repository for installation.

  2. Ensure pysot to your PYTHONPATH. You can check that the following import works:

    python -c 'from pysot.core.config import cfg'

Download model

Download configs and models from the PySOT model zoo.

Run tracker

  1. Run single-object tracker using the first frame of a track as the init:
python scripts/trackers/single_obj/pysot_trackers.py \
        --annotations ${TAO_ROOT}/annotations/train.json \
        --frames-dir ${TAO_ROOT}/train/ \
        --output-dir /path/to/pysot/output \
        --config-file /path/to/pysot/repo/experiments/siamrpn_r50_l234_dwxcorr/config.yaml \
        --model-path /path/to/pysot/model/siamrpn_r50_l234_dwxcorr_model.pth \
        --gpus 0 1 2 3 \
        --tasks-per-gpu 2
  1. Run tracker with "biggest" init strategy, as in Table 5 of our paper. To do this, you can add the --init biggest flag, as shown below:
python scripts/trackers/single_obj/pysot_trackers.py \
        --annotations ${TAO_ROOT}/annotations/train.json \
        --frames-dir ${TAO_ROOT}/ \
        --output-dir /path/to/pysot/output \
        --config-file /path/to/pysot/repo/experiments/siamrpn_r50_l234_dwxcorr/config.yaml \
        --model-path /path/to/pysot/model/siamrpn_r50_l234_dwxcorr_model.pth \
        --gpus 0 1 2 3 \
        --tasks-per-gpu 2 \
        --init biggest

Evaluate

python scripts/evaluation/evaluate.py \
    $TAO_ROOT/annotations/train.json \
    /path/to/pysot/output/train/results.json \
    SINGLE_OBJECT.ENABLED True \
    THRESHOLD 0.7

Note that 0.7 is the tuned threshold for the siamrpn_r50_l234_dwxcorr model. These thresholds are tuned on the training set, as described in Appendix C.2 of our paper, with results shown in Table 16. Below are the thresholds for a few PySOT models.

Model Threshold
siamrpn_r50_l234_dwxcorr 0.7
siamrpn_r50_l234_dwcorr_lt 0.9
siammask_r50_l3 0.8

The above command, for siamrpn_r50_l234_dwxcorr, should produce an AP of 31.5.