Discussion on metrics used in segmentation evaluation

[plbenveniste]

Here are useful ressources to look into:

• Our lab metrics definition: https://github.com/ivadomed/MetricsReloaded
• Panoptica: Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps
• Anima metrics: Generalized Overlap Measures for Evaluation and Validation in Medical Image Analysis
• Other repos: https://github.com/Jingnan-Jia/segmentation_metrics
• Another metric computed here: https://github.com/npnl/atlas2_grand_challenge/blob/main/isles/scoring.py#L341
• A website for chosing the right metric: https://metrics-reloaded.dkfz.de/
• A Github repo which references mutliple losses: https://github.com/JunMa11/SegLossOdyssey

[valosekj]

Thanks for opening this issue! @naga-karthik and I originally used the ANIMA package for the SCI lesion segmentation. However, after discovering that ANIMA might not handle some edge cases properly and since ANIMA is written in C++, making it difficult to adapt/extend, we switched to https://github.com/ivadomed/MetricsReloaded. A drawback of the current MetricsReloaded wrapper script is that it is relatively slow when computing multiple metrics across many subjects.