Skip to content

Commit

Permalink
Bump version to v3.2.0 (#11030)
Browse files Browse the repository at this point in the history
  • Loading branch information
hhaAndroid authored Oct 12, 2023
2 parents f78af77 + dfad04c commit fe3f809
Show file tree
Hide file tree
Showing 333 changed files with 30,994 additions and 3,378 deletions.
12 changes: 9 additions & 3 deletions .circleci/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,21 +94,27 @@ jobs:
type: string
cuda:
type: enum
enum: ["11.1", "11.7"]
enum: ["11.1", "11.7", "11.8"]
cudnn:
type: integer
default: 8
machine:
image: ubuntu-2004-cuda-11.4:202110-01
image: linux-cuda-11:default
# docker_layer_caching: true
resource_class: gpu.nvidia.small
resource_class: gpu.nvidia.small.multi
steps:
- checkout
- run:
# CLoning repos in VM since Docker doesn't have access to the private key
name: Clone Repos
command: |
git clone -b main --depth 1 ssh://[email protected]/open-mmlab/mmengine.git /home/circleci/mmengine
- run:
name: Install nvidia-container-toolkit and Restart Docker
command: |
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker
- run:
name: Build Docker image
command: |
Expand Down
4 changes: 3 additions & 1 deletion .dev_scripts/benchmark_train_models.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@ atss/atss_r50_fpn_1x_coco.py
faster_rcnn/faster-rcnn_r50_fpn_1x_coco.py
mask_rcnn/mask-rcnn_r50_fpn_1x_coco.py
cascade_rcnn/cascade-mask-rcnn_r50_fpn_1x_coco.py
configs/grounding_dino/grounding_dino_swin-t_finetune_16xb2_1x_coco.py
configs/glip/glip_atss_swin-t_a_fpn_dyhead_16xb2_ms-2x_funtune_coco.py
configs/ddq/ddq-detr-4scale_r50_8xb2-12e_coco.py
panoptic_fpn/panoptic-fpn_r50_fpn_1x_coco.py
retinanet/retinanet_r50_fpn_1x_coco.py
rtmdet/rtmdet_s_8xb32-300e_coco.py
rtmdet/rtmdet-ins_s_8xb32-300e_coco.py
deformable_detr/deformable-detr_r50_16xb2-50e_coco.py
fcos/fcos_r50-caffe_fpn_gn-head-center-normbbox-centeronreg-giou_1x_coco.py
centernet/centernet-update_r50-caffe_fpn_ms-1x_coco.py
dino/dino-4scale_r50_8xb2-12e_coco.py
Expand Down
4 changes: 2 additions & 2 deletions .dev_scripts/gather_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
import yaml
from mmengine.config import Config
from mmengine.fileio import dump
from mmengine.utils import mkdir_or_exist, scandir
from mmengine.utils import digit_version, mkdir_or_exist, scandir


def ordered_yaml_dump(data, stream=None, Dumper=yaml.SafeDumper, **kwds):
Expand Down Expand Up @@ -45,7 +45,7 @@ def process_checkpoint(in_file, out_file):

# if it is necessary to remove some sensitive data in checkpoint['meta'],
# add the code here.
if torch.__version__ >= '1.6':
if digit_version(torch.__version__) >= digit_version('1.6'):
torch.save(checkpoint, out_file, _use_new_zipfile_serialization=False)
else:
torch.save(checkpoint, out_file)
Expand Down
59 changes: 52 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
[![license](https://img.shields.io/github/license/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/blob/main/LICENSE)
[![open issues](https://isitmaintained.com/badge/open/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/issues)
[![issue resolution](https://isitmaintained.com/badge/resolution/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/issues)
[![Open in OpenXLab](https://cdn-static.openxlab.org.cn/app-center/openxlab_demo.svg)](https://openxlab.org.cn/apps?search=mmdet)

[📘Documentation](https://mmdetection.readthedocs.io/en/latest/) |
[🛠️Installation](https://mmdetection.readthedocs.io/en/latest/get_started.html) |
Expand Down Expand Up @@ -102,6 +103,52 @@ Apart from MMDetection, we also released [MMEngine](https://github.com/open-mmla

### Highlight

**v3.2.0** was released in 12/10/2023:

**1. Detection Transformer SOTA Model Collection**
(1) Supported four updated and stronger SOTA Transformer models: [DDQ](configs/ddq/README.md), [CO-DETR](projects/CO-DETR/README.md), [AlignDETR](projects/AlignDETR/README.md), and [H-DINO](projects/HDINO/README.md).
(2) Based on CO-DETR, MMDet released a model with a COCO performance of 64.1 mAP.
(3) Algorithms such as DINO support `AMP/Checkpoint/FrozenBN`, which can effectively reduce memory usage.

**2. [Comprehensive Performance Comparison between CNN and Transformer](<(projects/RF100-Benchmark/README.md)>)**
RF100 consists of a dataset collection of 100 real-world datasets, including 7 domains. It can be used to assess the performance differences of Transformer models like DINO and CNN-based algorithms under different scenarios and data volumes. Users can utilize this benchmark to quickly evaluate the robustness of their algorithms in various scenarios.

<div align=center>
<img src="https://github.com/open-mmlab/mmdetection/assets/17425982/86420903-36a8-410d-9251-4304b9704f7d"/>
</div>

**3. Support for [GLIP](configs/glip/README.md) and [Grounding DINO](configs/grounding_dino/README.md) fine-tuning, the only algorithm library that supports Grounding DINO fine-tuning**
The Grounding DINO algorithm in MMDet is the only library that supports fine-tuning. Its performance is one point higher than the official version, and of course, GLIP also outperforms the official version.
We also provide a detailed process for training and evaluating Grounding DINO on custom datasets. Everyone is welcome to give it a try.

| Model | Backbone | Style | COCO mAP | Official COCO mAP |
| :----------------: | :------: | :-------: | :--------: | :---------------: |
| Grounding DINO-T | Swin-T | Zero-shot | 48.5 | 48.4 |
| Grounding DINO-T | Swin-T | Finetune | 58.1(+0.9) | 57.2 |
| Grounding DINO-B | Swin-B | Zero-shot | 56.9 | 56.7 |
| Grounding DINO-B | Swin-B | Finetune | 59.7 | |
| Grounding DINO-R50 | R50 | Scratch | 48.9(+0.8) | 48.1 |

**4. Support for the open-vocabulary detection algorithm [Detic](projects/Detic_new/README.md) and multi-dataset joint training.**
**5. Training detection models using [FSDP and DeepSpeed](<(projects/example_largemodel/README.md)>).**

| ID | AMP | GC of Backbone | GC of Encoder | FSDP | Peak Mem (GB) | Iter Time (s) |
| :-: | :-: | :------------: | :-----------: | :--: | :-----------: | :-----------: |
| 1 | | | | | 49 (A100) | 0.9 |
| 2 || | | | 39 (A100) | 1.2 |
| 3 | || | | 33 (A100) | 1.1 |
| 4 ||| | | 25 (A100) | 1.3 |
| 5 | ||| | 18 | 2.2 |
| 6 |||| | 13 | 1.6 |
| 7 | |||| 14 | 2.9 |
| 8 ||||| 8.5 | 2.4 |

**6. Support for the [V3Det](configs/v3det/README.md) dataset, a large-scale detection dataset with over 13,000 categories.**

<div align=center>
<img width=960 src="https://github.com/open-mmlab/mmdetection/assets/17425982/9c216387-02be-46e6-b0f2-b856f80f6d84"/>
</div>

We are excited to announce our latest work on real-time object recognition tasks, **RTMDet**, a family of fully convolutional single-stage detectors. RTMDet not only achieves the best parameter-accuracy trade-off on object detection from tiny to extra-large model sizes but also obtains new state-of-the-art performance on instance segmentation and rotated object detection tasks. Details can be found in the [technical report](https://arxiv.org/abs/2212.07784). Pre-trained models are [here](configs/rtmdet).

[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/rtmdet-an-empirical-study-of-designing-real/real-time-instance-segmentation-on-mscoco)](https://paperswithcode.com/sota/real-time-instance-segmentation-on-mscoco?p=rtmdet-an-empirical-study-of-designing-real)
Expand All @@ -118,13 +165,6 @@ We are excited to announce our latest work on real-time object recognition tasks
<img src="https://user-images.githubusercontent.com/12907710/208044554-1e8de6b5-48d8-44e4-a7b5-75076c7ebb71.png"/>
</div>

**v3.1.0** was released in 30/6/2023:

- Supports tracking algorithms including multi-object tracking (MOT) algorithms SORT, DeepSORT, StrongSORT, OCSORT, ByteTrack, QDTrack, and video instance segmentation (VIS) algorithm MaskTrackRCNN, Mask2Former-VIS.
- Support [ViTDet](projects/ViTDet)
- Supports inference and evaluation of multimodal algorithms [GLIP](configs/glip) and [XDecoder](projects/XDecoder), and also supports datasets such as COCO semantic segmentation, COCO Caption, ADE20k general segmentation, and RefCOCO. GLIP fine-tuning will be supported in the future.
- Provides a [gradio demo](https://github.com/open-mmlab/mmdetection/blob/dev-3.x/projects/gradio_demo/README.md) for image type tasks of MMDetection, making it easy for users to experience.

## Installation

Please refer to [Installation](https://mmdetection.readthedocs.io/en/latest/get_started.html) for installation instructions.
Expand Down Expand Up @@ -236,9 +276,12 @@ Results and models are available in the [model zoo](docs/en/model_zoo.md).
<li><a href="configs/dab_detr">DAB-DETR (ICLR'2022)</a></li>
<li><a href="configs/dino">DINO (ICLR'2023)</a></li>
<li><a href="configs/glip">GLIP (CVPR'2022)</a></li>
<li><a href="configs/ddq">DDQ (CVPR'2023)</a></li>
<li><a href="projects/DiffusionDet">DiffusionDet (ArXiv'2023)</a></li>
<li><a href="projects/EfficientDet">EfficientDet (CVPR'2020)</a></li>
<li><a href="projects/ViTDet">ViTDet (ECCV'2022)</a></li>
<li><a href="projects/Detic">Detic (ECCV'2022)</a></li>
<li><a href="projects/CO-DETR">CO-DETR (ICCV'2023)</a></li>
</ul>
</td>
<td>
Expand All @@ -260,13 +303,15 @@ Results and models are available in the [model zoo](docs/en/model_zoo.md).
<li><a href="projects/SparseInst">SparseInst (CVPR'2022)</a></li>
<li><a href="configs/rtmdet">RTMDet (ArXiv'2022)</a></li>
<li><a href="configs/boxinst">BoxInst (CVPR'2021)</a></li>
<li><a href="projects/ConvNeXt-V2">ConvNeXt-V2 (Arxiv'2023)</a></li>
</ul>
</td>
<td>
<ul>
<li><a href="configs/panoptic_fpn">Panoptic FPN (CVPR'2019)</a></li>
<li><a href="configs/maskformer">MaskFormer (NeurIPS'2021)</a></li>
<li><a href="configs/mask2former">Mask2Former (ArXiv'2021)</a></li>
<li><a href="configs/XDecoder">XDecoder (CVPR'2023)</a></li>
</ul>
</td>
<td>
Expand Down
64 changes: 55 additions & 9 deletions README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
[![license](https://img.shields.io/github/license/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/blob/main/LICENSE)
[![open issues](https://isitmaintained.com/badge/open/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/issues)
[![issue resolution](https://isitmaintained.com/badge/resolution/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/issues)
[![Open in OpenXLab](https://cdn-static.openxlab.org.cn/app-center/openxlab_demo.svg)](https://openxlab.org.cn/apps?search=mmdet)

[📘使用文档](https://mmdetection.readthedocs.io/zh_CN/latest/) |
[🛠️安装教程](https://mmdetection.readthedocs.io/zh_CN/latest/get_started.html) |
Expand Down Expand Up @@ -101,6 +102,53 @@ MMDetection 是一个基于 PyTorch 的目标检测开源工具箱。它是 [Ope

### 亮点

**v3.2.0** 版本已经在 2023.10.12 发布:

**1. 检测 Transformer SOTA 模型大合集**
(1) 支持了 [DDQ](configs/ddq/README.md)[CO-DETR](projects/CO-DETR/README.md)[AlignDETR](projects/AlignDETR/README.md)[H-DINO](projects/HDINO/README.md) 4 个更新更强的 SOTA Transformer 模型
(2) 基于 CO-DETR, MMDet 中发布了 COCO 性能为 64.1 mAP 的模型
(3) DINO 等算法支持 AMP/Checkpoint/FrozenBN,可以有效降低显存

**2. [提供了全面的 CNN 和 Transformer 的性能对比](projects/RF100-Benchmark/README_zh-CN.md)**
RF100 是由 100 个现实收集的数据集组成,包括 7 个域,可以验证 DINO 等 Transformer 模型和 CNN 类算法在不同场景不同数据量下的性能差异。用户可以用这个 Benchmark 快速验证自己的算法在不同场景下的鲁棒性。

<div align=center>
<img src="https://github.com/open-mmlab/mmdetection/assets/17425982/86420903-36a8-410d-9251-4304b9704f7d"/>
</div>

**3. 支持了 [GLIP](configs/glip/README.md)[Grounding DINO](configs/grounding_dino/README.md) 微调,全网唯一支持 Grounding DINO 微调**
MMDet 中的 Grounding DINO 是全网唯一支持微调的算法库,且性能高于官方 1 个点,当然 GLIP 也比官方高。
我们还提供了详细的 Grounding DINO 在自定义数据集上训练评估的流程,欢迎大家试用。

| Model | Backbone | Style | COCO mAP | Official COCO mAP |
| :----------------: | :------: | :-------: | :--------: | :---------------: |
| Grounding DINO-T | Swin-T | Zero-shot | 48.5 | 48.4 |
| Grounding DINO-T | Swin-T | Finetune | 58.1(+0.9) | 57.2 |
| Grounding DINO-B | Swin-B | Zero-shot | 56.9 | 56.7 |
| Grounding DINO-B | Swin-B | Finetune | 59.7 | |
| Grounding DINO-R50 | R50 | Scratch | 48.9(+0.8) | 48.1 |

**4. 支持开放词汇检测算法 [Detic](projects/Detic_new/README.md) 并提供多数据集联合训练可能**

**5. 轻松使用 [FSDP 和 DeepSpeed 训练检测模型](projects/example_largemodel/README_zh-CN.md)**

| ID | AMP | GC of Backbone | GC of Encoder | FSDP | Peak Mem (GB) | Iter Time (s) |
| :-: | :-: | :------------: | :-----------: | :--: | :-----------: | :-----------: |
| 1 | | | | | 49 (A100) | 0.9 |
| 2 || | | | 39 (A100) | 1.2 |
| 3 | || | | 33 (A100) | 1.1 |
| 4 ||| | | 25 (A100) | 1.3 |
| 5 | ||| | 18 | 2.2 |
| 6 |||| | 13 | 1.6 |
| 7 | |||| 14 | 2.9 |
| 8 ||||| 8.5 | 2.4 |

**6. 支持了 [V3Det](configs/v3det/README.md) 1.3w+ 类别的超大词汇检测数据集**

<div align=center>
<img width=960 src="https://github.com/open-mmlab/mmdetection/assets/17425982/9c216387-02be-46e6-b0f2-b856f80f6d84"/>
</div>

我们很高兴向大家介绍我们在实时目标识别任务方面的最新成果 RTMDet,包含了一系列的全卷积单阶段检测模型。 RTMDet 不仅在从 tiny 到 extra-large 尺寸的目标检测模型上实现了最佳的参数量和精度的平衡,而且在实时实例分割和旋转目标检测任务上取得了最先进的成果。 更多细节请参阅[技术报告](https://arxiv.org/abs/2212.07784)。 预训练模型可以在[这里](configs/rtmdet)找到。

[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/rtmdet-an-empirical-study-of-designing-real/real-time-instance-segmentation-on-mscoco)](https://paperswithcode.com/sota/real-time-instance-segmentation-on-mscoco?p=rtmdet-an-empirical-study-of-designing-real)
Expand All @@ -117,13 +165,6 @@ MMDetection 是一个基于 PyTorch 的目标检测开源工具箱。它是 [Ope
<img src="https://user-images.githubusercontent.com/12907710/208044554-1e8de6b5-48d8-44e4-a7b5-75076c7ebb71.png"/>
</div>

**v3.1.0** 版本已经在 2023.6.30 发布:

- 支持 Tracking 类算法,包括多目标跟踪 MOT 算法 SORT、DeepSORT、StrongSORT、OCSORT、ByteTrack、QDTrack 和视频实例分割 VIS 算法 MaskTrackRCNN、Mask2Former-VIS。
- 支持 [ViTDet](projects/ViTDet)
- 支持多模态开放检测算法 [GLIP](configs/glip)[XDecoder](projects/XDecoder) 推理和评估,并同时支持了 COCO 语义分割、COCO Caption、ADE20k 通用分割、RefCOCO 等数据集。后续将支持 GLIP 微调
- 提供了包括 MMDetection 图片任务的 [gradio demo](https://github.com/open-mmlab/mmdetection/blob/dev-3.x/projects/gradio_demo/README.md),方便用户快速体验

## 安装

请参考[快速入门文档](https://mmdetection.readthedocs.io/zh_CN/latest/get_started.html)进行安装。
Expand Down Expand Up @@ -237,9 +278,12 @@ MMDetection 是一个基于 PyTorch 的目标检测开源工具箱。它是 [Ope
<li><a href="configs/dab_detr">DAB-DETR (ICLR'2022)</a></li>
<li><a href="configs/dino">DINO (ICLR'2023)</a></li>
<li><a href="configs/glip">GLIP (CVPR'2022)</a></li>
<li><a href="configs/ddq">DDQ (CVPR'2023)</a></li>
<li><a href="projects/DiffusionDet">DiffusionDet (ArXiv'2023)</a></li>
<li><a href="projects/EfficientDet">EfficientDet (CVPR'2020)</a></li>
<li><a href="projects/ViTDet">ViTDet (ECCV'2022)</a></li>
<li><a href="projects/Detic">Detic (ECCV'2022)</a></li>
<li><a href="projects/CO-DETR">CO-DETR (ICCV'2023)</a></li>
</ul>
</td>
<td>
Expand All @@ -261,13 +305,15 @@ MMDetection 是一个基于 PyTorch 的目标检测开源工具箱。它是 [Ope
<li><a href="projects/SparseInst">SparseInst (CVPR'2022)</a></li>
<li><a href="configs/rtmdet">RTMDet (ArXiv'2022)</a></li>
<li><a href="configs/boxinst">BoxInst (CVPR'2021)</a></li>
<li><a href="projects/ConvNeXt-V2">ConvNeXt-V2 (Arxiv'2023)</a></li>
</ul>
</td>
<td>
<ul>
<li><a href="configs/panoptic_fpn">Panoptic FPN (CVPR'2019)</a></li>
<li><a href="configs/maskformer">MaskFormer (NeurIPS'2021)</a></li>
<li><a href="configs/mask2former">Mask2Former (ArXiv'2021)</a></li>
<li><a href="configs/XDecoder">XDecoder (CVPR'2023)</a></li>
</ul>
</td>
<td>
Expand Down Expand Up @@ -443,10 +489,10 @@ MMDetection 是一款由来自不同高校和企业的研发人员共同参与

## 欢迎加入 OpenMMLab 社区

扫描下方的二维码可关注 OpenMMLab 团队的 [知乎官方账号](https://www.zhihu.com/people/openmmlab)加入 OpenMMLab 团队的 [官方交流 QQ 群](https://jq.qq.com/?_wv=1027&k=aCvMxdr3)
扫描下方的二维码可关注 OpenMMLab 团队的 [知乎官方账号](https://www.zhihu.com/people/openmmlab)扫描下方微信二维码添加喵喵好友,进入 MMDectection 微信交流社群。【加好友申请格式:研究方向+地区+学校/公司+姓名】

<div align="center">
<img src="resources/zhihu_qrcode.jpg" height="400" /> <img src="resources/qq_group_qrcode.jpg" height="400" />
<img src="resources/zhihu_qrcode.jpg" height="400" /> <img src="resources/miaomiao_qrcode.jpg" height="400" />
</div>

我们会在 OpenMMLab 社区为大家
Expand Down
Loading

0 comments on commit fe3f809

Please sign in to comment.